Eventlet 0.7 fixes some very long-standing bugs. First of all, there was a CPU leak in the select hub which would cause an http keep-alive connection to consume 100% CPU while it was open. The problem was that every file descriptor was being passed in to select, even if the callback for the readiness mode was None. This bug has been in since the very beginning of eventlet, and it's great to have it fixed!

Second, another old bug. It's now possible to use Eventlet's SSL client to talk to Eventlet's SSL server. There was a subtle bug in the way SSL sockets would raise an error in some conditions instead of returning '' to indicate the connection was closed.

Finally, some memory leaks in the libevent and libev hubs (fairly new code) were fixed, so if you're using Eventlet with libevent or libev try it out and see how it performs for you.

Also, this release pulls in a bunch of API additions from the Linden SVN repository. Ryan Williams is now maintaining an HG repository which is synched with the SVN repository, so integrating patches between branches will now be much easier.

Update July 30, 2008This release of eventlet also supports stackless-pypy again. I had to check for the absence of the socket.ssl object, and re-enable the poll hub. To try this out, check out and translate pypy-c following the instructions on the pypy site, and then run one of the eventlet examples (for example, "./pypy-c /Users/donovan/src/eventlet/examples/wsgi.py")

Download Eventlet 0.7 from PyPI: http://pypi.python.org/pypi/eventlet/0.7

Spawning 0.7

Spawning has improved a lot since I last wrote about it. It now has a command line script, "spawn", which makes it easy to quickly serve any wsgi application. The concurrency strategy is also now extremely flexible and can be configured for a plethora of use cases.

The default is to use one non-blocking i/o process with a threadpool, which makes it easy to use with any existing wsgi applications out there that assume shared memory and the ability to block.

However, it's possible to independently configure the number of i/o processes, the number of threads, and even configure it to be single-process, single-thread, with fully non-blocking i/o (thanks to eventlet's monkey patching abilities).

Update July 30, 2008This release of spawning also has an experimental Django factory. To run a Django app under Spawning, run "spawn --factory=spawning.django_factory.config_factory mysite.settings".

Take a look at the Spawning PyPI entry for more information: http://pypi.python.org/pypi/Spawning/0.7

This is very early still. The code is currently hard-coded to run one process, but once I figure out how to use Paste Deploy's configuration files a bit better I will make it configurable. I mostly wanted to get it out quickly because Ian Bicking asked for it in the comments of my last blog post, and to get feedback. I'd like more of this code to be shared between Spawning and mulib's 'mud' server. I also need a better name than Spawning.

You can download a tarball here or you can clone the Mercurial repository here.

The main change in this release is the ability to use libevent as the multiplexing api instead of raw select or poll. If libevent and the Python wrapping are not installed, eventlet will still fall back, first checking for the presence of poll and falling back to select if it is not available.

Another major change in this release is a much improved eventlet.wsgi server. The wsgi server now supports Transfer-Coding: chunked as well as Expect: 100 Continue, and is quite fast. I tested it against an eventlet based wsgi server I wrote which uses wsgiref (from the Python 2.5 standard library) and my informal tests showed eventlet.wsgi being several hundred requests a second faster at serving a "Hello, World!" wsgi application.

This release also features significant refactoring, cleaner code, support for cooperative operations on pipes (and unix domain sockets) as well as sockets, more tests, and docstrings for pretty much everything. The documentation, which was non-existant before, is now pretty comprehensive.

To install, just "easy_install eventlet" and start hacking!

• PyPI page: http://pypi.python.org/pypi/eventlet/0.5
• Overview: http://wiki.secondlife.com/wiki/Eventlet
• Documentation: http://wiki.secondlife.com/wiki/Eventlet/Documentation
• Mercurial Repository: http://donovanpreston.com:8888/eventlet

Ryan Williams wrote the current eventlet Actor (eventlet.coros.Actor) and it's much simpler and more straightforward: You override the received method to handle messages, and other actors call the cast method to send messages. This is different from my previous implementation (and also what my ideal would be) in that you get called back for every message, meaning the main coroutine is generic and there's no need to keep track of where the Actor's coroutine is to serialize an actor. This means it would be possible to request a representation of an Actor at any time between messages. The state would include all the Python instance variables along with all the unhandled messages currently in the Actor's mailbox.

So, with that realization, it suddenly becomes trivial to write a mulib handler for the Actor class. GET and PUT with the appropriate content types (application/json for example) would get or set the current state of the Actor. DELETE would delete it. POST enqueues a message in the actor's mailbox (it just calls cast with the body of the request). Simple and straightforward. I'm totally going to do this soon -- it probably would have been faster to just do the implementation rather than blog about it :-)

Oh, one more thing -- to enhance the experience of actually using these semantics, the cast method should become a generic method that dispatches based on pattern matching (using mulib.shaped). I haven't figured out what an efficient implementation of this would look like yet, but I'm going to try a brute-force implementation just for fun.

I've had a project on the back burner for a while that involves pushing data into LSL (the Linden Scripting Language runtime that Second Life uses) over Comet.

And it works amazingly well! Of course, I just had to toss REST in there as well.

I'll release this code as open source next week. Meanwhile, here's a movie to show you what the hell I am talking about. I also like how this movie shows off the surreal aspect of collaborative programming!

http://twitter.com/donovanpreston

Picked this up at Kid Robot in the Haight this weekend. It is so freaking awesome. I want more, but can't read japanese :-)

Recently I was tasked with discovering whether it is possible to do non-blocking file reads and writes to a filesystem that is mounted over NFS. I tried on OS X, and I was unable to get a read to return EWOULDBLOCK. So, I decided to install Ubuntu on Parallels. I downloaded the iso, burned it to a CD, created a new virtual machine, and installed it.

Everything worked flawlessly. Ubuntu has always been incredibly high quality, and it has only gotten nicer in the year since I used it last. It's polished, beautiful, and just works. It is definitely something that I could install on my machine for my Mom with a web browser and mail reader.

Here's a screenshot:

import readline
readline.write_history_file('my_history.py')

I'm sure this is going to come in handy many times.

Very busy at work lately. Getting a lot accomplished, and it is challenging, fun work. We're starting to ramp up pretty fast and I have been doing scalability work on the backend, which is surprisingly enjoyable after doing web front end work for so long.

I'm at SuperHappyDevHouse VII tonight. I went to II, and had a good time, but haven't been able to get back until now for various reasons. It was supposed to be last weekend, which falls on my weekend I am scheduled to work in the city, but for some reason it got bumped to this weekend. My goal for tonight is to release some code which was written almost a year ago: NevowPavel. If you have a short attention span, watch the screencast, wherein I explain the basic idea behind the project and give a demonstration of the currently implemented features.

This is the secret LivePage project I wrote about the last time I went to SHDH. It is a multiuser real-time updating wiki similar to and inspired by (the precursor to) Jot Live, the "live" version of Jot that broadcasts real-time updates to everyone that is participating. The difference with the NevowPavel implementation is that it is "spatial". Instead of editing a contiguous page of text, each piece of text is contained in a sticky note which can be moved and resized at will to organize the information as one wishes.

Links are also handled in an interesting way. Links are inserted by dragging out a link instead of a sticky and typing a new page name. Following the link takes you to the new page, where you can create and organize new pieces of information. Each page has a user list, so you can see who is there editing with you. To organize information cross-page, you drag stickies onto links. People who are on other pages see the information appear magically.

The ultimate goal of these template objects which you can spatially place in the page is for them to be pluggable. New types of objects should be able to be created which can provide code which renders content and handles input events for instances placed in the page. Eventually, the code should be through-the-web programmable. Obviously this involves lots of different things, like run-time code modification, restricted code execution, perhaps distributed code (to move code across multiple Pavel servers), and some sort of persistence. I'm also working on a Prototype-based object system with very simple persistence which is coming along well, but isn't integrated with the old project at all yet. This is a multi-year project, so check back again next year :-)

Get the latest version of Flash to view Slide Show, above. Visit Slide.com to create your own Slide Show

I have no idea how I came across your site. Probably after reading the wikipedia Chiptunes page and searching around a bit (since you don't seem to be linked on there... go add yourself!). I had the contents of your Chiptunes page on my ipod for a while before I really listened to it a lot. The first song I downloaded and listened to a lot was the thriller cover, which I thought was just so cheesy to be brilliant. I used it to test a Mac audio editing application I was working on this summer, SoundFarmer. Check it out:

http://donovanpreston.slide.com/c/SoundFarmer

Specifically:

http://item.slide.com/y/uid=5eilEp2bBu3zp7fdnfWThvdchdJNe1P50VhphDVUTV35s_UPnkCLsZB6WKutK_7Y/SoundFarmerAugust05.png

I didn't really listen to much of the other stuff from the chiptunes page or thriller for a while. Then, I was at the gym yesterday on the treadmill, sorta in the zone, just closing my eyes and concentrating fully on what I was hearing. I think it was in the middle of "blastoff", right about 1:08, where the progression slams right through the fucking roof and the chorus kicks in and breaks it down and then the next section starts slicing microthin shavings through your brain with the distorted guitar and then here we go up and up and up and arpeggio and staccato and tension hold and release and OH FUCK YEAH bring it back around to the chorus again!!! I was almost in tears. I actually had chills. Goosebumps. (All while sweating my ass off at the gym!) I am not shitting you.

Anyway, I went and downloaded every other single mp3 I could find on your site earlier this morning (say hi to your server logs for me ;-) and was digging the gameboy choons but it wasn't until I got to MC Nachbar that I realized that you are a fucking GENIUS. GEE-NEE-USSSSSS. A fucking polymath. So I wanted to mail you and let you know. You made my year.

Nostalgia and speculation on the nature of human creativity follows:

When I was in highschool, I found a copy of SoundTrecker (a Mac mod tracker) and a few mods somewhere. Pretty soon I was downloading mods from boards as fast as I could. At one point I had an entire syquest 44 cart filled up with em. Pretty amazing over a 9600 baud modem! I remember when I eventually had a 28.8 and I could download mods faster than I could listen to them... that was quite an epiphany (I had the same epiphany when I discovered mp3 ftp servers a few years later when I worked at the state of michigan, which had a T3. I have not yet had that epiphany with movie torrents yet, but I have a feeling we're close. I also don't really like video as a media). Then when I was in college I found my favorite song ever... FOOP.MOD, a brilliant breakbeat choon that I still love to this day. I was also buying CDs at the time from various random techno acts of the day, but with the exception of Aphex Twin, Luke Vibert, and Mike Paradinas, I was continually disappointed. There was just something about the raw emotion and uncut feel of mods that compelled me much more than some random crappy-ass major label release.

There is a psychic difference between the product that one creates for love versus money. When someone really, truly loves something and does their absolute best, it bleeds out between the lines, oozes all over your hands and sticks to you and won't wash off. I work at slide because I love programming, and the fact that I get paid is a pleasurable side effect. It is this raw hot plasma passion that fuels the internet, the blogs-versus-journalists and mp3-versus-record-mafia battles. The 9 to 5 work week and the boss-employee fiction is a recent creation in human evolution. The creative drive of which I speak is not. It may not happen tomorrow, but the politcal and social fads of the day will fade. They will be replaced with new social customs and norms. Each step of the way humanity will delude itself into thinking this custom is "natural" and has somehow always existed. But the only thing that has always existed is the creative big bang. God. The Universe. Order from chaos, strange attractors in the cellular automata. Find it, feel it, follow it, hold on.

So yeah. Thank you for who you are and for letting me see it through the magic of the internet.

Recently, I changed jobs. I joined a company called slide, which is building a hybrid photo sharing application using a combination of fat client and rich web technologies. In my drive to create responsive web applications I have been exploring techniques which make heavier and heavier use of client-side JavaScript. I now find myself doing less and less Python and more and more JavaScript rendering code.

I don't really mind this too much. I think JavaScript is a reasonable language, with reasonable anonymous function syntax, real closures with mutable parent scopes, and prototype inheritance (with the most bizarre implementation that unfortunately renders it almost unusable). However, every once in a while I hit some issue which drives me absolutely insane trying to debug.

Here is what I ran in to. The following:

{'foo': 1}

Is not valid syntax. However, the following:

foo = {'foo': 1}

Works fine. I ran in to this issue while trying to eval constructed strings of javascript. Even though eval returns a result, giving it a single JavaScript object literal results in an exception. Tacking on an assignment at the beginning fixes the issue. Ugly.