Defgen: Just one more reason that Twisted freakin’ rocks

This isn’t brand-spanking new (it’s been in trunk for quite a while already), but Bob Ippolito made an update to it recently, so it caught my eye. I think this is such a cool, and incredibly useful, part of Twisted, that it’s worth mentioning just for anybody who hasn’t seen it yet.


from twisted.internet import reactor
from twisted.internet import defer

waitFor = defer.waitForDeferred

def getSomeDeferred():
    d = defer.Deferred()
    reactor.callLater(
        2, d.callback, 'This is a string that yells "foo!"')
    return d

def anotherDeferred(needle, haystack):
    d = defer.Deferred()
    reactor.callLater(
        2, d.callback, haystack.find(needle))
    return d

@defer.deferredGenerator
def find(needle):
    d = waitFor(getSomeDeferred())
    yield d
    haystack = d.getResult()
    print 'I got my first deferred result'
    d = waitFor(anotherDeferred(needle, haystack))
    yield d
    print 'I found', repr(needle), 'at character', d.getResult()
    return

find('foo!')
reactor.run()

So now you can write deferred procedures that look synchronous. None of this “writing many small statements, nested, backwards” nonsense. That’s awesome.

Updated: I noticed that the code I wrote didn’t really demonstrate the power of deferredGenerator, so I’ve updated the code. I still think this is really cool.

Posted in Code on April 28th, 2005.

4 responses to “Defgen: Just one more reason that Twisted freakin’ rocks”

  1. oubiwann said on April 28th, 2005 at 7:31 pm

    Holy crap, this is awesome. Had never seen it till now. This could quite literally revolutionize user acceptance of twisted, eliminating learning curve issues for many…

  2. r said on April 28th, 2005 at 8:35 pm

    after reading it a few times I can kind of see what this code does… It is not so pretty code.

    I guess this example is for people who allready understand twisted.

  3. mesozoic said on April 28th, 2005 at 11:31 pm

    You’re right, this could have been cleaner (notice the time of day it was posted). I’ve updated the code to show off deferredGenerator a little more.

    However, you do need some understanding of how Deferreds work to appreciate how much cleaner this is than the standard method: writing many small nested functions that call each other every time a deferred is passed around.

  4. Mary said on April 29th, 2005 at 7:52 pm

    Cool! I knew of defgen in an abstract way but with these examples it will have to go to the top of the ‘to document’ queue.

    As a side-note, one thing I always find most useful as a user of this kind of documentation/example code is lots of comments. The kind of comments that you’d never use in your own code, the equivalent of commenting an if statement to note that it is an if statement. So in the above, if it was part of my documentation, each line that involves a Deferred would have a comment saying “create a new Deferred” or whatever. You’d never comment your own code like that, but I really like it for examples.

Leave a Reply