Minority Opinions

Not everyone can be mainstream, after all.

Archive for the ‘Python’ Category

Vague Python Ideas

leave a comment »

I recently spent some time perusing the python-ideas mailing list, particularly around the development of tulip, and a few ideas struck my fancy.

def __main__():

I’ve mentioned before that the standard scriptish prefix bugs me, but more recently I’ve noticed myself moving all logic out of an if __name__ == "__main__" block into a separate function.  Why not make that official?  The transition could be handled in a few steps:

  1. After loading the __main__ module, check to see whether it has a callable __main__ attribute.  If so, and if it hasn’t yet been called, call it.
  2. Give modules their real __name__ where appropriate.
  3. Remove the “hasn’t yet been called” hack.

I’m fine with parts of this process taking until Python 4, though I wouldn’t be surprised if the multiprocessing developers pushed it forward.

name := expression

A second assignment operator would be useful for while and elif conditions, but it was the discussion of a __bind__ operation that made it seem useful in its own right.  Basically, the statement above would translate into (expression).__bind__("name", locals()), where object.__bind__(self, name, namespace) would be implemented as namespace[name] = self.  (The translation doesn’t really have to call locals() to get the namespace, though.)

In general, it’s just a more expensive assignment statement.  The real power comes in letting an object know its canonical name.  The canonical examples here are namedtuple and ORM fields, which are usually passed the same names as they collect on assignment.

The object of the expression also discovers when it’s bound, and to what, either or both of which might be useful.  Something might want to raise an exception if it’s bound into an incompatible environment, for example, but considering that removing the colon bypasses that restriction, it might not be worth the code.

Finally, it can bind a new object in its place; in an example from the thread, NumPy array manipulation could use that signal to evaluate a chain of lazy operations.  Even more odd would be using it to unroll looped test cases, but it’s not likely to work well for functions.

Whether that power is worth the implementation cost, I’ll leave up to others.  In particular, I could be persuaded to implement it myself, but not on my own dime.  It may also have to fight against Greg’s secret plan.

Invisible async operations

The tulip discussion originally interested me as a potential way to move forward with a hobby project I’ve been neglecting.  At first, I had my own event loop, with support for calling tasks on network availability, at scheduled times, or in separate threads.  Eventually, I replaced it with Twisted, mostly to make it easier to support multiple protocols on the same port.  Since then, I’ve debated switching to something more like co-routines.  That such an event loop might become standard makes me happy.

The tulip project itself, though, highlights a design decision that might become a sticking point: yield from gets proliferated every time something could block.  Worse, some of the functions designed for iteration look just like functions you might call directly.  That makes me think that perhaps gevent has the right idea with its monkey patching:  Calls to certain socket interfaces are invisibly replaced with something that uses a magically created event loop under the hood.

Seriously, when would a coroutine yield control?  Mostly on network I/O, sometimes when sleeping either for a given duration or until a given time, occasionally on other blocking system calls, rarely while waiting for another thread to give it a result, and seldom explicitly just to be nice during a long calculation.  All of that blocking could be handled at a low level, by packing up the current frame as a closure and continuing it later.  Does it need to know that it’s going to be suspended?

Perhaps I’ve just been contaminated by having learned a bit of the Go language, but it would be nice to have it in the language I already know and love.


Written by eswald

19 Feb 2013 at 7:19 pm

Posted in Python, Technology

The Right Extraction

leave a comment »

At one point, I got annoyed with the sheer number of compression methods used to distribute source code and other packages, so I wrote a simple bash function to collapse the various incantations into a single word:

extract () {
  if [ -f $1 ] ; then
    case $1 in
      *.tar.bz2) tar xvjf $1 ;;
      *.tar.gz) tar xvzf $1 ;;
      *.bz2) bunzip2 $1 ;;
      *.rar) rar x $1 ;;
      *.gz) gunzip $1 ;;
      *.tar) tar xvf $1 ;;
      *.tbz2) tar xvjf $1 ;;
      *.tgz) tar xvzf $1 ;;
      *.zip) unzip $1 ;;
      *.Z) uncompress $1 ;;
      *.7z) 7z x $1 ;;
      *) echo "Don't know how to extract '$1'" ;;
    echo "'$1' is not a valid file!"

For nearly three years, this simple recipe sufficed.  I added lines for .jar and .tar.xz files, and wrapped the logic in a loop over each parameter, but that was it.  I intended to write bash completion for it, and perhaps honor a -q flag to decrease the verbosity, but it worked well enough.

Recently, however, I discovered dtrx, and it’s even better.  It handles even more compression types, it ensures that everything extracts to a single directory, it has a parameter to list the archive contents instead of extracting them, and it’s easier to type.  Most surprisingly, it handles the kind of .zip files that 7-zip creates, with a compression method that the standard unzip command can’t extract.  It requires Python, but that was barely an inconvenience even in a new Linux From Scratch environment.

It could still use a better bash completion method, though.  Maybe I’ll get around to that sometime.

Written by eswald

22 Jan 2013 at 7:00 pm

Posted in Linux, Python, Technology

Interfacing with Self

leave a comment »

Graham Dumpleton noticed a problem with simple WSGI middleware, and provided a couple of templates for fixing it.  He wasn’t particularly happy with either; one subtly defied the WSGI specification, and the other split state tracking across two classes.  I noticed a way to repair those issues, but my comment seems to have been swallowed; perhaps the comment system didn’t like the non-breaking spaces I used to ram something that looks like Python through the limited format options available, or maybe my typos caused it to be rejected by the author.  Fortunately, I had saved my code on Pastebin, so I can share it with you:

class ClosingMiddleware(object):
    def __init__(self, application):
        self.application = application
        self.iterable = None

    def __call__(self, environ, start_response):
        self.iterable = self.application(environ, start_response)
        return self

    def __iter__(self):
        for data in self.iterable:
            yield data

    def close(self):
        if hasattr(self.iterable, 'close'):

Read the rest of this entry »

Written by eswald

16 Oct 2012 at 6:00 pm

Posted in Python, Technology

Seamless Level Transitions

leave a comment »

My ideal roguelike has neither stairs nor perfectly rectangular levels.  Unfortunately, that makes level generation tricky; as Dwarf Fortress found, it’s less satisfying to depth without being able to go over and under your position.  Fortunately, I’ve found a way to walk between levels without using explicit stairs; at any given point, everything you can see looks like it could be on a single level.  This is the feature that lets Rainbow Rooms generate mazes of infinite size on a finite screen, without scrolling.

In fact, I found two ways to do so; the first made movement and field-of-view calculations simpler, but creates some annoying restrictions on level generation.  Given that level generation is a much harder problem than the first two, and will need to be created anew for any given game, I’m going to describe the second method here.

Read the rest of this entry »

Written by eswald

9 Oct 2012 at 6:31 pm

Posted in Python, Technology

New and Improved Rainbows

leave a comment »

So Rainbow Rooms didn’t do nearly as well as I would have liked.  One user went so far as to claim I should be disqualified for failing to follow the theme, and for doing no work of my own.  To be fair, half of the work I put in failed to make it into the version I submitted for judging, but I think a good portion of that critic’s problem was a failure to entirely understand the central mechanic.  At certain points, though not always on every early maze, the doors prevent you from going back.  Usually, this means you’re on the right track.

Unfortunately, version 1.0 had a problem where it could trap you in other places, as well.  It also could take forever to generate a new maze; after a certain point, the heat death of the universe might not be enough time.  Both of these issues are fixed in Rainbow Rooms 1.1, now available for download from either my personal server or from the PyWeek site.

Read the rest of this entry »

Written by eswald

2 Oct 2012 at 9:32 pm

Posted in Entertainment, Python

Content Generation

leave a comment »

PyWeek judging is in its second week, and I’m a little tense.  Granted, I have lingering tension from other sources, too, but it’s a bit unsettling to know that my work is being critiqued, and that I can’t fix the two major flaws I discovered just a little too late.  On the other hand, it has been fun to help judge the other entries, so the experience as a whole is probably a net positive for me.  I would do it again.

A select few of the games share one major disappointment: I wish they had more content.  Tee’s Walkway is a prime example; in essence, it’s a simple puzzle with some similarities to my own entry.  However, its levels are each created by hand, where mine are randomly generated on the spot.  Each approach has advantages.

To Walkway’s credit, each of its levels is beautiful.  Many of them are symmetric.  They have a nice progression of difficulty, coupled with relevant hints.  In contrast, my levels are chaotic, and often include inelegant portions, like a door that won’t get opened because the goal is right around the corner.  My mazes increase in size as you complete them, but that doesn’t always mean that each one is harder than the last; some of them have long corridors with few or pre-opened doors, zipping you onward with little effort.

However, I swiftly finished all fifteen Walkway levels, and wanted more.  Restarting can let me play the same levels again, finding new solutions for some of them, but the excitement has waned.  In contrast, I never know what Rainbow Rooms will throw at me unless I set a static seed for the random number generator.  I’ve probably played it more than anyone, and would pick it up again in a heartbeat.  My kids, too, keep asking to play it.

The difference may well be based on a difference of strengths.  Just as I’m not great at user interface design, I’m lousy at puzzle design.  It takes me less effort to write a level generator than to design a level by hand.  At that point, why would I select a static set of levels, instead of letting the user see something new each game?  Okay, perhaps I could have selected a set with a decent learning curve and no way to get trap yourself, and would have avoided the lengthy wait for the generator to find something it likes, but would I have played through it more than twice?

Of course, I also started out with a Roguelike idea that morphed into a puzzle game, so perhaps my values are slightly skewed.

Written by eswald

25 Sep 2012 at 6:06 pm

Posted in Entertainment, Python

Rainbow Rooms

leave a comment »

PyWeek is over, and I didn’t make a dent in the roguelike I had planned.  Instead, I realized while attempting to create a level generator that a simple maze game would be much better for the time allotted.  Fortunately, I was able to use some of my code and ideas from the roguelike exploration.

I ended up calling the new creation Rainbow Rooms, after the colored doors that occasionally block your path back to the start.  Almost all of Saturday was spent creating a level generator that wouldn’t trap you too often.  Much of the rest was spent playing it or watching my family play it.

Read the rest of this entry »

Written by eswald

18 Sep 2012 at 9:14 pm

Posted in Entertainment, Python