Minority Opinions

Not everyone can be mainstream, after all.

Parametric unittest cases

leave a comment »

I recently ran into a test case that was unexpectedly failing.  Worse, it wasn’t entirely obvious how, because the case in question contained a loop.  Cleaned up, the test case resembled:

def test_username_punctuation(self):
    for char in self.punctuation:
        self.reset()
        self.do_stuff(char)
        self.assertSomething()

Not all of the punctuation characters were failing, but it wasn’t trivial to see which ones were.  This is a case where nose’s generator tests might have come in handy, except that it’s in a TestCase class.  One that gets subclassed a few times, to ensure that certain API interfaces work identically.

It seems like people are considering support for this in unittest itself, but eventually doesn’t scratch my current itch.  Some minimal searching revealed a few related tools, but nothing simple to adapt.  Therefore, I coded my own:

from unittest import TestCase
from sys import stderr
from functools import update_wrapper
from itertools import product

class Parametric(object):
    r'''Paramaterized test case decorator.
    '''#"""#'''

    # Undescores are easier on test name collectors,
    # but hyphens, slashes, or commas make name clashes less likely.
    ARGSCHAR = "_"
    JOINCHAR = "_"

    def __init__(self, *args):
        self.params = args
        self.method = None
        self.case = None

    def __call__(self, method=None):
        self.method = method
        update_wrapper(self, method)
        return self

    def __get__(self, case, klass):
        if case:
            self.case = case
            return self.loop
        else:
            return self.method

    def cases(self):
        basename = self.method.__name__ + self.ARGSCHAR
        for items in product(*(self.items(arg) for arg in self.params)):
            name = basename + self.JOINCHAR.join(str(item[0]) for item in items)
            a = tuple(item[1] for item in items)
            yield name, self.create_case(name, a)

    def items(self, param):
        try:
            return param.items()
        except AttributeError:
            return enumerate(param)

    def loop(self):
        r'''Loops over the test case with each of its parameters.
            This allows the case to be run if the class decorator fails.
        '''#"""#'''
        first = True
        for name, runner in self.cases():
            if not first:
                self.case.tearDown()
                self.case.setUp()

            runner(self.case)
            first = False

    def create_case(self, name, args):
        # Yes, this needs to be defined outside of cases().
        def test_case(case):
            self.method(case, *args)
        update_wrapper(test_case, self.method)
        test_case.__name__ = name
        return test_case

    @classmethod
    def expand(klass, suite):
        r'''Expands each decorated test in the TestCase.
            Meant to be used as a class decorator.
        '''#"""#'''
        names = list(suite.__dict__)
        for name in names:
            item = suite.__dict__[name]
            if isinstance(item, klass):
                delattr(suite, name)
                for testname, runner in item.cases():
                    setattr(suite, testname, runner)
        return suite

@Parametric.expand
class ParamTests(TestCase):
    def setUp(self):
        stderr.write("[")

    def tearDown(self):
        stderr.write("]")

    @Parametric(range(5))
    def test_range(self, number):
        "Short string for the test case."
        stderr.write("#")
        self.assertTrue(number < 3)

    @Parametric({"one": 1, "two": 2, "three": 3, "four": 4})
    def test_dict(self, number):
        stderr.write("{}")
        self.assertTrue(number < 3)

    @Parametric("abcd")
    def test_string(self, letter):
        stderr.write('"')
        self.assertTrue(letter < "c")

    @Parametric("abcd", range(5))
    def test_two_args(self, letter, number):
        stderr.write("2")
        self.assertTrue(ord(letter) - ord("a")

The ParamTests class is simply for demonstration and example; I certainly wouldn’t advocate writing directly to stderr during test cases.  Mostly, I wanted to test the setUp() and tearDown() handling, which would allow me to eliminate the reset() method seen in the looped case.

This code isn’t perfect, by any means.  In particular, I would have liked to have collected the parameters from the argument list of the test itself, but that was too much work in Python 2.  I almost supported keyword arguments, but dropped that line when it started to make the code complicated.  The class decorator might work even better as a metaclass, particularly if it grants automatic expansion to subclasses, but that was also more work than I needed to use.  It doesn’t do anything special to docstrings, so any tests with one appear identical in the default output.

Very curiously, Parametric itself is more general than this use case; only loop() is specific to unittest.  I’m not sure what that could possibly buy you, though.

Advertisements

Written by eswald

26 Jun 2012 at 5:42 pm

Posted in Python, Technology

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s