EssentialCode

Automated Versioned Deployments For Flask (Or Any Python) Webapps

Working at VendAsta during the day, all of the development I do is for Google App Engine. For my personal projects, however, I have been developing with the Flask framework lately. Discussing the pros and cons of App Engine could fill a whole blog unto itself but the one feature I definitely miss when developing non-App Engine apps is the ease of deployment. With an App Engine app you increment a version number in your app.yaml file, call the update script and your application’s new version is uploaded to Google’s infrastructure. After that, you make the new version the default one to serve and you’re done. Even data schema changes are handled for you (though you still might need a Map-Reduce job to migrate data values). All of this can easily be rolled into a Continuous Integration system like TeamCity or Jenkins and that is exactly what we’ve done at VendAsta (with TeamCity).

In the non-App Engine world, things are a lot trickier and frankly, less fun. In this post I will detail how I have pulled together something almost as fun and easy as App Engine deployments for Flask apps. The secret sauce is Fabric, a python library/command-line tool for deployments and systems administration. Essentially, Fabric acts as a wrapper around the shell and, more importantly, ssh. I started out using it just for the deployment portion of my pet projects but have since used it to completely replace the ANT scripts I had been using previously. It is simpler, more powerful and doesn’t require you to use anything besides python to manage your targets. While I am using this with Flask apps specifically, there is nothing in this example which couldn’t apply to any other framework you happen to prefer using.

Let’s jump right into things with the actual deployment. Listing 1 contains two functions, one to pack up the files of your app (pack) and one to upload the files to the server (upload). Have a look at it and then I will step through the various pieces.

Listing 1: Pack and upload methods complete.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

# python imports
from __future__ import with_statement
import os, re
 
# fabric imports
from fabric.api import local, run, cd, put, env
 
env.host_string = "user@someserver.org"
env.password = 's3kr3tz'
 
def pack(target, version, revision, release_dir, tmp_dir='/tmp'):
    """
    Method to package a release for deployment
    """
    archive_name = '.'.join([target, version, revision, 'tgz'])
    full_archive_path = os.path.join(tmp_dir, archive_name)
    local('tar czf %s -C %s .' % (full_archive_path, release_dir))
    return archive_name
 
def upload(target, version, revision, release_dir):
    """
    Method to upload the released code to a server
    """
 
    # package the release for deployment
    tmp_dir = '/tmp'
    archive_name = pack(target, version, revision, release_dir, tmp_dir)
 
    with cd('public_html'):
        dir = run('ls')
        dir_contents = [d.strip() for d in re.split(r"\s*",dir) if d.strip()]
        if not target in dir_contents:
            run('mkdir %s' % target)
 
        with cd(target):
            dir = run('ls')
            dir_contents = [d.strip() for d in re.split(r"\s*",dir) if d.strip()]
 
            if not version in dir_contents:
                run('mkdir %s' % version)
 
            with cd(version):
                run('rm -rf *')
                put(os.path.join(tmp_dir, archive_name), '.')
                run('tar xzf %s' % archive_name)
                run('rm -f %s' % archive_name)
                run('mv fcgi.htaccess .htaccess')
                run('chmod 755 index.fcgi')
 
            run('rm -f current')
            run('ln -s %s current' % version)

First, let’s look at the pack(target, version, revision, release_dir, tmp_dir='/tmp') method, since it is relatively straight-forward. The arguments themselves are really a matter of personal preference here. I have chosen to build the archive name of the code I am releasing from the target I am deploying to (typically this would be ‘test’, ‘demo’ or ‘prod’ as needed), the version I am deploying (for test deployments I usually deploy to a version of ‘daily’ which gets overwritten with each subsequent release and for demo and prod I use a loose interpretation of Semantic Versioning with a <major>.<minor>.<patch> scheme) and the revision of the latest commit in the release. The release_dir argument specifies where the code to be packaged up resides and tmp_dir is simply the location of a temporary directory. Lines 15 and 16 of Listing 1 show the first three arguments being assembled, with the temp directory, into an archive filename. There are more sophisticated ways to package things up for distribution (such as Distribute which is given some attention on the Flask site) but I just prefer the simplicity of gzipping everything up and then sshing the file onto the server.

Line 17 of listing one utilizes the Fabric method local() which makes it easy to run shell commands from within your python code. Here, I am simply calling tar to archive the specified release directory into an archive with the filename I just put together. Finally, the pack method returns the filename of the newly created archive.

The pack method is all well and good, but all it has really done is saved you from having to write a little ANT code. There is nothing in there that couldn’t have been done very easily from a traditional build script. The upload method is where the real power of Fabric starts to shine. The arguments to this method are identical to those of pack with the exception of the temp directory which is defined in this method. More properly, this should be defined in a properties file (the subject of another post for another day) but for this example it is fine here. Secondly, upload just calls pack to package up the deliverables and return the name of the archive. Listing 2 shows the first piece of upload that is actually interesting.

Listing 2: Preparing the target directory

1
2
3
4
5

    with cd('public_html'):
        dir = run('ls')
        dir_contents = [d.strip() for d in re.split(r"\s*",dir) if d.strip()]
        if not target in dir_contents:
            run('mkdir %s' % target)

Line 1 of Listing 2 introduces the Fabric method cd which does exactly what its counterpart in your shell does. This version, however, does the change directory over an ssh connection. Two important lines in Listing 1 are repeated here in Listing 3:

Listing 3: SSH authentication

1
2

env.host_string = "user@someserver.org"
env.password = 's3kr3tz'

These two lines specify the user and server to make an SSH connection to and the password to use for that user’s login. For a production environment using key-based authentication is recommended if not required in order to keep everything properly secured. These lines set values in Fabric’s environment that it then uses to connect to the remote server for the remainder of the operations we have ahead.

Jumping back to Listing 2, we can see that the script changes into the directory public_html. This will be the root of your deployments, whatever that path might be. This is another value that should more properly be placed in a properties file. Utilizing the python with statement, lines 2, 3, 4 and 5 will all execute within the context of the public_html directory. Line 2 performs an ls in the public_html directory and line 3 parses the result into a list of names. Line 4 then checks if we have a directory in public_html which matches the name of the target we are deploying and if not, line 5 creates it. As an example, if this was our first time deploying a demo build, this would create a demo directory within public_html.

Listing 4 shows the next block, which does a similar operation only this time within the target directory, checking for and creating a version subdirectory.

Listing 4: Creating a directory for the version being deployed

1
2
3
4
5
6

        with cd(target):
            dir = run('ls')
            dir_contents = [d.strip() for d in re.split(r"\s*",dir) if d.strip()]
 
            if not version in dir_contents:
                run('mkdir %s' % version)

So, to continue our example, if we are deploying version 1.0.1 to demo, then we would now have the path public_html/demo/1.0.1/ created. Finally, listing 5 contains the actual deployment of code to the server!

Listing 5: Deploying the new version of the code

1
2
3
4
5
6
7

            with cd(version):
                run('rm -rf *')
                put(os.path.join(tmp_dir, archive_name), '.')
                run('tar xzf %s' % archive_name)
                run('rm -f %s' % archive_name)
                run('mv fcgi.htaccess .htaccess')
                run('chmod 755 index.fcgi')

Line 1 of Listing 5 should be familiar now, it is a with statement to perform the following operations within our version directory. Line 2 removes any files and directories that might already exist there. This allows for deploying the same version over and over for daily builds from a Continuous Integration platform. Line 3 does the actual transfer of files utilizing the Fabric put() method to send our archive file to the server. Line 4 extracts the files from the archive and line 5 removes the archive afterwards. Lines 6 and 7 are needed for my webhost (HostGator) with 6 renaming the deployed .htaccess file needed to have FastCGI function properly and 7 sets the permissions as needed for the .fcgi entry script.

Last but not least we have the two simple lines shown in Listing 6.

Listing 6: Serving up the correct version

1
2

            run('rm -f current')
            run('ln -s %s current' % version)

Now that the script is ready, you can deploy your code to the server using the fab shell command. For a real project you’d likely have a deploy method which does all of your prep work and then calls the upload method above. For now, however, you could call upload directly, adding the target, version, revision and release_dir arguments. Arguments to fab can be done just like calling the python methods themselves, either with position arguments, named arguments or a combination. Listing 7 shows some example calls to deploy your code.

Listing 7: Deploying with fab

1
2
3
4
5
6
7
8

# Positional arguments
fab upload:demo,1.0.1,0264103894,/home/jread/project
 
# Named arguments
fab upload:target=demo,version=1.0.1,revision=0264103894,release_dir=/home/jread/project
 
# combination
fab upload:demo,1.0.1,revision=0264103894,release_dir=/home/jread/project

It is all well and good that we now have the new version of our code deployed to the server, but it still needs to be available via your webserver. You can’t very well have your users change all of their URLs to point to the new version subdirectory, so you need to add a little magic. In this case, I have pointed my webserver to use public_html/demo/current as the webroot for the demo version of my app. The two lines in Listing 6 remove an existing current and then create a new one which is a symlink to the newly deployed version directory. No server configurations need to be changed and certainly no DNS needs changing. If you discover a nasty bug in a new deployment, you can simply shell to your server and remake the symlink to the previous version’s directory and you have an instant rollback of the release.

The possibilities of this technique are actually quite limitless. Deployments could be kicked off from certain commits to your source repo. It would be fairly trivial to implement a small dashboard which gives you manual control over the symlink current allowing for a web interface to set the currently active version, just like in the Google App Engine admin.

It is worth noting that the entire script as presented in Listing 1 is only a small fraction of the full deployment script that I am using for my real projects. In fact, I have everything split into several different files for easier organization. I will be adding future posts on this subject covering things like automated versioning, resource versioning, compilation of Coffeescript or LESS files, preparing files for release and the far more complicated problem of data schema deployment.

Update 2012-01-29: Added example of actually calling Fabric to deploy code.

“The last packet sent” MySQL Error Installing Jira

For tracking issues and bugs in my side work for Pinch Software, I use Jira by Atlassian. They offer a wonderful set of Starter Licenses which provide everything a small startup or freelancer would need to manage Agile software development.

Tonight I was planning on upgrading to the latest Jira release when I realized that I had never hooked the existing install up to MySQL. In light of that I decided to just do a fresh installation. During the course of this installation I ran into an error I had yet to encounter before:

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

After a good deal of poking around and Googling, I found and fixed the problem. You see, the default setup for MySQL and Jira is (if you follow Atlassian’s excellent how-tos) to connect to the MySQL server running on localhost. This works great most of the time. However, if you are like me and want your MySQL to be remote manageable from another machine on your network, you may have configured MySQL to listen on an IP (e.g. 192.168.0.40) instead of just localhost. The solution is actually quite simple.

Where the Atlassian docs say to put

url=”jdbc:mysql://localhost/jiradb?useUnicode=true&characterEncoding=UTF8″

into your server.xml file, you’ll actually want to specify the IP address you have MySQL listening on. So, if MySQL is listening on 192.168.0.40, you’d put

url=”jdbc:mysql://192.168.0.40/jiradb?useUnicode=true&characterEncoding=UTF8″

into the server.xml file.

Hopefully this will help anyone out there who comes across this error. As an aside, if you are interested in trying out Jira (and the rest of Atlassian’s excellent tools), I highly recommend taking on their Dragon Quest. You’ll get a tonne of excellent experience setting up their suite of tools, a working setup of said tools, and if you send them a screenshot when done, you’ll also get a free T-shirt!

Running JSLint in Automated Build Scripts

At VendAsta we use automated build scripts and a TeamCity server for continuous integration. Whenever someone commits code to a project, a whole suite of automated build targets are run against the code base to check things like unit tests, python syntax (via pylint) and code coverage. One thing that was missing from the mix was the automated running of jslint to check the syntax of our ever-increasing corpus of JavaScript code. As it turns out, adding this to our ANT targets wasn’t all that difficult.

The only real difficulty you are likely to encounter in creating a JSLint target for your build scripts is that JSLint is, itself, written in JavaScript. Other helpful libraries such as JSMin or the YUICompressor from Yahoo are distributed as jar files and can be executed easily with an exec block with the executable set to java. With JSLint, however, you are going to need a JavaScript engine which you can execute from the command line in order to run the JSLint script to validate your JavaScript code. THis actually sounds harder than it is.

For our implementation, I chose to use the Rhino engine since there is documented support for JSLint on it, and even a handy helper file to include in your system: rhino.js. Grab a copy of Rhino (the actual file you’ll want to include is js.jar), the Rhino helper script I just mentioned and JSLint itself. Place them all together into some sort of tools directory in your project. Ours is located in /tools/ant.

Now comes the fun part, writing the actual target in ANT. For our purposes, we want to process all .js files contained in a particular location, as defined in a build property called jsdir. We also want to ignore any files that are contained in a vendor subdirectory of jsdir, as we should not need to worry about third-party scripts that don’t pass JSLint. Let’s get things started with our basic, empty target:

1
2
3
4
5
6

<!--
jslint - Runs lint checks on js files
-->
<target name="jslint">
 
</target>

Listing 1: The basic, empty jslint target

Within the target there are two things we are going to need. First a way to reference all of the files we want to operate over and second, the actual call to run JSLint on those files. For the first requirement we’ll use a pathconvert tag with a nested fileset. The pathconvert will allow us to generate a space-separated list of files and stuff that into a property which we’ll call jsfiles. The fileset block will do the actual work of finding the files in question. It will operate over ${jsdir} including all .js files and excluding all .js files in the vendor subdirectory. Listing 2 shows the pathconvert block added to our empty target.

1
2
3
4
5
6
7
8
9
10
11

<!--
jslint - Runs lint checks on js files
-->
<target name="jslint">
    <pathconvert pathsep=" " property="jsfiles">
        <fileset dir="${jsdir}">
            <include name="**/*.js"/>
            <exclude name="vendor/**/*.js"/>
        </fileset>
    </pathconvert>
</target>

Listing 2: The pathconvert block added to our target.

Finally, it is time to add the actual call to JSLint. To allow our build scripts to work for multiple projects, we have pushed as much information into properties that get defined in project-specific files as possible. The location of the Rhino jar (js.jar) is placed into the property js.jar and the location of the JSLint script (jslint.js) is placed into the property jslint.js. With those properties suitably defined elsewhere, we have the final piece of our puzzle: the exec block. We have the executable set to java and failonerror set to true, so that our target will appropriately fail when JSLint fails to give our code a passing grade. The final target, including the exec block can be seen in listing 3.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

<!--
jslint - Runs lint checks on js files
-->
<target name="jslint">
    <pathconvert pathsep=" " property="jsfiles">
        <fileset dir="${jsdir}">
            <include name="**/*.js"/>
            <exclude name="vendor/**/*.js"/>
        </fileset>
    </pathconvert>
    <exec dir="${jsdir}" executable="java" failonerror="true">
        <arg line="-jar ${js.jar} ${jslint.js} ${jsfiles}"/>
    </exec>
</target>

Listing 3: The final target.

Put all together, the pathconvert block stuffs the list of files to work on into the property ${jsfiles} which is then passed as an argument to the jslint.js file. The jslint.js file is itself the first argument passed to the Rhino JavaScript engine (js.jar) which is run directly by java. Running your ant jslint target should give you some nice feedback on the syntax of your JavaScript code.

Splitting Django Models Into Separate Files

A little while ago I wrote a post about getting Django unittests to work on DreamHost. The particular problem with the DreamHost setup is that they don’t allow you to programmatically create and drop databases and the default Django unittest behaviour is to do just that: create a brand new test db, use it and then drop it. Since I wrote that post I have begun migrating all of my domains away from DreamHost but I have still found that technique to be useful with new projects.

I first started using Django when I started at VendAsta and this issue didn’t creep up there because we are use Google App Engine and its DataStore on the backend instead of a traditional database. With no database there was no issue with creating/dropping anything. It was only when I started using Django on my own that I came across that problem. There was one more issue with Django that I came across only when I started doing things outside of work: Django expects all of your models to be in a single file called models.py. I hate that. A lot.

I did some googling about the problem and found that there were a bunch of other people who didn’t like it much either. There were a few solutions but they all involved manually mucking about with the __all__ attribute in the __init__.py file of your models directory. Sure, it works, but I don’t like having to define a model in two places to use it. Oddly enough it was in some test code at work that I found the answer.

For our project at VendAsta we are using the Django App Engine Helper to provide a nice easy interface to the DataStore on the backend. A part of this is a small __init__.py file which allows you to spread your tests out in multiple files. Django likes its tests to be in a single tests.py file just like it likes its models to be in models.py. The wee bit of Google code is shown in Listing 1.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

#!/usr/bin/python2.4
#
# Copyright 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
 
 
 
 
"""Loads all the _test.py files into the top level of the package.
 
This file is a hack around the fact that Django expects the tests "module" to
be a single tests.py file and cannot handle a tests package inside an
application.
 
All _test.py files inside this package are imported and any classes derived
from unittest.TestCase are then referenced from this file itself so that they
appear at the top level of the tests "module" that Django will import.
"""
 
import os
import re
import types
import unittest
 
PACKAGE = 'appname.tests'
TEST_RE = r"^.*_test.py$"
 
# Search through every file inside this package.
test_names = []
test_dir = os.path.dirname( __file__)
for filename in os.listdir(test_dir):
  if not re.match(TEST_RE, filename):
    continue
  # Import the test file and find all TestClass clases inside it.
  test_module = __import__('%s.%s' % (PACKAGE, filename[:-3]),
                           {}, {},
                           filename[:-3])
  for name in dir(test_module):
    item = getattr(test_module, name)
    if not (isinstance(item, (type, types.ClassType)) and
            issubclass(item, unittest.TestCase)):
      continue
    # Found a test, bring into the module namespace.
    exec "%s = item" % name
    test_names.append(name)
 
# Hide everything other than the test cases from other modules.
__all__ = test_names

Listing 1: __init__.py file from Google for a tests module

Essentially what this file does is loop through all files in the same directory and for each on that matches the pattern listed on line 40 (in this case, files that end in ‘_test.py’) and import all of the Classes from those files that are subclasses of the TestCase class. Finally, the code on line 62 does the magic with __all__ that I wanted to avoid doing manually for my models. It was this line that caught my attention and started me wondering if this code could be used for spreading models across multiple files.

In order for this to work for models there are a few things that need to be changed. First of all, on line 40, the pattern should be changed to match on *.py to grab all of the python files in the directory. Then, on line 46, add

or filename == “__init__.py”

to ensure that we don’t try to import the file we are in. Finally, remove the subclass requirement from line 55, as we are just concerned with importing classes here not what they inherit from. Putting it all together results in Listing 2. Place that in the __init__.py of a directory called models and you’ll not have to manually manage the __all__ list for your models in separate files.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

#!/usr/bin/python2.4
 
import os
import re
import types
import unittest
 
PACKAGE = 'my_app.models'
MODEL_RE = r"^.*.py$"
 
# Search through every file inside this package.
model_names = []
model_dir = os.path.dirname( __file__)
for filename in os.listdir(model_dir):
  if not re.match(MODEL_RE, filename) or filename == "__init__.py":
    continue
  # Import the model file and find all clases inside it.
  model_module = __import__('%s.%s' % (PACKAGE, filename[:-3]),
                           {}, {},
                           filename[:-3])
  for name in dir(model_module):
    item = getattr(model_module, name)
    if not isinstance(item, (type, types.ClassType)):
      continue
    # Found a model, bring into the module namespace.
    exec "%s = item" % name
    model_names.append(name)
 
# Hide everything other than the classes from other modules.
__all__ = model_names

Listing 2: __init__.py file modified for a models module

Now, there is one last thing that you are going to need to do in order for everything to run properly in Django. It is a very simple addition to the end of each of your classes. I have seem some people complaining about the need to do this in various forums, but honestly, if you are using any form of modern text editing application, it should only take you about 2 minutes to create a new template for a model that has the little piece of code shown in Listing 3 at the bottom of the class (replace your_app_name with your actual app name).

1
2
3

class Meta:
        """Meta Class for your model."""
        app_label = 'your_app_name'

Listing 3: Meta class required at the end of each model for proper functioning.

Django Unittesting on Dreamhost

Unit testing. It’s important. We all know that, even if we don’t practice what we preach. There was zero policy on unit testing at my last job, but now that I have started working at VendAsta, there is plenty of support for proper testing at the highest levels. Now that I am getting back into the habit of writing unit tests for my code at work I have been starting to focus on proper testing for my code at home. Combining Django and hosting with Dreamhost has proven to be a bit of a challenge.

The first hurdle is actually getting a Django site running on Dreamhost. Luckily, there is this blog post which details exactly how to get your site up and running on Dreamhost. I have used this method a couple of times (once modified to get a Satchmo install running) and have met with success each time.

Once the site is running, setting up tests poses another problem. The default test behaviour for a Django site creates and then destroys a whole test database on each test run. Normally this is not a problem, but the way Dreamhost has their MySQL setup running, the commands to create and drop databases are not available outside of their administration panel. What follows is the solution I have come up with and am currently using to run Django unittests with my Dreamhost MySQL databases.

First of all, it is important to understand a little about the default testing setup that Django uses. When you run the command python manage.py test, Django looks at the TEST_RUNNER setting to determine what to do. By default this setting points to django.test.simple.run_tests. A quick look and we can see that this is the functionality we want to alter. Specifically, lines 2 and 4 shown in listing #1 are what we need to replace.

1
2
3
4

from django.db import connection
connection.creation.create_test_db(verbosity, autoclobber=not interactive)
result = unittest.TextTestRunner(verbosity=verbosity).run(suite)
connection.creation.destroy_test_db(old_name, verbosity)

Listing 1: Excerpt from django.test.simple.run_tests

It is these lines which make the calls to create and then drop the test database. The functionality needed, however, in order to work with Dreamhost’s setup, is to “flush” and existing test database both before and after the test run is complete. I have accomplished this through the creation of a method called flush_test_db, shown in Listing #2.

1
2
3
4
5
6
7
8
9
10
11

def flush_test_db(test_db):
    """Method to flush out the test database, removing tables, before
    and after test runs"""
 
    from django.db import connection
    cursor = connection.cursor()
 
    current_tables =  connection.introspection.table_names()
    for table in current_tables:
        cursor.execute("use %s" % test_db)
        cursor.execute("drop table %s" % table)

Listing 2: New method to remove all tables from the test db

With this method, we can drop all of the tables in the test db both before and after a test run. It is necessary to do both just in case something goes horribly awry during a test run and we end up with leftover data from a previous test run. With this new method, we can then create a new version of the run_tests method and place both methods into a new file. I called my file test_runner.py and placed it in a tests directory within my app. The complete test_runner.py is shown in Listing 3.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

"""Override of the simple tests module from django in order to skip
database creation and use an existing db"""
import unittest
from django.conf import settings
from django.db.models import get_app, get_apps
from django.db.backends.creation import TEST_DATABASE_PREFIX
from django.test import simple
from django.test.utils import setup_test_environment, teardown_test_environment
from django.core import management
 
def flush_test_db(test_db):
    """Method to flush out the test database, removing tables, before
    and after test runs"""
 
    from django.db import connection
    cursor = connection.cursor()
 
    current_tables =  connection.introspection.table_names()
    for table in current_tables:
        cursor.execute("use %s" % test_db)
        cursor.execute("drop table %s" % table)
 
def run_tests(test_labels, verbosity=1, interactive=True, extra_tests=[]):
    """
    Replacement for the run_tests in Django.
    """
    setup_test_environment()
 
    settings.DEBUG = False
    suite = unittest.TestSuite()
 
    if test_labels:
        for label in test_labels:
            if '.' in label:
                suite.addTest(simple.build_test(label))
            else:
                app = get_app(label)
                suite.addTest(simple.build_suite(app))
    else:
        for app in get_apps():
            suite.addTest(simple.build_suite(app))
 
    for test in extra_tests:
        suite.addTest(test)
 
    old_name = settings.DATABASE_NAME
 
    if settings.TEST_DATABASE_NAME:
        test_database_name = settings.TEST_DATABASE_NAME
    else:
        test_database_name = TEST_DATABASE_PREFIX + settings.DATABASE_NAME
 
    #replace the normal DATABASE_NAME with the name of the test db
    settings.DATABASE_NAME = test_database_name
 
    #flush the test db
    flush_test_db(test_database_name)
    #load the test db with models
    management.call_command('syncdb')
 
    result = unittest.TextTestRunner(verbosity=verbosity).run(suite)
 
    #flush the test db again
    flush_test_db(test_database_name)
 
    #reset the value of DATABASE_NAME in settings
    settings.DATABASE_NAME = old_name
    teardown_test_environment()
 
    return len(result.failures) + len(result.errors)

Listing 3: The complete test_runner.py

The final step is to point the test framework to your new test runner by placing a new line in your settings.py. The exact value will vary based on where in your project you have placed the new test runner. In this example I have placed the test_runner.py file, containing the run_tests method in the tests directory of an app called ‘your_app’. This is shown in listing 4.

1

TEST_RUNNER = 'your_app.tests.test_runner.run_tests'

Listing 4: Settings.py setting pointing the test framework towards your new test_runner

Finally a few words of caveat. I have omitted any comments I am using to tell pylint to ignore certain things and I have done little in this sample code to properly handle Exceptions that might arise. Nevertheless, this should get you a good start on running unit tests in Django if you host your site with Dreamhost.