Test a Reusable Django Application for Support of Multiple Django Releases with Tox and TravisCI

Instructions for adding Tox and TravisCI to an existing reusable Django application in order to test for backwards and forwards compatibility with various Django and Python point releases.

There are 3 main steps to this:

  1. Adding support for testing a reusable Django application against one version of Python and one version of Django using a demo Django project and Setup Test.
  2. Automating the local testing of a reusable Django application against multiple versions of Python and multiple versions of Django in virtual environments using Tox.
  3. Automating the remote testing of a reusable Django application against multiple containerized versions of Python and multiple versions of Django in virtual environments using Tox on the TravisCI platform.

Assumptions

  1. a reusable Django application is already created and hosted on GitHub
  2. a TravisCI account has been created and associated with the GitHub user (TravisCI is free for OpenSource projects)

Create a local copy of the reusable Django application if it doesn't already exist; clone the repo and create a virtual environment to work in:

$ git clone __repo_url__
$ cd $_
$ mkvirtualenv -a . __env_name__

Calling the Test Suite from the Setup script

Since this is a reusable Django application, it needs to be tested in a Django project. Update the project setup script to call the Django test suite. Eric Holser explains this in depth in a still mostly relevant post from 2009. The Django documentation also includes an example.

As Eric explains, running the test suite without the Django management command relies on setting a test_suite in the setup() method call of the project's setup.py file that calls the test runner. That should look something like:

...
setup(
  ...
  test_suite="runtests.runtests",
  ...
)

test_suite is calling a function called runtests in the runtests module which in turn is going to call the test runner. To make that module, create a file called runtests.py also in the root of the project:

$ touch runtests.py

Edit that to look like this:

import os
import sys
import django

from django.conf import settings
from django.test.utils import get_runner


def runtests():
    os.environ['DJANGO_SETTINGS_MODULE'] = 'tests.test_settings'
    django.setup()
    TestRunner = get_runner(settings)
    test_runner = TestRunner()
    failures = test_runner.run_tests(["tests"])
    sys.exit(bool(failures))

if __name__ == '__main__':
    runtests()

The runtests() function calls the test runner for a demo Django project called tests. Now, create that project. This could be done by installing the Django package and creating the project using django-admin createproject tests but that's probably overkill. Instead, create the project from scratch. At the minimum that requires a tests directory in the root of the reusable application, an empty __init__.py to identify it a module, an empty models.py so tests can run, a settings file named test_settings.py, a urls.py file and a test suite (an empty tests.py file for now):

$ mkdir tests; touch tests/__init__.py tests/models.py  tests/test_settings.py tests/urls.py tests/tests.py

The Django documentation specifies the minimum requirements for the settings file to be a SECRET_KEY and INSTALLED_APPS list. Depending on the tests, there can be quite a bit more required. This is what I've found to work:

import os

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

SECRET_KEY = 'fake-key'

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    '__my_reusable_apps_dependency_app__',
    '__my_reusable_app__',
]

ROOT_URLCONF = 'tests.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'APP_DIRS': True,
    },
]

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

In either case, add the name of the reusable app and any dependency apps to the INSTALLED_APPS constant.

Populate the test application's tests.py file and the urls.py file if required.

Populating tests can be pretty simple, I had tests in the reusable Django app itself from development, so I just copy-pasted the tests in that file into the one in the tests project directory and removed the original file. The tests for django-robots, a very simple reusable app on which I based this post, can be viewed on GitHub.

Now the test suite can be called by running $ python setup.py test. This will run the test suite against whatever the active Python version is in the current working environment (Django will be installed according to the requirements in the setup() function, in my case I have install_requires=['Django>=1.7'], so the latest version of Django will be installed by default.

Tox

When Tox runs, it creates unique virtual environments for each list of dependencies, in this simple example, different versions of Django. Instead of just testing on whatever active version of Python the development test environment has and whatever version of Django is specified in the setup script, a variety of combinations can be tested against with one command.

Begin by installing Tox:

$ pip install tox

Tox needs a config file called tox.ini. Create this in the root of the project and then open it for editing:

$ touch tox.ini

Taking a quick look at the Django website to find the Supported Versions table to determine which versions of Django need to be supported.

Use the Django FAQ to check the Python requirements for the various Django versions. Django 1.8 requires Python 2.7, 3.2, 3.3, 3.4, or 3.5, Django 1.9 requires Python 2.7, 3.4 or 3.5, and so on.

With that knowledge, the Tox config file should look like:

[tox]
envlist =
    py27-{18,19,110},
    py32-{18},
    py33-{18},
    py34-{18,19,110},
    py35-{18,19,110}
[testenv]
deps =
    18: Django >= 1.8, < 1.9
    19: Django >= 1.9, < 1.10
    110: Django >= 1.10, < 1.11
commands = python setup.py test

There are 2 components, the envlist and the testenv. The envlist includes the versions of Python to be tested against (2.7, 3.2, etc. as defined in the Tox documentation list of environments) and a reference to a variable in the testenv deps below. The testenv deps assigns a variable to a specific Django point release, e.g. 17 references Django >= 1.7, < 1.8. Finally the config calls the test script to run.

With this in place, tox can be locally. Note that the system needs to have all the versions of Python listed in the Tox config to test against installed or it will err, although, execution will continue if it can find any of the other Python versions. To run Tox locally:

$ tox

TravisCI

TravisCI automates the process of running Tox every time the code gets pushed to Git. Additionally, it handles the installation of all the various versions of Python in their own isolated environments.

TravisCI needs a config file to run, create that in the root of the project:

$ touch .travis.yml

Edit that:

language: python
python: 3.5
env:
  - TOX_ENV=py27-18
  - TOX_ENV=py27-19
  - TOX_ENV=py27-110
  - TOX_ENV=py32-18
  - TOX_ENV=py33-18
  - TOX_ENV=py34-18
  - TOX_ENV=py34-19
  - TOX_ENV=py34-110
  - TOX_ENV=py35-18
  - TOX_ENV=py35-19
  - TOX_ENV=py35-110
# command to install dependencies
install:
  - pip install tox
# command to run tests
script:
  - tox -e $TOX_ENV
# containers
sudo: false

The TravisCI docs give an overview of the config. Breaking it down:

  1. the language is defined as python
  2. the version is set to 3.5 to allow Python version 3.5 to run (per issue #4794)
  3. a list of environments is set to each environment defined in the envlist of the Tox file
  4. pip installs tox on the TravisCI container's operating system
  5. tox is called, passing the environment defined above
  6. finally, the file specifies the use of containers rather than sudo

With this in place, TravisCI will run the Tox tests each time a push is made to GitHub:

$ echo ".tox/" >> .gitignore
$ echo ".eggs/" >> .gitignore
$ echo "*.egg-info" >> .gitignore
$ echo "build/" >> .gitignore
$ git add -A; git commit -m "refactored for CI"; git push origin __branch_name__

Revisions

  1. August 29, 2016 -- Added Django 1.10. Removed Django 1.7. Updated links to find supported versions and Python requirements therein. Changed commands to create files to use touch rather than vim. Added commands to populate .gitignore with egg, build, and Tox directories and files.