Testing

Continuous Integration

Learning Objectives

  • Understand how continuous integration speeds software development
  • Understand the benefits of continuous integration
  • Implement a continuous integration server
  • Identify a few options for hosting a continuous integration server

To make running the tests as easy as possible, many software development teams implement a strategy called continuous integration. As its name implies, continuous integration integrates the test suite into the development process. Every time a change is made to the repository, the continuous integration system builds and checks that code.

Thought Experiment: Does Your Software Work on Your Colleague’s Computer?

Imagine you developed software on a MacOSX computer. Last week, you helped your office mate build and run it on their Linux computer. You’ve made some changes since then.

  1. How can you be sure it will still work if they update their repository when they come back from vacation?
  2. How long will that process take? machine until you try rebuilding it on their machine.

The typical story in a research lab is that, well, you don’t know whether it will work on your colleagues’ machine until you try rebuilding it on their machine. If you have a build system, it might take a few minutes to update the repository, rebuild the code, and run the tests. If you don’t have a build system, it could take all afternoon just to see if your new changes are compatible.

Since the first step the server conducts is to check out the code from a repository, we’ll need to put our code online to make use of this kind of server (unless we are able/willing to set up our own CI server).

Set Up a Mean Git Repository on GitHub

Your mean.py test_mean.py files can be the contents of a repository on GitHub.

  1. Go to GitHub and create a repository called mean.
  2. Clone that repository (git clone https://github.com:yourusername/mean)
  3. Copy the mean.py and test_mean.py files into the repository directory.
  4. Use git to add, commit, and push the two files to GitHub.

Giving Instructions

Your work on the mean function has both code and tests. Let’s copy that code into its own repository and add continuous integration to that repository.

Decide How To Set Up the Server

It doesn’t need a build system, because Python does not need to be compiled.

  1. What does it need?
  2. Write the names of the software dependencies in a file called requirements.txt and save the file.
  3. In fact, why don’t you go ahead and version control it?

Travis-CI

Travis is a continous integration server hosting platform. It’s commonly used in Ruby development circles as well as in the scientific python community.

Get a Travis-CI Account

To use Travis, all you need is an account. It’s free.

  1. Someone in your group should sign up for a Travis account.
  2. Follow the instructions on the Travis website to connect your Travis account with GitHub.

A file called .travis.yml in your repository will signal to Travis that you want to build and test this repository on Travis-CI. Such a file, for our purposes, is very simple:

language: python
python:
  - "2.6"
  - "2.7"
  - "3.2"
  - "3.3"
  - "3.4"
  - "nightly"
# command to install dependencies
install:
  - "pip install -r requirements.txt"
# command to run tests
script: nosetests

You can see how the python package manager, pip, will use your requirements.txt file from the previous exercise. That requirements.txt file is a conventional way to list all of the python packages that we need. If we needed nose, numpy, and pymol, the requirements.txt file would look like this:

nose
numpy
pymol

Add a .travis.yml file

  1. Add .travis.yml to your repository
  2. Commit and push it.
  3. Check the situation at your server

Continuous Integration Hosting

We gave the example of Travis because it’s very very simple to spin up. While it is able to run many flavors of Linux, it currently doesn’t support other platforms as well. Depending on your needs, you may consider other services such as:

= buildbot - CDash - Jenkins

Key Points

  • Servers exist for automatically running your tests
  • Running the tests can be triggered by a GitHub pull request
  • CI allows cross platform build testing
  • A .travis.yml file configures a build on the travis-ci servers
  • Many free CI servers are available
Software Carpentry Source Contact License