Writing tests

We need to create unit tests for each module and test each function of the module.

An introductory example

Suppose we have a file called data_utils.py with two functions func1 and func2, we should have a corresponding test file called test_data_utils.py with two corresponding test functions called test_func1 and test_func2 and data_utils.py imported.

  • Note that we use pytest in our framework. Always import pytest at the top of the test file.
  • Please follow the naming convention: test_*.py for test file and test_* for test functions.
# content of mousestyles/mousestyles/utils.py
def func1():
   return 3

def func2(x):
   return x + 1

# content of mousestyles/mousestyles/tests/test_utils.py
from __future__ import (absolute_import,
                        division, print_function,
                        unicode_literals)

import pytest

from mousestyles.utils import func1, func2

def test_func1():
   assert f() == 3

def test_func2():
   assert func(3) == 4

Run all the tests for mousestyles

  • For our mousestyles package, you can run all the tests with make test. Note that make test always run make make install, thus you do not need to install beforehand.
  • Run make test-fast for quicker (but less foolproof) development.
  • If you want to look at the details, code for make command can be found here.

Pytest: basic usage

Installation:

pip install -U pytest

Run all files in the current directory and its subdirectories of the form test_*.py or *_test.py:

$ py.test

Or you may run one test file with or without “quiet” reporting mode:

$ py.test test_data_utils.py
$ py.test -q test_data_utils.py

Pytest: writing and reporting of assertions

pytest allows you to use the standard python assert for verifying expectations and values. For example:

# contents of example1.py
def f():
   return 3

def test_function():
   assert f() == 4

If we run example1.py using py.test example1.py, it will fail because the expected value is 4, but f return 3.

We can also specify a message with the assertion like this:

assert a % 2 == 0, "value was odd, should be even"

In order to write assertions about raised exceptions, you can use pytest.raises as a context manager like this:

import pytest

def test_zero_division():
   with pytest.raises(ZeroDivisionError):
      1 / 0

def test_exception():
   with pytest.raises(Exception):
      x = 1 / 0

See Built-in Exceptions for more about raising errors.

If you need to have access to the actual exception info you may use:

def test_recursion_depth():
   with pytest.raises(RuntimeError) as excinfo:
      def f():
         f()
      f()
 assert 'maximum recursion' in str(excinfo.value)

Pytest also support expected warnings, see pytest.warn

For more about assertions, see Assertions in pytest

What aspects of a function need to be tested:

  • check if it returns correct type of object
  • check if it returns correct dimension
  • check if it returns the correct value, by
    • prior knowledge
    • different implementation: i.e. use R vs Python; use \(Var(X)\) vs. \(E(X^2) - E(X)^2\)
    • theoretical derivation
  • “regression”: if the function is improved (by speed, efficiency) while has same functionality, check the output is the same with older version.
  • assert errors occured: i.e. when the function takes three arguements while we only give two, make sure the function will throw an error message

Reference

Most example above comes from Pytest documentation. See Pytest Documentation for more detail.