Testing Your Python Projects#
Your Very First Python Test#
Testing in Python is fairly straightforward. Here's an example: Say we wrote a function get_answer, which should return 42, the answer to life, the universe and everything. (In the real world,
the test and the code to be tested would live in different files, but we'll keep them together for simplicity here.)
# function to test
def get_answer():
return 42
# actual test
def test_answer_to_life_is_42():
assert get_answer() == 42
This is pretty simple. The function is just your everyday Python function, and the test is a separate
function that calls the function get_answer. The interesting part here is the keyword assert, which
evaluates a condition. If the condition evaluates to True, we say that the test passed; if it returns False, we say the test failed.
There are many types of tests. The case above is called a unit test, which comes from the notion that we are testing a unit of code (not the entire project). The idea is to have one unit test for each unit of code (e.g. a function).
Software testing is an entire subject on its own, but keep it simple for now and follow these guidelines when writing your tests:
- Your tests should check that one thing works (a function for example)
- Your tests should be independent of other tests (the outcome of one should not affect others)
- Given the same input, your test should always return the result (when testing a Data Science pipeline this gets tricky given its probabilistic nature)
Running Tests with py.test#
One of the best tools for testing in Python is pytest, which provides some useful features to reduce
the amount of boilerplate code for your tests, as well as a command to automatically find Python files with
tests and run them.
You can install it with pip:
pip install pytest
If you want to actually run the test above, you need to get a copy of this repo.
git clone https://github.com/dssg/hitchhikers-guide
cd hitchhikers-guide
To run your tests (note the dot in the middle):
py.test
pytest will look for all the tests in your project. In our case there's a copy of our test in a file called test_meaning.py on this folder.
The output should look like this:
==================== test session starts ===========================
platform darwin -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: stuff/hitchhikers-guide/tech-tutorials/testtesttest, inifile:
collected 1 items
test_life.py .
==================== 1 passed in 0.01 seconds =======================
Now, imagine that we accidentally modify our function to:
# function to test
def get_answer():
return 41
If we run py.test again, we'll see the following:
======================== test session starts ====================
platform darwin -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /Users/Edu/development/dsapp/hitchhikers-guide/tech-tutorials/testtesttest, inifile:
collected 1 items
test_meaning.py F
============================== FAILURES ========================
__________________ test_answer_to_file_is_42 __________________
def test_answer_to_file_is_42():
> assert get_answer() == 42
E assert 41 == 42
E + where 41 = get_answer()
test_meaning.py:8: AssertionError
===================== 1 failed in 0.02 seconds ==================
We can see from the output that our test failed. Now, every time you modify your code just run py.test to make sure your code still works! (But remember: just because you tested something doesn't necessarily mean it works. You can keep adding new tests as you identify new edge cases and errors).
Where to Store Your Tests#
The are no strict rules on where to store your tests. In a Data Science project, you are going to have a lot of folders with code for many tasks (e.g. ETL, modeling). The first thing to take into account is to separate your pipeline steps from your source code: simply speaking, source code is functions and classes that you want to reuse in various steps of your pipeline (e.g. creating a database connection). Let's see an example to make this clear:
.
├── docs
│ └── documentation_here.txt
├── etl
│ └── code_to_load_to_db.txt
├── evaluation
│ └── code_to_evaluate_models_here.txt
├── exploration
│ └── jupyter_notebooks_with_cool_plots_here.txt
├── features
│ └── code_to_generate_features_here.txt
├── lib
│ ├── lib
│ │ ├── __init__.py
│ │ ├── db
│ │ │ ├── __init__.py
│ │ │ ├── load.py
│ │ │ └── process.py
│ │ └── util.py
│ └── tests
│ ├── test_db.py
│ └── test_util.py
└── model
└── code_to_train_models_here.txt
In the diagram above, you can see the different steps in your pipeline (ETL, exploration, feature generation, modeling, model evaluation) all those folders will contain a mix of Python, shell and SQL scripts. Then, there's another folder called lib, which stores the source code for this project. Inside such a folder, you'll find another two folders lib and tests, the first one stores the actual source code and the later stores the tests for the code inside lib.
Of course, that doesn't mean you should limit your tests to your source code! You should also test your pipeline, but a well designed pipeline will put the complicated parts in the source code, so your pipeline steps are short and simple.
The problem with testing a pipeline is that some steps won't be deterministic, but there are some things you can do.
Tip: To access the code in lib you can either create a setup.py file to install or add the folder to your PYTHONPATH.
#
Other Python Testing Tools#
- unittest or unitest2 if you use Python 3 (part of the Python standard library)
- nose - another good option to run your tests