Changing standard (Python) test discovery

Ignore paths during test collection

You can easily ignore certain test directories and modules during collection by passing the --ignore=path option on the cli. pytest allows multiple --ignore options. Example:

tests/
|-- example
|   |-- test_example_01.py
|   |-- test_example_02.py
|   '-- test_example_03.py
|-- foobar
|   |-- test_foobar_01.py
|   |-- test_foobar_02.py
|   '-- test_foobar_03.py
'-- hello
    '-- world
        |-- test_world_01.py
        |-- test_world_02.py
        '-- test_world_03.py

Now if you invoke pytest with --ignore=tests/foobar/test_foobar_03.py --ignore=tests/hello/, you will see that pytest only collects test-modules, which do not match the patterns specified:

=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 5 items

tests/example/test_example_01.py .                                   [ 20%]
tests/example/test_example_02.py .                                   [ 40%]
tests/example/test_example_03.py .                                   [ 60%]
tests/foobar/test_foobar_01.py .                                     [ 80%]
tests/foobar/test_foobar_02.py .                                     [100%]

========================= 5 passed in 0.02 seconds =========================

The --ignore-glob option allows to ignore test file paths based on Unix shell-style wildcards. If you want to exclude test-modules that end with _01.py, execute pytest with --ignore-glob='*_01.py'.

Deselect tests during test collection

Tests can individually be deselected during collection by passing the --deselect=item option. For example, say tests/foobar/test_foobar_01.py contains test_a and test_b. You can run all of the tests within tests/ except for tests/foobar/test_foobar_01.py::test_a by invoking pytest with --deselect tests/foobar/test_foobar_01.py::test_a. pytest allows multiple --deselect options.

Keeping duplicate paths specified from command line

Default behavior of pytest is to ignore duplicate paths specified from the command line. Example:

pytest path_a path_a

...
collected 1 item
...

Just collect tests once.

To collect duplicate tests, use the --keep-duplicates option on the cli. Example:

pytest --keep-duplicates path_a path_a

...
collected 2 items
...

As the collector just works on directories, if you specify twice a single test file, pytest will still collect it twice, no matter if the --keep-duplicates is not specified. Example:

pytest test_a.py test_a.py

...
collected 2 items
...

Changing directory recursion

You can set the norecursedirs option in an ini-file, for example your pytest.ini in the project root directory:

# content of pytest.ini
[pytest]
norecursedirs = .svn _build tmp*

This would tell pytest to not recurse into typical subversion or sphinx-build directories or into any tmp prefixed directory.

Changing naming conventions

You can configure different naming conventions by setting the python_files, python_classes and python_functions in your configuration file. Here is an example:

# content of pytest.ini
# Example 1: have pytest look for "check" instead of "test"
[pytest]
python_files = check_*.py
python_classes = Check
python_functions = *_check

This would make pytest look for tests in files that match the check_* .py glob-pattern, Check prefixes in classes, and functions and methods that match *_check. For example, if we have:

# content of check_myapp.py
class CheckMyApp:
    def simple_check(self):
        pass

    def complex_check(self):
        pass

The test collection would look like this:

$ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
configfile: pytest.ini
collected 2 items

<Dir pythoncollection.rst-205>
  <Module check_myapp.py>
    <Class CheckMyApp>
      <Function simple_check>
      <Function complex_check>

======================== 2 tests collected in 0.12s ========================

You can check for multiple glob patterns by adding a space between the patterns:

# Example 2: have pytest look for files with "test" and "example"
# content of pytest.ini
[pytest]
python_files = test_*.py example_*.py

Note

the python_functions and python_classes options has no effect for unittest.TestCase test discovery because pytest delegates discovery of test case methods to unittest code.

Interpreting cmdline arguments as Python packages

You can use the --pyargs option to make pytest try interpreting arguments as python package names, deriving their file system path and then running the test. For example if you have unittest2 installed you can type:

pytest --pyargs unittest2.test.test_skipping -q

which would run the respective test module. Like with other options, through an ini-file and the addopts option you can make this change more permanently:

# content of pytest.ini
[pytest]
addopts = --pyargs

Now a simple invocation of pytest NAME will check if NAME exists as an importable package/module and otherwise treat it as a filesystem path.

Finding out what is collected

You can always peek at the collection tree without running tests like this:

. $ pytest --collect-only pythoncollection.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
configfile: pytest.ini
collected 3 items

<Dir pythoncollection.rst-205>
  <Dir CWD>
    <Module pythoncollection.py>
      <Function test_function>
      <Class TestClass>
        <Function test_method>
        <Function test_anothermethod>

======================== 3 tests collected in 0.12s ========================

Customizing test collection

You can easily instruct pytest to discover tests from every Python file:

# content of pytest.ini
[pytest]
python_files = *.py

However, many projects will have a setup.py which they don’t want to be imported. Moreover, there may files only importable by a specific python version. For such cases you can dynamically define files to be ignored by listing them in a conftest.py file:

# content of conftest.py
import sys

collect_ignore = ["setup.py"]
if sys.version_info[0] > 2:
    collect_ignore.append("pkg/module_py2.py")

and then if you have a module file like this:

# content of pkg/module_py2.py
def test_only_on_python2():
    try:
        assert 0
    except Exception, e:
        pass

and a setup.py dummy file like this:

# content of setup.py
0 / 0  # will raise exception if imported

If you run with a Python 2 interpreter then you will find the one test and will leave out the setup.py file:

#$ pytest --collect-only
====== test session starts ======
platform linux2 -- Python 2.7.10, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
collected 1 items
<Module 'pkg/module_py2.py'>
  <Function 'test_only_on_python2'>

====== 1 tests found in 0.04 seconds ======

If you run with a Python 3 interpreter both the one test and the setup.py file will be left out:

$ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
configfile: pytest.ini
collected 0 items

======================= no tests collected in 0.12s ========================

It’s also possible to ignore files based on Unix shell-style wildcards by adding patterns to collect_ignore_glob.

The following example conftest.py ignores the file setup.py and in addition all files that end with *_py2.py when executed with a Python 3 interpreter:

# content of conftest.py
import sys

collect_ignore = ["setup.py"]
if sys.version_info[0] > 2:
    collect_ignore_glob = ["*_py2.py"]

Since Pytest 2.6, users can prevent pytest from discovering classes that start with Test by setting a boolean __test__ attribute to False.

# Will not be discovered as a test
class TestClass:
    __test__ = False