Changelog¶
Versions follow Semantic Versioning (<major>.<minor>.<patch>
).
Backward incompatible (breaking) changes will only be introduced in major versions with advance notice in the Deprecations section of releases.
pytest 8.3.3 (2024-09-09)¶
Bug fixes¶
#12446: Avoid calling
@property
(and other instance descriptors) during fixture discovery – by @asottile#12659: Fixed the issue of not displaying assertion failure differences when using the parameter
--import-mode=importlib
in pytest>=8.1.#12667: Fixed a regression where type change in
ExceptionInfo.errisinstance
causedmypy
to fail.#12744: Fixed typing compatibility with Python 3.9 or less – replaced
typing.Self
withtyping_extensions.Self
– by @Avasam#12745: Fixed an issue with backslashes being incorrectly converted in nodeid paths on Windows, ensuring consistent path handling across environments.
#6682: Fixed bug where the verbosity levels where not being respected when printing the “msg” part of failed assertion (as in
assert condition, msg
).#9422: Fix bug where disabling the terminal plugin via
-p no:terminal
would cause crashes related to missing theverbose
option.– by @GTowers1
Improved documentation¶
Miscellaneous internal changes¶
#12769: Fix typos discovered by codespell and add codespell to pre-commit hooks.
pytest 8.3.2 (2024-07-24)¶
Bug fixes¶
#12652: Resolve regression
conda
environments where no longer being automatically detected.– by @RonnyPfannschmidt
pytest 8.3.1 (2024-07-20)¶
The 8.3.0 release failed to include the change notes and docs for the release. This patch release remedies this. There are no other changes.
pytest 8.3.0 (2024-07-20)¶
New features¶
#12231: Added
--xfail-tb
flag, which turns on traceback output for XFAIL results.If the
--xfail-tb
flag is not given, tracebacks for XFAIL results are NOT shown.The style of traceback for XFAIL is set with
--tb
, and can beauto|long|short|line|native|no
.Note: Even if you have
--xfail-tb
set, you won’t see them if--tb=no
.
Some history:
With pytest 8.0,
-rx
or-ra
would not only turn on summary reports for xfail, but also report the tracebacks for xfail results. This caused issues with some projects that utilize xfail, but don’t want to see all of the xfail tracebacks.This change detaches xfail tracebacks from
-rx
, and now we turn on xfail tracebacks with--xfail-tb
. With this, the default-rx
/-ra
behavior is identical to pre-8.0 with respect to xfail tracebacks. While this is a behavior change, it brings default behavior back to pre-8.0.0 behavior, which ultimately was considered the better course of action.– by @okken
#12281: Added support for keyword matching in marker expressions.
Now tests can be selected by marker keyword arguments. Supported values are
int
, (unescaped)str
,bool
&None
.See marker examples for more information.
– by @lovetheguitar
#12567: Added
--no-fold-skipped
command line option.If this option is set, then skipped tests in short summary are no longer grouped by reason but all tests are printed individually with their nodeid in the same way as other statuses.
– by @pbrezina
Improvements in existing functionality¶
#12469: The console output now uses the “third-party plugins” terminology, replacing the previously established but confusing and outdated reference to setuptools – by @webknjaz.
#12544, #12545: Python virtual environment detection was improved by checking for a
pyvenv.cfg
file, ensuring reliable detection on various platforms – by @zachsnickers.#2871: Do not truncate arguments to functions in output when running with
-vvv
.#389: The readability of assertion introspection of bound methods has been enhanced – by @farbodahm, @webknjaz, @obestwalter, @flub and @glyphack.
Earlier, it was like:
=================================== FAILURES =================================== _____________________________________ test _____________________________________ def test(): > assert Help().fun() == 2 E assert 1 == 2 E + where 1 = <bound method Help.fun of <example.Help instance at 0x256a830>>() E + where <bound method Help.fun of <example.Help instance at 0x256a830>> = <example.Help instance at 0x256a830>.fun E + where <example.Help instance at 0x256a830> = Help() example.py:7: AssertionError =========================== 1 failed in 0.03 seconds ===========================
And now it’s like:
=================================== FAILURES =================================== _____________________________________ test _____________________________________ def test(): > assert Help().fun() == 2 E assert 1 == 2 E + where 1 = fun() E + where fun = <test_local.Help object at 0x1074be230>.fun E + where <test_local.Help object at 0x1074be230> = Help() test_local.py:13: AssertionError =========================== 1 failed in 0.03 seconds ===========================
#7662: Added timezone information to the testsuite timestamp in the JUnit XML report.
Bug fixes¶
#11706: Fixed reporting of teardown errors in higher-scoped fixtures when using
--maxfail
or--stepwise
.Originally added in pytest 8.0.0, but reverted in 8.0.2 due to a regression in pytest-xdist. This regression was fixed in pytest-xdist 3.6.1.
#11797:
pytest.approx()
now correctly handlesSequence
-like objects.#12204, #12264: Fixed a regression in pytest 8.0 where tracebacks get longer and longer when multiple tests fail due to a shared higher-scope fixture which raised – by @bluetech.
Also fixed a similar regression in pytest 5.4 for collectors which raise during setup.
The fix necessitated internal changes which may affect some plugins:
FixtureDef.cached_result[2]
is now a tuple(exc, tb)
instead ofexc
.SetupState.stack
failures are now a tuple(exc, tb)
instead ofexc
.
#12275: Fixed collection error upon encountering an
abstract
class, including abstractunittest.TestCase
subclasses.#12328: Fixed a regression in pytest 8.0.0 where package-scoped parameterized items were not correctly reordered to minimize setups/teardowns in some cases.
#12424: Fixed crash with
assert testcase is not None
assertion failure when re-running unittest tests using plugins like pytest-rerunfailures. Regressed in 8.2.2.#12472: Fixed a crash when returning category
"error"
or"failed"
with a custom test status frompytest_report_teststatus
hook – @pbrezina.#12505: Improved handling of invalid regex patterns in
pytest.raises(match=r'...')
by providing a clear error message.#12580: Fixed a crash when using the cache class on Windows and the cache directory was created concurrently.
#6962: Parametrization parameters are now compared using
==
instead ofis
(is
is still used as a fallback if the parameter does not support==
). This fixes use of parameters such as lists, which have a differentid
but compare equal, causing fixtures to be re-computed instead of being cached.#7166: Fixed progress percentages (the
[ 87%]
at the edge of the screen) sometimes not aligning correctly when running with pytest-xdist-n
.
Improved documentation¶
#12153: Documented using
PYTEST_VERSION
to detect if code is running from within a pytest run.#12469: The external plugin mentions in the documentation now avoid mentioning setuptools entry-points as the concept is much more generic nowadays. Instead, the terminology of “external”, “installed”, or “third-party” plugins (or packages) replaces that.
– by @webknjaz
#12577:
CI
andBUILD_NUMBER
environment variables role is described in the reference doc. They now also appear when doingpytest -h
– by @MarcBresson.
Contributor-facing changes¶
#12467: Migrated all internal type-annotations to the python3.10+ style by using the
annotations
future import.– by @RonnyPfannschmidt
#11771, #12557: The PyPy runtime version has been updated to 3.9 from 3.8 that introduced a flaky bug at the garbage collector which was not expected to fix there as the 3.8 is EoL.
– by @x612skm
#12493: The change log draft preview integration has been refactored to use a third party extension
sphinxcontib-towncrier
. The previous in-repo script was putting the change log preview file atdoc/en/_changelog_towncrier_draft.rst
. Said file is no longer ignored in Git and might show up among untracked files in the development environments of the contributors. To address that, the contributors can run the following command that will clean it up:$ git clean -x -i -- doc/en/_changelog_towncrier_draft.rst
– by @webknjaz
#12498: All the undocumented
tox
environments now have descriptions. They can be listed in one’s development environment by invokingtox -av
in a terminal.– by @webknjaz
#12501: The changelog configuration has been updated to introduce more accurate audience-tailored categories. Previously, there was a
trivial
change log fragment type with an unclear and broad meaning. It was removed and we now havecontrib
,misc
andpackaging
in place of it.The new change note types target the readers who are downstream packagers and project contributors. Additionally, the miscellaneous section is kept for unspecified updates that do not fit anywhere else.
– by @webknjaz
#12502: The UX of the GitHub automation making pull requests to update the plugin list has been updated. Previously, the maintainers had to close the automatically created pull requests and re-open them to trigger the CI runs. From now on, they only need to click the
Ready for review
button instead.– by @webknjaz
#12522: The
:pull:
RST role has been replaced with a shorter:pr:
due to starting to use the implementation from the third-party sphinx-issues Sphinx extension – by @webknjaz.#12531: The coverage reporting configuration has been updated to exclude pytest’s own tests marked as expected to fail from the coverage report. This has an effect of reducing the influence of flaky tests on the resulting number.
– by @webknjaz
#12533: The
extlinks
Sphinx extension is no longer enabled. The:bpo:
role it used to declare has been removed with that. BPO itself has migrated to GitHub some years ago and it is possible to link the respective issues by using their GitHub issue numbers and the:issue:
role that thesphinx-issues
extension implements.– by @webknjaz
#12562: Possible typos in using the
:user:
RST role is now being linted through the pre-commit tool integration – by @webknjaz.
pytest 8.2.2 (2024-06-04)¶
Bug Fixes¶
#12355: Fix possible catastrophic performance slowdown on a certain parametrization pattern involving many higher-scoped parameters.
#12367: Fix a regression in pytest 8.2.0 where unittest class instances (a fresh one is created for each test) were not released promptly on test teardown but only on session teardown.
#12381: Fix possible “Directory not empty” crashes arising from concurrent cache dir (
.pytest_cache
) creation. Regressed in pytest 8.2.0.
Improved Documentation¶
#12290: Updated Sphinx theme to use Furo instead of Flask, enabling Dark mode theme.
#12356: Added a subsection to the documentation for debugging flaky tests to mention lack of thread safety in pytest as a possible source of flakiness.
#12363: The documentation webpages now links to a canonical version to reduce outdated documentation in search engine results.
pytest 8.2.1 (2024-05-19)¶
Improvements¶
#12334: Support for Python 3.13 (beta1 at the time of writing).
Bug Fixes¶
#12120: Fix
PermissionError
crashes arising from directories which are not selected on the command-line.#12191: Keyboard interrupts and system exits are now properly handled during the test collection.
#12300: Fixed handling of ‘Function not implemented’ error under squashfuse_ll, which is a different way to say that the mountpoint is read-only.
#12308: Fix a regression in pytest 8.2.0 where the permissions of automatically-created
.pytest_cache
directories becamerwx------
instead of the expectedrwxr-xr-x
.
Trivial/Internal Changes¶
#12333: pytest releases are now attested using the recent Artifact Attestation support from GitHub, allowing users to verify the provenance of pytest’s sdist and wheel artifacts.
pytest 8.2.0 (2024-04-27)¶
Breaking Changes¶
#12089: pytest now requires that
unittest.TestCase
subclasses can be instantiated freely usingMyTestCase('runTest')
.If the class doesn’t allow this, you may see an error during collection such as
AttributeError: 'MyTestCase' object has no attribute 'runTest'
.Classes which do not override
__init__
, or do not access the test method in__init__
usinggetattr
or similar, are unaffected.Classes which do should take care to not crash when
"runTest"
is given, as is shown in unittest.TestCases’s implementation. Alternatively, consider usingsetUp
instead of__init__
.If you run into this issue using
tornado.AsyncTestCase
, please see issue 12263.If you run into this issue using an abstract
TestCase
subclass, please see issue 12275.Historical note: the effect of this change on custom TestCase implementations was not properly considered initially, this is why it was done in a minor release. We apologize for the inconvenience.
Deprecations¶
#12069: A deprecation warning is now raised when implementations of one of the following hooks request a deprecated
py.path.local
parameter instead of thepathlib.Path
parameter which replaced it:pytest_ignore_collect
- thepath
parameter - usecollection_path
instead.pytest_collect_file
- thepath
parameter - usefile_path
instead.pytest_pycollect_makemodule
- thepath
parameter - usemodule_path
instead.pytest_report_header
- thestartdir
parameter - usestart_path
instead.pytest_report_collectionfinish
- thestartdir
parameter - usestart_path
instead.
The replacement parameters are available since pytest 7.0.0. The old parameters will be removed in pytest 9.0.0.
See py.path.local arguments for hooks replaced with pathlib.Path for more details.
Features¶
#11871: Added support for reading command line arguments from a file using the prefix character
@
, like e.g.:pytest @tests.txt
. The file must have one argument per line.See Read arguments from file for details.
Improvements¶
#11523:
pytest.importorskip()
will now issue a warning if the module could be found, but raisedImportError
instead ofModuleNotFoundError
.The warning can be suppressed by passing
exc_type=ImportError
topytest.importorskip()
.See pytest.importorskip default behavior regarding ImportError for details.
#11728: For
unittest
-based tests, exceptions during class cleanup (as raised by functions registered withTestCase.addClassCleanup
) are now reported instead of silently failing.#11777: Text is no longer truncated in the
short test summary info
section when-vv
is given.#12112: Improved namespace packages detection when
consider_namespace_packages
is enabled, covering more situations (like editable installs).#9502: Added
PYTEST_VERSION
environment variable which is defined at the start of the pytest session and undefined afterwards. It contains the value ofpytest.__version__
, and among other things can be used to easily check if code is running from within a pytest run.
Bug Fixes¶
#12065: Fixed a regression in pytest 8.0.0 where test classes containing
setup_method
and tests using@staticmethod
or@classmethod
would crash withAttributeError: 'NoneType' object has no attribute 'setup_method'
.Now the
request.instance
attribute of tests using@staticmethod
and@classmethod
is no longerNone
, but a fresh instance of the class, like in non-static methods. Previously it wasNone
, and all fixtures of such tests would share a singleself
.#12135: Fixed issue where fixtures adding their finalizer multiple times to fixtures they request would cause unreliable and non-intuitive teardown ordering in some instances.
#12194: Fixed a bug with
--importmode=importlib
and--doctest-modules
where child modules did not appear as attributes in parent modules.#1489: Fixed some instances where teardown of higher-scoped fixtures was not happening in the reverse order they were initialized in.
Trivial/Internal Changes¶
pytest 8.1.2 (2024-04-26)¶
Bug Fixes¶
#12114: Fixed error in
pytest.approx()
when used withnumpy
arrays and comparing with other types.
pytest 8.1.1 (2024-03-08)¶
Note
This release is not a usual bug fix release – it contains features and improvements, being a follow up
to 8.1.0
, which has been yanked from PyPI.
Features¶
#11475: Added the new
consider_namespace_packages
configuration option, defaulting toFalse
.If set to
True
, pytest will attempt to identify modules that are part of namespace packages when importing modules.#11653: Added the new
verbosity_test_cases
configuration option for fine-grained control of test execution verbosity. See Fine-grained verbosity for more details.
Improvements¶
#10865:
pytest.warns()
now validates thatwarnings.warn()
was called with astr
or aWarning
. Currently in Python it is possible to use other types, however this causes an exception whenwarnings.filterwarnings()
is used to filter those warnings (see CPython #103577 for a discussion). While this can be considered a bug in CPython, we decided to put guards in pytest as the error message produced without this check in place is confusing.#11311: When using
--override-ini
for paths in invocations without a configuration file defined, the current working directory is used as the relative directory.Previously this would raise an
AssertionError
.#11475: –import-mode=importlib now tries to import modules using the standard import mechanism (but still without changing
sys.path
), falling back to importing modules directly only if that fails.This means that installed packages will be imported under their canonical name if possible first, for example
app.core.models
, instead of having the module name always be derived from their path (for example.env310.lib.site_packages.app.core.models
).#11801: Added the
iter_parents()
helper method on nodes. It is similar tolistchain
, but goes from bottom to top, and returns an iterator, not a list.#11850: Added support for
sys.last_exc
for post-mortem debugging on Python>=3.12.#11962: In case no other suitable candidates for configuration file are found, a
pyproject.toml
(even without a[tool.pytest.ini_options]
table) will be considered as the configuration file and define therootdir
.#11978: Add
--log-file-mode
option to the logging plugin, enabling appending to log-files. This option accepts either"w"
or"a"
and defaults to"w"
.Previously, the mode was hard-coded to be
"w"
which truncates the file before logging.#12047: When multiple finalizers of a fixture raise an exception, now all exceptions are reported as an exception group. Previously, only the first exception was reported.
Bug Fixes¶
#11475: Fixed regression where
--importmode=importlib
would import non-test modules more than once.#11904: Fixed a regression in pytest 8.0.0 that would cause test collection to fail due to permission errors when using
--pyargs
.This change improves the collection tree for tests specified using
--pyargs
, see #12043 for a comparison with pytest 8.0 and <8.#12011: Fixed a regression in 8.0.1 whereby
setup_module
xunit-style fixtures are not executed when--doctest-modules
is passed.#12014: Fix the
stacklevel
used when warning about marks used on fixtures.#12039: Fixed a regression in
8.0.2
where tests created usingtmp_path
have been collected multiple times in CI under Windows.
Improved Documentation¶
#11790: Documented the retention of temporary directories created using the
tmp_path
fixture in more detail.
Trivial/Internal Changes¶
#11785: Some changes were made to private functions which may affect plugins which access them:
FixtureManager._getautousenames()
now takes aNode
itself instead of the nodeid.FixtureManager.getfixturedefs()
now takes theNode
itself instead of the nodeid.The
_pytest.nodes.iterparentnodeids()
function is removed without replacement. Prefer to traverse the node hierarchy itself instead. If you really need to, copy the function from the previous pytest release.
#12069: Delayed the deprecation of the following features to
9.0.0
:fspath argument for Node constructors replaced with pathlib.Path.
py.path.local arguments for hooks replaced with pathlib.Path.
It was discovered after
8.1.0
was released that the warnings about the impeding removal were not being displayed, so the team decided to revert the removal.This is the reason for
8.1.0
being yanked.
pytest 8.1.0 (YANKED)¶
Note
This release has been yanked: it broke some plugins without the proper warning period, due to some warnings not showing up as expected.
See #12069.
pytest 8.0.2 (2024-02-24)¶
Bug Fixes¶
#11895: Fix collection on Windows where initial paths contain the short version of a path (for example
c:\PROGRA~1\tests
).#11953: Fix an
IndexError
crash raising fromgetstatementrange_ast
.#12021: Reverted a fix to
--maxfail
handling in pytest 8.0.0 because it caused a regression in pytest-xdist whereby session fixture teardowns may get executed multiple times when the max-fails is reached.
pytest 8.0.1 (2024-02-16)¶
Bug Fixes¶
#11875: Correctly handle errors from
getpass.getuser()
in Python 3.13.#11879: Fix an edge case where
ExceptionInfo._stringify_exception
could crashpytest.raises()
.#11906: Fix regression with
pytest.warns()
using custom warning subclasses which have more than one parameter in their__init__
.#11907: Fix a regression in pytest 8.0.0 whereby calling
pytest.skip()
and similar control-flow exceptions within apytest.warns()
block would get suppressed instead of propagating.#11929: Fix a regression in pytest 8.0.0 whereby autouse fixtures defined in a module get ignored by the doctests in the module.
#11937: Fix a regression in pytest 8.0.0 whereby items would be collected in reverse order in some circumstances.
pytest 8.0.0 (2024-01-27)¶
Bug Fixes¶
pytest 8.0.0rc2 (2024-01-17)¶
Improvements¶
#11233: Improvements to
-r
for xfailures and xpasses:Report tracebacks for xfailures when
-rx
is set.Report captured output for xpasses when
-rX
is set.For xpasses, add
-
in summary between test name and reason, to match how xfail is displayed.
#11825: The
pytest_plugin_registered
hook has a newplugin_name
parameter containing the name by whichplugin
is registered.
Bug Fixes¶
#11706: Fix reporting of teardown errors in higher-scoped fixtures when using
--maxfail
or--stepwise
.NOTE: This change was reverted in pytest 8.0.2 to fix a regression it caused in pytest-xdist.
#11758: Fixed
IndexError: string index out of range
crash inif highlighted[-1] == "\n" and source[-1] != "\n"
. This bug was introduced in pytest 8.0.0rc1.#9765, #11816: Fixed a frustrating bug that afflicted some users with the only error being
assert mod not in mods
. The issue was caused by the fact thatstr(Path(mod))
andmod.__file__
don’t necessarily produce the same string, and was being erroneously used interchangeably in some places in the code.This fix also broke the internal API of
PytestPluginManager.consider_conftest
by introducing a new parameter – we mention this in case it is being used by external code, even if marked as private.
pytest 8.0.0rc1 (2023-12-30)¶
Breaking Changes¶
Old Deprecations Are Now Errors¶
#7363: PytestRemovedIn8Warning deprecation warnings are now errors by default.
Following our plan to remove deprecated features with as little disruption as possible, all warnings of type
PytestRemovedIn8Warning
now generate errors instead of warning messages by default.The affected features will be effectively removed in pytest 8.1, so please consult the Deprecations and Removals section in the docs for directions on how to update existing code.
In the pytest
8.0.X
series, it is possible to change the errors back into warnings as a stopgap measure by adding this to yourpytest.ini
file:[pytest] filterwarnings = ignore::pytest.PytestRemovedIn8Warning
But this will stop working when pytest
8.1
is released.If you have concerns about the removal of a specific feature, please add a comment to #7363.
Version Compatibility¶
#11151: Dropped support for Python 3.7, which reached end-of-life on 2023-06-27.
pluggy>=1.3.0
is now required.
Collection Changes¶
In this version we’ve made several breaking changes to pytest’s collection phase, particularly around how filesystem directories and Python packages are collected, fixing deficiencies and allowing for cleanups and improvements to pytest’s internals. A deprecation period for these changes was not possible.
#7777: Files and directories are now collected in alphabetical order jointly, unless changed by a plugin. Previously, files were collected before directories. See below for an example.
#8976: Running
pytest pkg/__init__.py
now collects thepkg/__init__.py
file (module) only. Previously, it collected the entirepkg
package, including other test files in the directory, but excluding tests in the__init__.py
file itself (unlesspython_files
was changed to allow__init__.py
file).To collect the entire package, specify just the directory:
pytest pkg
.#11137:
pytest.Package
is no longer apytest.Module
orpytest.File
.The
Package
collector node designates a Python package, that is, a directory with an__init__.py
file. PreviouslyPackage
was a subtype ofpytest.Module
(which represents a single Python module), the module being the__init__.py
file. This has been deemed a design mistake (see #11137 and #7777 for details).The
path
property ofPackage
nodes now points to the package directory instead of the__init__.py
file.Note that a
Module
node for__init__.py
(which is not aPackage
) may still exist, if it is picked up during collection (e.g. if you configuredpython_files
to include__init__.py
files).#7777: Added a new
pytest.Directory
base collection node, which all collector nodes for filesystem directories are expected to subclass. This is analogous to the existingpytest.File
for file nodes.Changed
pytest.Package
to be a subclass ofpytest.Directory
. APackage
represents a filesystem directory which is a Python package, i.e. contains an__init__.py
file.pytest.Package
now only collects files in its own directory; previously it collected recursively. Sub-directories are collected as their own collector nodes, which then collect themselves, thus creating a collection tree which mirrors the filesystem hierarchy.Added a new
pytest.Dir
concrete collection node, a subclass ofpytest.Directory
. This node represents a filesystem directory, which is not apytest.Package
, that is, does not contain an__init__.py
file. Similarly toPackage
, it only collects the files in its own directory.pytest.Session
now only collects the initial arguments, without recursing into directories. This work is now done by therecursive expansion process
of directory collector nodes.session.name
is now""
; previously it was the rootdir directory name. This matchessession.nodeid
which has always been""
.The collection tree now contains directories/packages up to the rootdir, for initial arguments that are found within the rootdir. For files outside the rootdir, only the immediate directory/package is collected – note however that collecting from outside the rootdir is discouraged.
As an example, given the following filesystem tree:
myroot/ pytest.ini top/ ├── aaa │ └── test_aaa.py ├── test_a.py ├── test_b │ ├── __init__.py │ └── test_b.py ├── test_c.py └── zzz ├── __init__.py └── test_zzz.py
the collection tree, as shown by
pytest --collect-only top/
but with the otherwise-hiddenSession
node added for clarity, is now the following:<Session> <Dir myroot> <Dir top> <Dir aaa> <Module test_aaa.py> <Function test_it> <Module test_a.py> <Function test_it> <Package test_b> <Module test_b.py> <Function test_it> <Module test_c.py> <Function test_it> <Package zzz> <Module test_zzz.py> <Function test_it>
Previously, it was:
<Session> <Module top/test_a.py> <Function test_it> <Module top/test_c.py> <Function test_it> <Module top/aaa/test_aaa.py> <Function test_it> <Package test_b> <Module test_b.py> <Function test_it> <Package zzz> <Module test_zzz.py> <Function test_it>
Code/plugins which rely on a specific shape of the collection tree might need to update.
#11676: The classes
Node
,Collector
,Item
,File
,FSCollector
are now marked abstract (seeabc
).We do not expect this change to affect users and plugin authors, it will only cause errors when the code is already wrong or problematic.
Other breaking changes¶
These are breaking changes where deprecation was not possible.
#11282: Sanitized the handling of the
default
parameter when defining configuration options.Previously if
default
was not supplied forparser.addini
and the configuration option value was not defined in a test session, then calls toconfig.getini
returned an empty list or an empty string depending on whethertype
was supplied or not respectively, which is clearly incorrect. Also,None
was not honored even ifdefault=None
was used explicitly while defining the option.Now the behavior of
parser.addini
is as follows:If
default
is NOT passed buttype
is provided, then a type-specific default will be returned. For exampletype=bool
will returnFalse
,type=str
will return""
, etc.If
default=None
is passed and the option is not defined in a test session, thenNone
will be returned, regardless of thetype
.If neither
default
nortype
are provided, assumetype=str
and return""
as default (this is as per previous behavior).
The team decided to not introduce a deprecation period for this change, as doing so would be complicated both in terms of communicating this to the community as well as implementing it, and also because the team believes this change should not break existing plugins except in rare cases.
#11667: pytest’s
setup.py
file is removed. If you relied on this file, e.g. to install pytest usingsetup.py install
, please see Why you shouldn’t invoke setup.py directly for alternatives.#9288:
warns()
now re-emits unmatched warnings when the context closes – previously it would consume all warnings, hiding those that were not matched by the function.While this is a new feature, we announce it as a breaking change because many test suites are configured to error-out on warnings, and will therefore fail on the newly-re-emitted warnings.
The internal
FixtureManager.getfixtureclosure
method has changed. Plugins which use this method or which subclassFixtureManager
and overwrite that method will need to adapt to the change.
Deprecations¶
#10465: Test functions returning a value other than
None
will now issue apytest.PytestWarning
instead ofpytest.PytestRemovedIn8Warning
, meaning this will stay a warning instead of becoming an error in the future.#3664: Applying a mark to a fixture function now issues a warning: marks in fixtures never had any effect, but it is a common user error to apply a mark to a fixture (for example
usefixtures
) and expect it to work.This will become an error in pytest 9.0.
Features and Improvements¶
Improved Diffs¶
These changes improve the diffs that pytest prints when an assertion fails.
Note that syntax highlighting requires the pygments
package.
#11520: The very verbose (
-vv
) diff output is now colored as a diff instead of a big chunk of red.Python code in error reports is now syntax-highlighted as Python.
The sections in the error reports are now better separated.
#1531: The very verbose diff (
-vv
) for every standard library container type is improved. The indentation is now consistent and the markers are on their own separate lines, which should reduce the diffs shown to users.Previously, the standard Python pretty printer was used to generate the output, which puts opening and closing markers on the same line as the first/last entry, in addition to not having consistent indentation.
#10617: Added more comprehensive set assertion rewrites for comparisons other than equality
==
, with the following operations now providing better failure messages:!=
,<=
,>=
,<
, and>
.
Separate Control For Assertion Verbosity¶
#11387: Added the new
verbosity_assertions
configuration option for fine-grained control of failed assertions verbosity.If you’ve ever wished that pytest always show you full diffs, but without making everything else verbose, this is for you.
See Fine-grained verbosity for more details.
For plugin authors,
config.get_verbosity
can be used to retrieve the verbosity level for a specific verbosity type.
Additional Support For Exception Groups and __notes__
¶
These changes improve pytest’s support for exception groups.
#10441: Added
ExceptionInfo.group_contains()
, an assertion helper that tests if anExceptionGroup
contains a matching exception.See Matching exception groups for an example.
#11227: Allow
pytest.raises()
match
argument to match againstPEP-678 <https://peps.python.org/pep-0678/>
__notes__
.
Custom Directory collectors¶
#7777: Added a new hook
pytest_collect_directory
, which is called by filesystem-traversing collector nodes, such aspytest.Session
,pytest.Dir
andpytest.Package
, to create a collector node for a sub-directory. It is expected to return a subclass ofpytest.Directory
. This hook allows plugins to customize the collection of directories.
“New-style” Hook Wrappers¶
#11122: pytest now uses “new-style” hook wrappers internally, available since pluggy 1.2.0. See pluggy’s 1.2.0 changelog and the updated docs for details.
Plugins which want to use new-style wrappers can do so if they require
pytest>=8
.
Other Improvements¶
#11216: If a test is skipped from inside an xunit setup fixture, the test summary now shows the test location instead of the fixture location.
#11314: Logging to a file using the
--log-file
option will use--log-level
,--log-format
and--log-date-format
as fallback if--log-file-level
,--log-file-format
and--log-file-date-format
are not provided respectively.#11610: Added the
LogCaptureFixture.filtering()
context manager which adds a givenlogging.Filter
object to thecaplog
fixture.#11447:
pytest.deprecated_call()
now also considers warnings of typeFutureWarning
.#11600: Improved the documentation and type signature for
pytest.mark.xfail
’scondition
param to useFalse
as the default value.#7469:
FixtureDef
is now exported aspytest.FixtureDef
for typing purposes.#11353: Added typing to
PytestPluginManager
.
Bug Fixes¶
#10701:
pytest.WarningsRecorder.pop()
will return the most-closely-matched warning in the list, rather than the first warning which is an instance of the requested type.#11255: Fixed crash on
parametrize(..., scope="package")
without a package present.#11277: Fixed a bug that when there are multiple fixtures for an indirect parameter, the scope of the highest-scope fixture is picked for the parameter set, instead of that of the one with the narrowest scope.
#11456: Parametrized tests now really do ensure that the ids given to each input are unique - for example,
a, a, a0
now results ina1, a2, a0
instead of the previous (buggy)a0, a1, a0
. This necessarily means changing nodeids where these were previously colliding, and for readability adds an underscore when non-unique ids end in a number.#11563: Fixed a crash when using an empty string for the same parametrized value more than once.
#11712: Fixed handling
NO_COLOR
andFORCE_COLOR
to ignore an empty value.#9036:
pytest.warns
and similar functions now capture warnings when an exception is raised inside awith
block.
Improved Documentation¶
Trivial/Internal Changes¶
#11208: The (internal)
FixtureDef.cached_result
type has changed. Now the third itemcached_result[2]
, when set, is an exception instance instead of an exception triplet.#11218: (This entry is meant to assist plugins which access private pytest internals to instantiate
FixtureRequest
objects.)FixtureRequest
is now an abstract class which can’t be instantiated directly. A new concreteTopRequest
subclass ofFixtureRequest
has been added for therequest
fixture in test functions, as counterpart to the existingSubRequest
subclass for therequest
fixture in fixture functions.#11315: The
pytester
fixture now uses themonkeypatch
fixture to manage the current working directory. If you usepytester
in combination withmonkeypatch.undo()
, the CWD might get restored. Usemonkeypatch.context()
instead.#11333: Corrected the spelling of
Config.ArgsSource.INVOCATION_DIR
. The previous spellingINCOVATION_DIR
remains as an alias.#11638: Fixed the selftests to pass correctly if
FORCE_COLOR
,NO_COLOR
orPY_COLORS
is set in the calling environment.
pytest 7.4.4 (2023-12-31)¶
Bug Fixes¶
#11140: Fix non-string constants at the top of file being detected as docstrings on Python>=3.8.
#11572: Handle an edge case where
sys.stderr
andsys.__stderr__
might already be closed when Fault Handler is tearing down.#11710: Fixed tracebacks from collection errors not getting pruned.
#7966: Removed unhelpful error message from assertion rewrite mechanism when exceptions are raised in
__iter__
methods. Now they are treated un-iterable instead.
Improved Documentation¶
#11091: Updated documentation to refer to hyphenated options: replaced
--junitxml
with--junit-xml
and--collectonly
with--collect-only
.
pytest 7.4.3 (2023-10-24)¶
Bug Fixes¶
#10447: Markers are now considered in the reverse mro order to ensure base class markers are considered first – this resolves a regression.
#11239: Fixed
:=
in asserts impacting unrelated test cases.#11439: Handled an edge case where
sys.stderr
might already be closed when Fault Handler is tearing down.
pytest 7.4.2 (2023-09-07)¶
Bug Fixes¶
#11237: Fix doctest collection of
functools.cached_property
objects.#11306: Fixed bug using
--importmode=importlib
which would cause package__init__.py
files to be imported more than once in some cases.#11367: Fixed bug where
user_properties
where not being saved in the JUnit XML file if a fixture failed during teardown.#11394: Fixed crash when parsing long command line arguments that might be interpreted as files.
Improved Documentation¶
#11391: Improved disclaimer on pytest plugin reference page to better indicate this is an automated, non-curated listing.
pytest 7.4.1 (2023-09-02)¶
Bug Fixes¶
#10337: Fixed bug where fake intermediate modules generated by
--import-mode=importlib
would not include the child modules as attributes of the parent modules.#10702: Fixed error assertion handling in
pytest.approx()
whenNone
is an expected or received value when comparing dictionaries.#10811: Fixed issue when using
--import-mode=importlib
together with--doctest-modules
that caused modules to be imported more than once, causing problems with modules that have import side effects.
pytest 7.4.0 (2023-06-23)¶
Features¶
#10901: Added
ExceptionInfo.from_exception()
, a simpler way to create anExceptionInfo
from an exception. This can replaceExceptionInfo.from_exc_info()
for most uses.
Improvements¶
#10872: Update test log report annotation to named tuple and fixed inconsistency in docs for
pytest_report_teststatus
hook.#10907: When an exception traceback to be displayed is completely filtered out (by mechanisms such as
__tracebackhide__
, internal frames, and similar), now only the exception string and the following message are shown:“All traceback entries are hidden. Pass
--full-trace
to see hidden and internal frames.”.Previously, the last frame of the traceback was shown, even though it was hidden.
#10940: Improved verbose output (
-vv
) ofskip
andxfail
reasons by performing text wrapping while leaving a clear margin for progress output.Added
TerminalReporter.wrap_write()
as a helper for that.#10991: Added handling of
%f
directive to print microseconds in log format options, such aslog-date-format
.#11005: Added the underlying exception to the cache provider’s path creation and write warning messages.
#11013: Added warning when
testpaths
is set, but paths are not found by glob. In this case, pytest will fall back to searching from the current directory.#11043: When
--confcutdir
is not specified, and there is no config file present, the conftest cutoff directory (--confcutdir
) is now set to the rootdir. Previously in such cases,conftest.py
files would be probed all the way to the root directory of the filesystem. If you are badly affected by this change, consider adding an empty config file to your desired cutoff directory, or explicitly set--confcutdir
.#11081: The
norecursedirs
check is now performed in apytest_ignore_collect
implementation, so plugins can affect it.If after updating to this version you see that your
norecursedirs
setting is not being respected, it means that a conftest or a plugin you use has a badpytest_ignore_collect
implementation. Most likely, your hook returnsFalse
for paths it does not want to ignore, which ends the processing and doesn’t allow other plugins, including pytest itself, to ignore the path. The fix is to returnNone
instead ofFalse
for paths your hook doesn’t want to ignore.#8711:
caplog.set_level()
andcaplog.at_level()
will temporarily enable the requestedlevel
iflevel
was disabled globally vialogging.disable(LEVEL)
.
Bug Fixes¶
#10831: Terminal Reporting: Fixed bug when running in
--tb=line
mode wherepytest.fail(pytrace=False)
tests reportNone
.#11068: Fixed the
--last-failed
whole-file skipping functionality (“skipped N files”) for non-python test files.#11104: Fixed a regression in pytest 7.3.2 which caused to
testpaths
to be considered for loading initial conftests, even when it was not utilized (e.g. when explicit paths were given on the command line). Now thetestpaths
are only considered when they are in use.#1904: Fixed traceback entries hidden with
__tracebackhide__ = True
still being shown for chained exceptions (parts after “… the above exception …” message).#7781: Fix writing non-encodable text to log file when using
--debug
.
Improved Documentation¶
#9146: Improved documentation for
caplog.set_level()
.
Trivial/Internal Changes¶
#11031: Enhanced the CLI flag for
-c
to now include--config-file
to make it clear that this flag applies to the usage of a custom config file.
pytest 7.3.2 (2023-06-10)¶
Bug Fixes¶
#10169: Fix bug where very long option names could cause pytest to break with
OSError: [Errno 36] File name too long
on some systems.#10894: Support for Python 3.12 (beta at the time of writing).
#10999: The
monkeypatch
setitem
/delitem
type annotations now allowTypedDict
arguments.#11028: Fixed bug in assertion rewriting where a variable assigned with the walrus operator could not be used later in a function call.
#11054: Fixed
--last-failed
’s “(skipped N files)” functionality for files inside of packages (directories with__init__.py
files).
pytest 7.3.1 (2023-04-14)¶
Improvements¶
Bug Fixes¶
#10896: Fixed performance regression related to
tmp_path
and the newtmp_path_retention_policy
option.#10903: Fix crash
INTERNALERROR IndexError: list index out of range
which happens when displaying an exception where all entries are hidden. This reverts the change “Correctly handle__tracebackhide__
for chained exceptions.” introduced in version 7.3.0.
pytest 7.3.0 (2023-04-08)¶
Features¶
#10525: Test methods decorated with
@classmethod
can now be discovered as tests, following the same rules as normal methods. This fills the gap that static methods were discoverable as tests but not class methods.#10755:
console_output_style
now supportsprogress-even-when-capture-no
to force the use of the progress output even when capture is disabled. This is useful in large test suites where capture may have significant performance impact.#7431:
--log-disable
CLI option added to disable individual loggers.#8141: Added
tmp_path_retention_count
andtmp_path_retention_policy
configuration options to control how directories created by thetmp_path
fixture are kept.
Improvements¶
#10226: If multiple errors are raised in teardown, we now re-raise an
ExceptionGroup
of them instead of discarding all but the last.#10658: Allow
-p
arguments to include spaces (eg:-p no:logging
instead of-pno:logging
). Mostly useful in theaddopts
section of the configuration file.#10710: Added
start
andstop
timestamps toTestReport
objects.#10727: Split the report header for
rootdir
,config file
andtestpaths
so each has its own line.#10840: pytest should no longer crash on AST with pathological position attributes, for example testing AST produced by
Hylang <https://github.com/hylang/hy>__
.#6267: The full output of a test is no longer truncated if the truncation message would be longer than the hidden text. The line number shown has also been fixed.
Bug Fixes¶
#10743: The assertion rewriting mechanism now works correctly when assertion expressions contain the walrus operator.
#10765: Fixed
tmp_path
fixture always raisingOSError
onemscripten
platform due to missingos.getuid()
.#1904: Correctly handle
__tracebackhide__
for chained exceptions. NOTE: This change was reverted in version 7.3.1.
Improved Documentation¶
#10782: Fixed the minimal example in Good Integration Practices:
pip install -e .
requires aversion
entry inpyproject.toml
to run successfully.
Trivial/Internal Changes¶
#10669: pytest no longer directly depends on the attrs package. While we at pytest all love the package dearly and would like to thank the
attrs
team for many years of cooperation and support, it makes sense forpytest
to have as little external dependencies as possible, as this helps downstream projects. With that in mind, we have replaced the pytest’s limited internal usage to use the standard library’sdataclasses
instead.Nice diffs for
attrs
classes are still supported though.
pytest 7.2.2 (2023-03-03)¶
Bug Fixes¶
#10533: Fixed
pytest.approx()
handling of dictionaries containing one or more values of0.0
.#10592: Fixed crash if
--cache-show
and--help
are passed at the same time.#10597: Fixed bug where a fixture method named
teardown
would be called as part ofnose
teardown stage.#10626: Fixed crash if
--fixtures
and--help
are passed at the same time.#10660: Fixed
pytest.raises()
to return a ‘ContextManager’ so that type-checkers could narrowpytest.raises(...) if ... else nullcontext()
down to ‘ContextManager’ rather than ‘object’.
Improved Documentation¶
pytest 7.2.1 (2023-01-13)¶
Bug Fixes¶
#10452: Fix ‘importlib.abc.TraversableResources’ deprecation warning in Python 3.12.
#10457: If a test is skipped from inside a fixture, the test summary now shows the test location instead of the fixture location.
#10506: Fix bug where sometimes pytest would use the file system root directory as rootdir on Windows.
#10607: Fix a race condition when creating junitxml reports, which could occur when multiple instances of pytest execute in parallel.
#10641: Fix a race condition when creating or updating the stepwise plugin’s cache, which could occur when multiple xdist worker nodes try to simultaneously update the stepwise plugin’s cache.
pytest 7.2.0 (2022-10-23)¶
Deprecations¶
#10012: Update
pytest.PytestUnhandledCoroutineWarning
to a deprecation; it will raise an error in pytest 8.#10396: pytest no longer depends on the
py
library.pytest
provides a vendored copy ofpy.error
andpy.path
modules but will use thepy
library if it is installed. If you need otherpy.*
modules, continue to install the deprecatedpy
library separately, otherwise it can usually be removed as a dependency.#4562: Deprecate configuring hook specs/impls using attributes/marks.
Instead use
pytest.hookimpl()
andpytest.hookspec()
. For more details, see the docs.#9886: The functionality for running tests written for
nose
has been officially deprecated.This includes:
Plain
setup
andteardown
functions and methods: this might catch users by surprise, assetup()
andteardown()
are not pytest idioms, but part of thenose
support.Setup/teardown using the @with_setup decorator.
For more details, consult the deprecation docs.
#7337: A deprecation warning is now emitted if a test function returns something other than
None
. This prevents a common mistake among beginners that expect that returning abool
(for examplereturn foo(a, b) == result
) would cause a test to pass or fail, instead of usingassert
. The plan is to make returning non-None
from tests an error in the future.
Features¶
#9897: Added shell-style wildcard support to
testpaths
.
Improvements¶
#10218:
@pytest.mark.parametrize()
(and similar functions) now accepts anySequence[str]
for the argument names, instead of justlist[str]
andtuple[str, ...]
.(Note that
str
, which is itself aSequence[str]
, is still treated as a comma-delimited name list, as before).#10381: The
--no-showlocals
flag has been added. This can be passed directly to tests to override--showlocals
declared throughaddopts
.#3426: Assertion failures with strings in NFC and NFD forms that normalize to the same string now have a dedicated error message detailing the issue, and their utf-8 representation is expressed instead.
#8508: Introduce multiline display for warning matching via
pytest.warns()
and enhance match comparison forpytest.ExceptionInfo.match()
as returned bypytest.raises()
.#8646: Improve
pytest.raises()
. Previously passing an empty tuple would give a confusing error. We now raise immediately with a more helpful message.#9741: On Python 3.11, use the standard library’s
tomllib
to parse TOML.tomli
is no longer a dependency on Python 3.11.#9742: Display assertion message without escaped newline characters with
-vv
.#9823: Improved error message that is shown when no collector is found for a given file.
#9873: Some coloring has been added to the short test summary.
#9883: Normalize the help description of all command-line options.
#9920: Display full crash messages in
short test summary info
, when running in a CI environment.#9987: Added support for hidden configuration file by allowing
.pytest.ini
as an alternative topytest.ini
.
Bug Fixes¶
#10150:
sys.stdin
now contains all expected methods of a file-like object when capture is enabled.#10382: Do not break into pdb when
raise unittest.SkipTest()
appears top-level in a file.#7792: Marks are now inherited according to the full MRO in test classes. Previously, if a test class inherited from two or more classes, only marks from the first super-class would apply.
When inheriting marks from super-classes, marks from the sub-classes are now ordered before marks from the super-classes, in MRO order. Previously it was the reverse.
When inheriting marks from super-classes, the
pytestmark
attribute of the sub-class now only contains the marks directly applied to it. Previously, it also contained marks from its super-classes. Please note that this attribute should not normally be accessed directly; useNode.iter_markers
instead.#9159: Showing inner exceptions by forcing native display in
ExceptionGroups
even when using display options other than--tb=native
. A temporary step before full implementation of pytest-native display for inner exceptions inExceptionGroups
.#9877: Ensure
caplog.get_records(when)
returns current/correct data after invokingcaplog.clear()
.
Improved Documentation¶
#10344: Update information on writing plugins to use
pyproject.toml
instead ofsetup.py
.#9248: The documentation is now built using Sphinx 5.x (up from 3.x previously).
#9291: Update documentation on how
pytest.warns()
affectsDeprecationWarning
.
Trivial/Internal Changes¶
#10313: Made
_pytest.doctest.DoctestItem
exportpytest.DoctestItem
for type check and runtime purposes. Made_pytest.doctest
use internal APIs to avoid circular imports.#9906: Made
_pytest.compat
re-exportimportlib_metadata
in the eyes of type checkers.#9910: Fix default encoding warning (
EncodingWarning
) incacheprovider
#9984: Improve the error message when we attempt to access a fixture that has been torn down. Add an additional sentence to the docstring explaining when it’s not a good idea to call
getfixturevalue
.
pytest 7.1.3 (2022-08-31)¶
Bug Fixes¶
#10060: When running with
--pdb
,TestCase.tearDown
is no longer called for tests when the class has been skipped viaunittest.skip
orpytest.mark.skip
.#10190: Invalid XML characters in setup or teardown error messages are now properly escaped for JUnit XML reports.
#10230: Ignore
.py
files created bypyproject.toml
-based editable builds introduced in pip 21.3.#3396: Doctests now respect the
--import-mode
flag.#9514: Type-annotate
FixtureRequest.param
asAny
as a stop gap measure until #8073 is fixed.#9791: Fixed a path handling code in
rewrite.py
that seems to work fine, but was incorrect and fails in some systems.#9917: Fixed string representation for
pytest.approx()
when used to compare tuples.
Improved Documentation¶
Trivial/Internal Changes¶
#10114: Replace atomicwrites dependency on windows with
os.replace
.
pytest 7.1.2 (2022-04-23)¶
Bug Fixes¶
#9726: An unnecessary
numpy
import insidepytest.approx()
was removed.#9820: Fix comparison of
dataclasses
withInitVar
.#9869: Increase
stacklevel
for theNODE_CTOR_FSPATH_ARG
deprecation to point to the user’s code, not pytest.#9871: Fix a bizarre (and fortunately rare) bug where the
temp_path
fixture could raise an internal error while attempting to get the current user’s username.
pytest 7.1.1 (2022-03-17)¶
Bug Fixes¶
#9767: Fixed a regression in pytest 7.1.0 where some conftest.py files outside of the source tree (e.g. in the
site-packages
directory) were not picked up.
pytest 7.1.0 (2022-03-13)¶
Breaking Changes¶
#8838: As per our policy, the following features have been deprecated in the 6.X series and are now removed:
pytest._fillfuncargs
function.pytest_warning_captured
hook - usepytest_warning_recorded
instead.-k -foobar
syntax - use-k 'not foobar'
instead.-k foobar:
syntax.pytest.collect
module - import frompytest
directly.
For more information consult Deprecations and Removals in the docs.
#9437: Dropped support for Python 3.6, which reached end-of-life at 2021-12-23.
Improvements¶
#5192: Fixed test output for some data types where
-v
would show less information.Also, when showing diffs for sequences,
-q
would produce full diffs instead of the expected diff.#9362: pytest now avoids specialized assert formatting when it is detected that the default
__eq__
is overridden inattrs
ordataclasses
.#9536: When
-vv
is given on command line, show skipping and xfail reasons in full instead of truncating them to fit the terminal width.#9644: More information about the location of resources that led Python to raise
ResourceWarning
can now be obtained by enablingtracemalloc
.See Resource Warnings for more information.
#9678: More types are now accepted in the
ids
argument to@pytest.mark.parametrize
. Previously onlystr
,float
,int
andbool
were accepted; nowbytes
,complex
,re.Pattern
,Enum
and anything with a__name__
are also accepted.#9692:
pytest.approx()
now raises aTypeError
when given an unordered sequence (such asset
).Note that this implies that custom classes which only implement
__iter__
and__len__
are no longer supported as they don’t guarantee order.
Bug Fixes¶
#8242: The deprecation of raising
unittest.SkipTest
to skip collection of tests during the pytest collection phase is reverted - this is now a supported feature again.#9493: Symbolic link components are no longer resolved in conftest paths. This means that if a conftest appears twice in collection tree, using symlinks, it will be executed twice. For example, given
tests/real/conftest.py tests/real/test_it.py tests/link -> tests/real
running
pytest tests
now imports the conftest twice, once astests/real/conftest.py
and once astests/link/conftest.py
. This is a fix to match a similar change made to test collection itself in pytest 6.0 (see #6523 for details).#9626: Fixed count of selected tests on terminal collection summary when there were errors or skipped modules.
If there were errors or skipped modules on collection, pytest would mistakenly subtract those from the selected count.
#9645: Fixed regression where
--import-mode=importlib
used together withPYTHONPATH
orpythonpath
would cause import errors in test suites.#9708:
pytester
now requests amonkeypatch
fixture instead of creating one internally. This solves some issues with tests that involve pytest environment variables.#9730: Malformed
pyproject.toml
files now produce a clearer error message.
pytest 7.0.1 (2022-02-11)¶
Bug Fixes¶
#9608: Fix invalid importing of
importlib.readers
in Python 3.9.#9610: Restore
UnitTestFunction.obj
to return unbound rather than bound method. Fixes a crash during a failed teardown in unittest TestCases with non-default__init__
. Regressed in pytest 7.0.0.#9636: The
pythonpath
plugin was renamed topython_path
. This avoids a conflict with thepytest-pythonpath
plugin.#9642: Fix running tests by id with
::
in the parametrize portion.#9643: Delay issuing a
PytestWarning
about diamond inheritance involvingItem
andCollector
so it can be filtered using standard warning filters.
pytest 7.0.0 (2022-02-03)¶
(Please see the full set of changes for this release also in the 7.0.0rc1 notes below)
Deprecations¶
#9488: If custom subclasses of nodes like
pytest.Item
override the__init__
method, they should take**kwargs
. See Constructors of custom Node subclasses should take **kwargs for details.Note that a deprecation warning is only emitted when there is a conflict in the arguments pytest expected to pass. This deprecation was already part of pytest 7.0.0rc1 but wasn’t documented.
Bug Fixes¶
#9355: Fixed error message prints function decorators when using assert in Python 3.8 and above.
#9396: Ensure
pytest.Config.inifile
is available during thepytest_cmdline_main
hook (regression during7.0.0rc1
).
Improved Documentation¶
Trivial/Internal Changes¶
#9521: Add test coverage to assertion rewrite path.
pytest 7.0.0rc1 (2021-12-06)¶
Breaking Changes¶
#7259: The Node.reportinfo() function first return value type has been expanded from
py.path.local | str
toos.PathLike[str] | str
.Most plugins which refer to
reportinfo()
only define it as part of a custompytest.Item
implementation. Sincepy.path.local
is anos.PathLike[str]
, these plugins are unaffected.Plugins and users which call
reportinfo()
, use the first return value and interact with it as apy.path.local
, would need to adjust by callingpy.path.local(fspath)
. Although preferably, avoid the legacypy.path.local
and usepathlib.Path
, or useitem.location
oritem.path
, instead.Note: pytest was not able to provide a deprecation period for this change.
#8246:
--version
now writes version information tostdout
rather thanstderr
.#8733: Drop a workaround for pyreadline that made it work with
--pdb
.The workaround was introduced in #1281 in 2015, however since then pyreadline seems to have gone unmaintained, is generating warnings, and will stop working on Python 3.10.
#9061: Using
pytest.approx()
in a boolean context now raises an error hinting at the proper usage.It is apparently common for users to mistakenly use
pytest.approx
like this:assert pytest.approx(actual, expected)
While the correct usage is:
assert actual == pytest.approx(expected)
The new error message helps catch those mistakes.
#9277: The
pytest.Instance
collector type has been removed. Importingpytest.Instance
or_pytest.python.Instance
returns a dummy type and emits a deprecation warning. See The pytest.Instance collector for details.#9308: PytestRemovedIn7Warning deprecation warnings are now errors by default.
Following our plan to remove deprecated features with as little disruption as possible, all warnings of type
PytestRemovedIn7Warning
now generate errors instead of warning messages by default.The affected features will be effectively removed in pytest 7.1, so please consult the Deprecations and Removals section in the docs for directions on how to update existing code.
In the pytest
7.0.X
series, it is possible to change the errors back into warnings as a stopgap measure by adding this to yourpytest.ini
file:[pytest] filterwarnings = ignore::pytest.PytestRemovedIn7Warning
But this will stop working when pytest
7.1
is released.If you have concerns about the removal of a specific feature, please add a comment to #9308.
Deprecations¶
#7259:
py.path.local
arguments for hooks have been deprecated. See the deprecation note for full details.py.path.local
arguments to Node constructors have been deprecated. See the deprecation note for full details.Note
The name of the
Node
arguments and attributes (the new attribute beingpath
) is the opposite of the situation for hooks (the old argument beingpath
).This is an unfortunate artifact due to historical reasons, which should be resolved in future versions as we slowly get rid of the py dependency (see #9283 for a longer discussion).
#7469: Directly constructing the following classes is now deprecated:
_pytest.mark.structures.Mark
_pytest.mark.structures.MarkDecorator
_pytest.mark.structures.MarkGenerator
_pytest.python.Metafunc
_pytest.runner.CallInfo
_pytest._code.ExceptionInfo
_pytest.config.argparsing.Parser
_pytest.config.argparsing.OptionGroup
_pytest.pytester.HookRecorder
These constructors have always been considered private, but now issue a deprecation warning, which may become a hard error in pytest 8.
#8242: Raising
unittest.SkipTest
to skip collection of tests during the pytest collection phase is deprecated. Usepytest.skip()
instead.Note: This deprecation only relates to using
unittest.SkipTest
during test collection. You are probably not doing that. Ordinary usage ofunittest.SkipTest
/unittest.TestCase.skipTest()
/unittest.skip()
in unittest test cases is fully supported.Note
This deprecation has been reverted in pytest 7.1.0.
#8315: Several behaviors of
Parser.addoption
are now scheduled for removal in pytest 8 (deprecated since pytest 2.4.0):parser.addoption(..., help=".. %default ..")
- use%(default)s
instead.parser.addoption(..., type="int/string/float/complex")
- usetype=int
etc. instead.
#8447: Defining a custom pytest node type which is both an
Item
and aCollector
(e.g.File
) now issues a warning. It was never sanely supported and triggers hard to debug errors.See the deprecation note for full details.
#8592:
pytest_cmdline_preparse
has been officially deprecated. It will be removed in a future release. Usepytest_load_initial_conftests
instead.See the deprecation note for full details.
#8645:
pytest.warns(None)
is now deprecated because many people used it to mean “this code does not emit warnings”, but it actually had the effect of checking that the code emits at least one warning of any type - likepytest.warns()
orpytest.warns(Warning)
.#8948:
pytest.skip(msg=...)
,pytest.fail(msg=...)
andpytest.exit(msg=...)
signatures now accept areason
argument instead ofmsg
. Usingmsg
still works, but is deprecated and will be removed in a future release.This was changed for consistency with
pytest.mark.skip
andpytest.mark.xfail
which both acceptreason
as an argument.#8174: The following changes have been made to types reachable through
pytest.ExceptionInfo.traceback
:The
path
property of_pytest.code.Code
returnsPath
instead ofpy.path.local
.The
path
property of_pytest.code.TracebackEntry
returnsPath
instead ofpy.path.local
.
There was no deprecation period for this change (sorry!).
Features¶
#5196: Tests are now ordered by definition order in more cases.
In a class hierarchy, tests from base classes are now consistently ordered before tests defined on their subclasses (reverse MRO order).
#7132: Added two environment variables
PYTEST_THEME
andPYTEST_THEME_MODE
to let the users customize the pygments theme used.#7259: Added
cache.mkdir()
, which is similar to the existingcache.makedir()
, but returns apathlib.Path
instead of a legacypy.path.local
.Added a
paths
type toparser.addini()
, as inparser.addini("mypaths", "my paths", type="paths")
, which is similar to the existingpathlist
, but returns a list ofpathlib.Path
instead of legacypy.path.local
.#7469: The types of objects used in pytest’s API are now exported so they may be used in type annotations.
The newly-exported types are:
pytest.Config
forConfig
.pytest.Mark
formarks
.pytest.MarkDecorator
formark decorators
.pytest.MarkGenerator
for thepytest.mark
singleton.pytest.Metafunc
for themetafunc
argument to thepytest_generate_tests
hook.pytest.CallInfo
for theCallInfo
type passed to various hooks.pytest.PytestPluginManager
forPytestPluginManager
.pytest.ExceptionInfo
for theExceptionInfo
type returned frompytest.raises()
and passed to various hooks.pytest.Parser
for theParser
type passed to thepytest_addoption
hook.pytest.OptionGroup
for theOptionGroup
type returned from theparser.addgroup
method.pytest.HookRecorder
for theHookRecorder
type returned fromPytester
.pytest.RecordedHookCall
for theRecordedHookCall
type returned fromHookRecorder
.pytest.RunResult
for theRunResult
type returned fromPytester
.pytest.LineMatcher
for theLineMatcher
type used inRunResult
and others.pytest.TestReport
for theTestReport
type used in various hooks.pytest.CollectReport
for theCollectReport
type used in various hooks.
Constructing most of them directly is not supported; they are only meant for use in type annotations. Doing so will emit a deprecation warning, and may become a hard-error in pytest 8.0.
Subclassing them is also not supported. This is not currently enforced at runtime, but is detected by type-checkers such as mypy.
#7856: –import-mode=importlib now works with features that depend on modules being on
sys.modules
, such aspickle
anddataclasses
.#8144: The following hooks now receive an additional
pathlib.Path
argument, equivalent to an existingpy.path.local
argument:pytest_ignore_collect
- Thecollection_path
parameter (equivalent to existingpath
parameter).pytest_collect_file
- Thefile_path
parameter (equivalent to existingpath
parameter).pytest_pycollect_makemodule
- Themodule_path
parameter (equivalent to existingpath
parameter).pytest_report_header
- Thestart_path
parameter (equivalent to existingstartdir
parameter).pytest_report_collectionfinish
- Thestart_path
parameter (equivalent to existingstartdir
parameter).
Note
The name of the
Node
arguments and attributes (the new attribute beingpath
) is the opposite of the situation for hooks (the old argument beingpath
).This is an unfortunate artifact due to historical reasons, which should be resolved in future versions as we slowly get rid of the py dependency (see #9283 for a longer discussion).
#8251: Implement
Node.path
as apathlib.Path
. Both the oldfspath
and this new attribute gets set no matter whetherpath
orfspath
(deprecated) is passed to the constructor. It is a replacement for thefspath
attribute (which represents the same path aspy.path.local
). Whilefspath
is not deprecated yet due to the ongoing migration of methods likereportinfo()
, we expect to deprecate it in a future release.Note
The name of the
Node
arguments and attributes (the new attribute beingpath
) is the opposite of the situation for hooks (the old argument beingpath
).This is an unfortunate artifact due to historical reasons, which should be resolved in future versions as we slowly get rid of the py dependency (see #9283 for a longer discussion).
#8421:
pytest.approx()
now works onDecimal
within mappings/dicts and sequences/lists.#8606: pytest invocations with
--fixtures-per-test
and--fixtures
have been enriched with:Fixture location path printed with the fixture name.
First section of the fixture’s docstring printed under the fixture name.
Whole of fixture’s docstring printed under the fixture name using
--verbose
option.
#8761: New pytest.version_tuple attribute, which makes it simpler for users to do something depending on the pytest version (such as declaring hooks which are introduced in later versions).
#8789: Switch TOML parser from
toml
totomli
for TOML v1.0.0 support inpyproject.toml
.#8920: Added
pytest.Stash
, a facility for plugins to store their data onConfig
andNode
s in a type-safe and conflict-free manner. See Storing data on items across hook functions for details.#8953:
RunResult
methodassert_outcomes()
now accepts awarnings
argument to assert the total number of warnings captured.#8954:
--debug
flag now accepts astr
file to route debug logs into, remains defaulted topytestdebug.log
.#9023: Full diffs are now always shown for equality assertions of iterables when
CI
orBUILD_NUMBER
is found in the environment, even when-v
isn’t used.#9113:
RunResult
methodassert_outcomes()
now accepts adeselected
argument to assert the total number of deselected tests.#9114: Added
pythonpath
setting that adds listed paths tosys.path
for the duration of the test session. If you currently use the pytest-pythonpath or pytest-srcpaths plugins, you should be able to replace them with built-inpythonpath
setting.
Improvements¶
#7480: A deprecation scheduled to be removed in a major version X (e.g. pytest 7, 8, 9, …) now uses warning category
PytestRemovedInXWarning
, a subclass ofPytestDeprecationWarning
, instead ofPytestDeprecationWarning
directly.See Backwards Compatibility Policy for more details.
#7864: Improved error messages when parsing warning filters.
Previously pytest would show an internal traceback, which besides being ugly sometimes would hide the cause of the problem (for example an
ImportError
while importing a specific warning type).#8335: Improved
pytest.approx()
assertion messages for sequences of numbers.The assertion messages now dumps a table with the index and the error of each diff. Example:
> assert [1, 2, 3, 4] == pytest.approx([1, 3, 3, 5]) E assert comparison failed for 2 values: E Index | Obtained | Expected E 1 | 2 | 3 +- 3.0e-06 E 3 | 4 | 5 +- 5.0e-06
#8403: By default, pytest will truncate long strings in assert errors so they don’t clutter the output too much, currently at
240
characters by default.However, in some cases the longer output helps, or is even crucial, to diagnose a failure. Using
-v
will now increase the truncation threshold to2400
characters, and-vv
or higher will disable truncation entirely.#8509: Fixed issue where
unittest.TestCase.setUpClass()
is not called when a test has/
in its name since pytest 6.2.0.This refers to the path part in pytest node IDs, e.g.
TestClass::test_it
in the node IDtests/test_file.py::TestClass::test_it
.Now, instead of assuming that the test name does not contain
/
, it is assumed that test path does not contain::
. We plan to hopefully make both of these work in the future.#8803: It is now possible to add colors to custom log levels on cli log.
By using
add_color_level
from apytest_configure
hook, colors can be added:logging_plugin = config.pluginmanager.get_plugin('logging-plugin') logging_plugin.log_cli_handler.formatter.add_color_level(logging.INFO, 'cyan') logging_plugin.log_cli_handler.formatter.add_color_level(logging.SPAM, 'blue')
See Customizing Colors for more information.
#8822: When showing fixture paths in
--fixtures
or--fixtures-by-test
, fixtures coming from pytest itself now display an elided path, rather than the full path to the file in thesite-packages
directory.#8898: Complex numbers are now treated like floats and integers when generating parameterization IDs.
#9062:
--stepwise-skip
now implicitly enables--stepwise
and can be used on its own.#9205:
pytest.Cache.set()
now preserves key order when saving dicts.
Bug Fixes¶
#7124: Fixed an issue where
__main__.py
would raise anImportError
when--doctest-modules
was provided.#8061: Fixed failing
staticmethod
test cases if they are inherited from a parent test class.#8192:
testdir.makefile
now silently accepts values which don’t start with.
to maintain backward compatibility with older pytest versions.pytester.makefile
now issues a clearer error if the.
is missing in theext
argument.#8258: Fixed issue where pytest’s
faulthandler
support would not dump traceback on crashes if thefaulthandler
module was already enabled during pytest startup (usingpython -X dev -m pytest
for example).#8317: Fixed an issue where illegal directory characters derived from
getpass.getuser()
raised anOSError
.#8367: Fix
Class.from_parent
so it forwards extra keyword arguments to the constructor.#8377: The test selection options
pytest -k
andpytest -m
now support matching names containing forward slash (/
) characters.#8384: The
@pytest.mark.skip
decorator now correctly handles its arguments. When thereason
argument is accidentally given both positional and as a keyword (e.g. because it was confused withskipif
), aTypeError
now occurs. Before, such tests were silently skipped, and the positional argument ignored. Additionally,reason
is now documented correctly as positional or keyword (rather than keyword-only).#8394: Use private names for internal fixtures that handle classic setup/teardown so that they don’t show up with the default
--fixtures
invocation (but they still show up with--fixtures -v
).#8456: The
required_plugins
config option now works correctly when pre-releases of plugins are installed, rather than falsely claiming that those plugins aren’t installed at all.#8464:
-c <config file>
now also properly definesrootdir
as the directory that contains<config file>
.#8503:
pytest.MonkeyPatch.syspath_prepend()
no longer fails whensetuptools
is not installed. It now only callspkg_resources.fixup_namespace_packages
ifpkg_resources
was previously imported, because it is not needed otherwise.#8548: Introduce fix to handle precision width in
log-cli-format
in turn to fix output coloring for certain formats.#8796: Fixed internal error when skipping doctests.
#8983: The test selection options
pytest -k
andpytest -m
now support matching names containing backslash (\
) characters. Backslashes are treated literally, not as escape characters (the values being matched against are already escaped).#8990: Fix
pytest -vv
crashing with an internal exceptionAttributeError: 'str' object has no attribute 'relative_to'
in some cases.#9077: Fixed confusing error message when
request.fspath
/request.path
was accessed from a session-scoped fixture.#9163: The end line number and end column offset are now properly set for rewritten assert statements.
#9169: Support for the
files
API fromimportlib.resources
within rewritten files.#9272: The nose compatibility module-level fixtures
setup()
andteardown()
are now only called once per module, instead of for each test function. They are now called even if object-levelsetup
/teardown
is defined.
Improved Documentation¶
#4320: Improved docs for
pytester.copy_example
.#5105: Add automatically generated Pytest Plugin List. The list is updated on a periodic schedule.
#8337: Recommend numpy.testing module on
pytest.approx()
documentation.#8655: Help text for
--pdbcls
more accurately reflects the option’s behavior.#9210: Remove incorrect docs about
confcutdir
being a configuration option: it can only be set through the--confcutdir
command-line option.#9242: Upgrade readthedocs configuration to use a
newer Ubuntu version <https://blog.readthedocs.com/new-build-specification/>`__
with better unicode support for PDF docs.#9341: Various methods commonly used for Working with non-python tests are now correctly documented in the reference docs. They were undocumented previously.
Trivial/Internal Changes¶
#8133: Migrate to
setuptools_scm
6.x to useSETUPTOOLS_SCM_PRETEND_VERSION_FOR_PYTEST
for more robust release tooling.#8174: The following changes have been made to internal pytest types/functions:
The
_pytest.code.getfslineno()
function returnsPath
instead ofpy.path.local
.The
_pytest.python.path_matches_patterns()
function takesPath
instead ofpy.path.local
.The
_pytest._code.Traceback.cut()
function accepts anyos.PathLike[str]
, not justpy.path.local
.
#8248: Internal Restructure: let
python.PyObjMixin
inherit fromnodes.Node
to carry over typing information.#8432: Improve error message when
pytest.skip()
is used at module level without passingallow_module_level=True
.#8818: Ensure
regendoc
opts out ofTOX_ENV
cachedir selection to ensure independent example test runs.#8913: The private
CallSpec2._arg2scopenum
attribute has been removed after an internal refactoring.#8967:
pytest_assertion_pass
is no longer considered experimental and future changes to it will be considered more carefully.#9202: Add github action to upload coverage report to codecov instead of bash uploader.
#9225: Changed the command used to create sdist and wheel artifacts: using the build package instead of setup.py.
#9351: Correct minor typos in doc/en/example/special.rst.
pytest 6.2.5 (2021-08-29)¶
Trivial/Internal Changes¶
pytest 6.2.4 (2021-05-04)¶
Bug Fixes¶
#8539: Fixed assertion rewriting on Python 3.10.
pytest 6.2.3 (2021-04-03)¶
Bug Fixes¶
#8414: pytest used to create directories under
/tmp
with world-readable permissions. This means that any user in the system was able to read information written by tests in temporary directories (such as those created by thetmp_path
/tmpdir
fixture). Now the directories are created with private permissions.pytest used to silently use a preexisting
/tmp/pytest-of-<username>
directory, even if owned by another user. This means another user could pre-create such a directory and gain control of another user’s temporary directory. Now such a condition results in an error.
pytest 6.2.2 (2021-01-25)¶
Bug Fixes¶
pytest 6.2.1 (2020-12-15)¶
Bug Fixes¶
#7678: Fixed bug where
ImportPathMismatchError
would be raised for files compiled in the host and loaded later from an UNC mounted path (Windows).#8132: Fixed regression in
approx
: in 6.2.0approx
no longer raisesTypeError
when dealing with non-numeric types, falling back to normal comparison. Before 6.2.0, array types like tf.DeviceArray fell through to the scalar case, and happened to compare correctly to a scalar if they had only one element. After 6.2.0, these types began failing, because they inherited neither from standard Python number hierarchy nor fromnumpy.ndarray
.approx
now converts arguments tonumpy.ndarray
if they expose the array protocol and are not scalars. This treats array-like objects like numpy arrays, regardless of size.
pytest 6.2.0 (2020-12-12)¶
Breaking Changes¶
#7808: pytest now supports python3.6+ only.
Deprecations¶
#7469: Directly constructing/calling the following classes/functions is now deprecated:
_pytest.cacheprovider.Cache
_pytest.cacheprovider.Cache.for_config()
_pytest.cacheprovider.Cache.clear_cache()
_pytest.cacheprovider.Cache.cache_dir_from_config()
_pytest.capture.CaptureFixture
_pytest.fixtures.FixtureRequest
_pytest.fixtures.SubRequest
_pytest.logging.LogCaptureFixture
_pytest.pytester.Pytester
_pytest.pytester.Testdir
_pytest.recwarn.WarningsRecorder
_pytest.recwarn.WarningsChecker
_pytest.tmpdir.TempPathFactory
_pytest.tmpdir.TempdirFactory
These have always been considered private, but now issue a deprecation warning, which may become a hard error in pytest 8.0.0.
#7530: The
--strict
command-line option has been deprecated, use--strict-markers
instead.We have plans to maybe in the future to reintroduce
--strict
and make it an encompassing flag for all strictness related options (--strict-markers
and--strict-config
at the moment, more might be introduced in the future).#7988: The
@pytest.yield_fixture
decorator/function is now deprecated. Usepytest.fixture()
instead.yield_fixture
has been an alias forfixture
for a very long time, so can be search/replaced safely.
Features¶
#5299: pytest now warns about unraisable exceptions and unhandled thread exceptions that occur in tests on Python>=3.8. See Warning about unraisable exceptions and unhandled thread exceptions for more information.
#7425: New
pytester
fixture, which is identical totestdir
but its methods returnpathlib.Path
when appropriate instead ofpy.path.local
.This is part of the movement to use
pathlib.Path
objects internally, in order to remove the dependency topy
in the future.Internally, the old
pytest.Testdir
is now a thin wrapper aroundPytester
, preserving the old interface.#7695: A new hook was added,
pytest_markeval_namespace
which should return a dictionary. This dictionary will be used to augment the “global” variables available to evaluate skipif/xfail/xpass markers.Pseudo example
conftest.py
:def pytest_markeval_namespace(): return {"color": "red"}
test_func.py
:@pytest.mark.skipif("color == 'blue'", reason="Color is not red") def test_func(): assert False
#8006: It is now possible to construct a
MonkeyPatch
object directly aspytest.MonkeyPatch()
, in cases when themonkeypatch
fixture cannot be used. Previously some users imported it from the private_pytest.monkeypatch.MonkeyPatch
namespace.Additionally,
MonkeyPatch.context
is now a classmethod, and can be used aswith MonkeyPatch.context() as mp: ...
. This is the recommended way to useMonkeyPatch
directly, since unlike themonkeypatch
fixture, an instance created directly is notundo()
-ed automatically.
Improvements¶
#1265: Added an
__str__
implementation to theLineMatcher
class which is returned frompytester.run_pytest().stdout
and similar. It returns the entire output, like the existingstr()
method.#2044: Verbose mode now shows the reason that a test was skipped in the test’s terminal line after the “SKIPPED”, “XFAIL” or “XPASS”.
#7469 The types of builtin pytest fixtures are now exported so they may be used in type annotations of test functions. The newly-exported types are:
pytest.FixtureRequest
for therequest
fixture.pytest.Cache
for thecache
fixture.pytest.CaptureFixture[str]
for thecapfd
andcapsys
fixtures.pytest.CaptureFixture[bytes]
for thecapfdbinary
andcapsysbinary
fixtures.pytest.LogCaptureFixture
for thecaplog
fixture.pytest.Pytester
for thepytester
fixture.pytest.Testdir
for thetestdir
fixture.pytest.TempdirFactory
for thetmpdir_factory
fixture.pytest.TempPathFactory
for thetmp_path_factory
fixture.pytest.MonkeyPatch
for themonkeypatch
fixture.pytest.WarningsRecorder
for therecwarn
fixture.
Constructing them is not supported (except for
MonkeyPatch
); they are only meant for use in type annotations. Doing so will emit a deprecation warning, and may become a hard-error in pytest 8.0.Subclassing them is also not supported. This is not currently enforced at runtime, but is detected by type-checkers such as mypy.
#7527: When a comparison between
namedtuple
instances of the same type fails, pytest now shows the differing field names (possibly nested) instead of their indexes.#7615:
Node.warn
now permits any subclass ofWarning
, not justPytestWarning
.#7701: Improved reporting when using
--collected-only
. It will now show the number of collected tests in the summary stats.#7710: Use strict equality comparison for non-numeric types in
pytest.approx()
instead of raisingTypeError
.This was the undocumented behavior before 3.7, but is now officially a supported feature.
#7938: New
--sw-skip
argument which is a shorthand for--stepwise-skip
.#8023: Added
'node_modules'
to default value fornorecursedirs
.#8032:
doClassCleanups
(introduced inunittest
in Python and 3.8) is now called appropriately.
Bug Fixes¶
#4824: Fixed quadratic behavior and improved performance of collection of items using autouse fixtures and xunit fixtures.
#7758: Fixed an issue where some files in packages are getting lost from
--lf
even though they contain tests that failed. Regressed in pytest 5.4.0.#7911: Directories created by by
tmp_path
andtmpdir
are now considered stale after 3 days without modification (previous value was 3 hours) to avoid deleting directories still in use in long running test suites.#7913: Fixed a crash or hang in
pytester.spawn
when thereadline
module is involved.#7951: Fixed handling of recursive symlinks when collecting tests.
#7981: Fixed symlinked directories not being followed during collection. Regressed in pytest 6.1.0.
#8016: Fixed only one doctest being collected when using
pytest --doctest-modules path/to/an/__init__.py
.
Improved Documentation¶
#7429: Add more information and use cases about skipping doctests.
#7780: Classes which should not be inherited from are now marked
final class
in the API reference.#7872:
_pytest.config.argparsing.Parser.addini()
accepts explicitNone
and"string"
.#7878: In pull request section, ask to commit after editing changelog and authors file.
Trivial/Internal Changes¶
pytest 6.1.2 (2020-10-28)¶
Bug Fixes¶
#7758: Fixed an issue where some files in packages are getting lost from
--lf
even though they contain tests that failed. Regressed in pytest 5.4.0.#7911: Directories created by
tmpdir
are now considered stale after 3 days without modification (previous value was 3 hours) to avoid deleting directories still in use in long running test suites.
Improved Documentation¶
#7815: Improve deprecation warning message for
pytest._fillfuncargs()
.
pytest 6.1.1 (2020-10-03)¶
Bug Fixes¶
pytest 6.1.0 (2020-09-26)¶
Breaking Changes¶
#5585: As per our policy, the following features which have been deprecated in the 5.X series are now removed:
The
funcargnames
read-only property ofFixtureRequest
,Metafunc
, andFunction
classes. Usefixturenames
attribute.@pytest.fixture
no longer supports positional arguments, pass all arguments by keyword instead.Direct construction of
Node
subclasses now raise an error, usefrom_parent
instead.The default value for
junit_family
has changed toxunit2
. If you require the old format, addjunit_family=xunit1
to your configuration file.The
TerminalReporter
no longer has awriter
attribute. Plugin authors may use the public functions of theTerminalReporter
instead of accessing theTerminalWriter
object directly.The
--result-log
option has been removed. Users are recommended to use the pytest-reportlog plugin instead.
For more information consult Deprecations and Removals in the docs.
Deprecations¶
#6981: The
pytest.collect
module is deprecated: all its names can be imported frompytest
directly.#7097: The
pytest._fillfuncargs
function is deprecated. This function was kept for backward compatibility with an older plugin.It’s functionality is not meant to be used directly, but if you must replace it, use
function._request._fillfixtures()
instead, though note this is not a public API and may break in the future.#7210: The special
-k '-expr'
syntax to-k
is deprecated. Use-k 'not expr'
instead.The special
-k 'expr:'
syntax to-k
is deprecated. Please open an issue if you use this and want a replacement.#7255: The
pytest_warning_captured
hook is deprecated in favor ofpytest_warning_recorded
, and will be removed in a future version.#7648: The
gethookproxy()
andisinitpath()
methods ofFSCollector
andPackage
are deprecated; useself.session.gethookproxy()
andself.session.isinitpath()
instead. This should work on all pytest versions.
Features¶
#7667: New
--durations-min
command-line flag controls the minimal duration for inclusion in the slowest list of tests shown by--durations
. Previously this was hard-coded to0.005s
.
Improvements¶
#6681: Internal pytest warnings issued during the early stages of initialization are now properly handled and can filtered through
filterwarnings
or--pythonwarnings/-W
.This also fixes a number of long standing issues: #2891, #7620, #7426.
#7572: When a plugin listed in
required_plugins
is missing or an unknown config key is used with--strict-config
, a simple error message is now shown instead of a stacktrace.#7685: Added two new attributes
rootpath
andinipath
toConfig
. These attributes arepathlib.Path
versions of the existingrootdir
andinifile
attributes, and should be preferred over them when possible.#7780: Public classes which are not designed to be inherited from are now marked
@final
. Code which inherits from these classes will trigger a type-checking (e.g. mypy) error, but will still work in runtime. Currently thefinal
designation does not appear in the API Reference but hopefully will in the future.
Bug Fixes¶
#1953: Fixed error when overwriting a parametrized fixture, while also reusing the super fixture value.
# conftest.py import pytest @pytest.fixture(params=[1, 2]) def foo(request): return request.param # test_foo.py import pytest @pytest.fixture def foo(foo): return foo * 2
#4984: Fixed an internal error crash with
IndexError: list index out of range
when collecting a module which starts with a decorated function, the decorator raises, and assertion rewriting is enabled.#7591: pylint shouldn’t complain anymore about unimplemented abstract methods when inheriting from File.
#7628: Fixed test collection when a full path without a drive letter was passed to pytest on Windows (for example
\projects\tests\test.py
instead ofc:\projects\tests\pytest.py
).#7638: Fix handling of command-line options that appear as paths but trigger an OS-level syntax error on Windows, such as the options used internally by
pytest-xdist
.#7742: Fixed INTERNALERROR when accessing locals / globals with faulty
exec
.
Improved Documentation¶
#1477: Removed faq.rst and its reference in contents.rst.
Trivial/Internal Changes¶
#7536: The internal
junitxml
plugin has rewritten to usexml.etree.ElementTree
. The order of attributes in XML elements might differ. Some unneeded escaping is no longer performed.#7587: The dependency on the
more-itertools
package has been removed.#7631: The result type of
capfd.readouterr()
(and similar) is no longer a namedtuple, but should behave like one in all respects. This was done for technical reasons.#7671: When collecting tests, pytest finds test classes and functions by examining the attributes of python objects (modules, classes and instances). To speed up this process, pytest now ignores builtin attributes (like
__class__
,__delattr__
and__new__
) without consulting thepython_classes
andpython_functions
configuration options and without passing them to plugins using thepytest_pycollect_makeitem
hook.
pytest 6.0.2 (2020-09-04)¶
Bug Fixes¶
#7148: Fixed
--log-cli
potentially causing unrelatedprint
output to be swallowed.#7672: Fixed log-capturing level restored incorrectly if
caplog.set_level
is called more than once.#7686: Fixed
NotSetType.token
being used as the parameter ID when the parametrization list is empty. Regressed in pytest 6.0.0.#7707: Fix internal error when handling some exceptions that contain multiple lines or the style uses multiple lines (
--tb=line
for example).
pytest 6.0.1 (2020-07-30)¶
Bug Fixes¶
#7394: Passing an empty
help
value toParser.add_option
is now accepted instead of crashing when runningpytest --help
. PassingNone
raises a more informativeTypeError
.#7558: Fix pylint
not-callable
lint onpytest.mark.parametrize()
and the other builtin marks:skip
,skipif
,xfail
,usefixtures
,filterwarnings
.#7559: Fix regression in plugins using
TestReport.longreprtext
(such aspytest-html
) whenTestReport.longrepr
is not a string.#7569: Fix logging capture handler’s level not reset on teardown after a call to
caplog.set_level()
.
pytest 6.0.0 (2020-07-28)¶
(Please see the full set of changes for this release also in the 6.0.0rc1 notes below)
Breaking Changes¶
#5584: PytestDeprecationWarning are now errors by default.
Following our plan to remove deprecated features with as little disruption as possible, all warnings of type
PytestDeprecationWarning
now generate errors instead of warning messages.The affected features will be effectively removed in pytest 6.1, so please consult the Deprecations and Removals section in the docs for directions on how to update existing code.
In the pytest
6.0.X
series, it is possible to change the errors back into warnings as a stopgap measure by adding this to yourpytest.ini
file:[pytest] filterwarnings = ignore::pytest.PytestDeprecationWarning
But this will stop working when pytest
6.1
is released.If you have concerns about the removal of a specific feature, please add a comment to #5584.
#7472: The
exec_()
andis_true()
methods of_pytest._code.Frame
have been removed.
Features¶
#7464: Added support for
NO_COLOR
andFORCE_COLOR
environment variables to control colored output.
Improvements¶
#7467:
--log-file
CLI option andlog_file
ini marker now create subdirectories if needed.#7489: The
pytest.raises()
function has a clearer error message whenmatch
equals the obtained string but is not a regex match. In this case it is suggested to escape the regex.
Bug Fixes¶
#7392: Fix the reported location of tests skipped with
@pytest.mark.skip
when--runxfail
is used.#7491:
tmpdir
andtmp_path
no longer raise an error if the lock to check for stale temporary directories is not accessible.#7517: Preserve line endings when captured via
capfd
.#7534: Restored the previous formatting of
TracebackEntry.__str__
which was changed by accident.
Improved Documentation¶
Trivial/Internal Changes¶
#7389: Fixture scope
package
is no longer considered experimental.
pytest 6.0.0rc1 (2020-07-08)¶
Breaking Changes¶
#1316:
TestReport.longrepr
is now always an instance ofReprExceptionInfo
. Previously it was astr
when a test failed withpytest.fail(..., pytrace=False)
.#5965: symlinks are no longer resolved during collection and matching
conftest.py
files with test file paths.Resolving symlinks for the current directory and during collection was introduced as a bugfix in 3.9.0, but it actually is a new feature which had unfortunate consequences in Windows and surprising results in other platforms.
The team decided to step back on resolving symlinks at all, planning to review this in the future with a more solid solution (see discussion in #6523 for details).
This might break test suites which made use of this feature; the fix is to create a symlink for the entire test tree, and not only to partial files/tress as it was possible previously.
#6505:
Testdir.run().parseoutcomes()
now always returns the parsed nouns in plural form.Originally
parseoutcomes()
would always returns the nouns in plural form, but a change meant to improve the terminal summary by using singular form single items (1 warning
or1 error
) caused an unintended regression by changing the keys returned byparseoutcomes()
.Now the API guarantees to always return the plural form, so calls like this:
result = testdir.runpytest() result.assert_outcomes(error=1)
Need to be changed to:
result = testdir.runpytest() result.assert_outcomes(errors=1)
#6903: The
os.dup()
function is now assumed to exist. We are not aware of any supported Python 3 implementations which do not provide it.#7040:
-k
no longer matches against the names of the directories outside the test session root.Also,
pytest.Package.name
is now just the name of the directory containing the package’s__init__.py
file, instead of the full path. This is consistent with how the other nodes are named, and also one of the reasons why-k
would match against any directory containing the test suite.#7122: Expressions given to the
-m
and-k
options are no longer evaluated using Python’seval()
. The format supportsor
,and
,not
, parenthesis and general identifiers to match against. Python constants, keywords or other operators are no longer evaluated differently.#7135: Pytest now uses its own
TerminalWriter
class instead of using the one from thepy
library. Plugins generally access this class throughTerminalReporter.writer
,TerminalReporter.write()
(and similar methods), or_pytest.config.create_terminal_writer()
.The following breaking changes were made:
Output (
write()
method and others) no longer flush implicitly; the flushing behavior of the underlying file is respected. To flush explicitly (for example, if you want output to be shown before an end-of-line is printed), usewrite(flush=True)
orterminal_writer.flush()
.Explicit Windows console support was removed, delegated to the colorama library.
Support for writing
bytes
was removed.The
reline
method andchars_on_current_line
property were removed.The
stringio
andencoding
arguments was removed.Support for passing a callable instead of a file was removed.
#7224: The
item.catch_log_handler
anditem.catch_log_handlers
attributes, set by the logging plugin and never meant to be public, are no longer available.The deprecated
--no-print-logs
option andlog_print
ini option are removed. Use--show-capture
instead.#7226: Removed the unused
args
parameter frompytest.Function.__init__
.#7418: Removed the
pytest_doctest_prepare_content
hook specification. This hook hasn’t been triggered by pytest for at least 10 years.#7438: Some changes were made to the internal
_pytest._code.source
, listed here for the benefit of plugin authors who may be using it:The
deindent
argument toSource()
has been removed, now it is always true.Support for zero or multiple arguments to
Source()
has been removed.Support for comparing
Source
with anstr
has been removed.The methods
Source.isparseable()
andSource.putaround()
have been removed.The method
Source.compile()
and function_pytest._code.compile()
have been removed; use plaincompile()
instead.The function
_pytest._code.source.getsource()
has been removed; useSource()
directly instead.
Deprecations¶
Features¶
#1556: pytest now supports
pyproject.toml
files for configuration.The configuration options is similar to the one available in other formats, but must be defined in a
[tool.pytest.ini_options]
table to be picked up by pytest:# pyproject.toml [tool.pytest.ini_options] minversion = "6.0" addopts = "-ra -q" testpaths = [ "tests", "integration", ]
More information can be found in the docs.
#3342: pytest now includes inline type annotations and exposes them to user programs. Most of the user-facing API is covered, as well as internal code.
If you are running a type checker such as mypy on your tests, you may start noticing type errors indicating incorrect usage. If you run into an error that you believe to be incorrect, please let us know in an issue.
The types were developed against mypy version 0.780. Versions before 0.750 are known not to work. We recommend using the latest version. Other type checkers may work as well, but they are not officially verified to work by pytest yet.
#4049: Introduced a new hook named
pytest_warning_recorded
to convey information about warnings captured by the internalpytest
warnings plugin.This hook is meant to replace
pytest_warning_captured
, which is deprecated and will be removed in a future release.#6471: New command-line flags:
--no-header
: disables the initial header, including platform, version, and plugins.--no-summary
: disables the final test summary, including warnings.
#6856: A warning is now shown when an unknown key is read from a config INI file.
The
--strict-config
flag has been added to treat these warnings as errors.#6906: Added
--code-highlight
command line option to enable/disable code highlighting in terminal output.#7245: New
--import-mode=importlib
option that usesimportlib
to import test modules.Traditionally pytest used
__import__
while changingsys.path
to import test modules (which also changessys.modules
as a side-effect), which works but has a number of drawbacks, like requiring test modules that don’t live in packages to have unique names (as they need to reside under a unique name insys.modules
).--import-mode=importlib
uses more fine-grained import mechanisms fromimportlib
which don’t require pytest to changesys.path
orsys.modules
at all, eliminating much of the drawbacks of the previous mode.We intend to make
--import-mode=importlib
the default in future versions, so users are encouraged to try the new mode and provide feedback (both positive or negative) in issue #7245.You can read more about this option in the documentation.
#7305: New
required_plugins
configuration option allows the user to specify a list of plugins, including version information, that are required for pytest to run. An error is raised if any required plugins are not found when running pytest.
Improvements¶
#4375: The
pytest
command now suppresses theBrokenPipeError
error message that is printed to stderr when the output ofpytest
is piped and the pipe is closed by the piped-to program (common examples areless
andhead
).#4391: Improved precision of test durations measurement.
CallInfo
items now have a new<CallInfo>.duration
attribute, created usingtime.perf_counter()
. This attribute is used to fill the<TestReport>.duration
attribute, which is more accurate than the previous<CallInfo>.stop - <CallInfo>.start
(as these are based ontime.time()
).#4675: Rich comparison for dataclasses and
attrs
-classes is now recursive.#6285: Exposed the
pytest.FixtureLookupError
exception which is raised byrequest.getfixturevalue()
(whererequest
is aFixtureRequest
fixture) when a fixture with the given name cannot be returned.#6433: If an error is encountered while formatting the message in a logging call, for example
logging.warning("oh no!: %s: %s", "first")
(a second argument is missing), pytest now propagates the error, likely causing the test to fail.Previously, such a mistake would cause an error to be printed to stderr, which is not displayed by default for passing tests. This change makes the mistake visible during testing.
You may suppress this behavior temporarily or permanently by setting
logging.raiseExceptions = False
.#6817: Explicit new-lines in help texts of command-line options are preserved, allowing plugins better control of the help displayed to users.
#6940: When using the
--duration
option, the terminal message output is now more precise about the number and duration of hidden items.#6991: Collected files are displayed after any reports from hooks, e.g. the status from
--lf
.#7091: When
fd
capturing is used, through--capture=fd
or thecapfd
andcapfdbinary
fixtures, and the file descriptor (0, 1, 2) cannot be duplicated, FD capturing is still performed. Previously, direct writes to the file descriptors would fail or be lost in this case.#7119: Exit with an error if the
--basetemp
argument is empty, is the current working directory or is one of the parent directories. This is done to protect against accidental data loss, as any directory passed to this argument is cleared.#7128:
pytest --version
now displays just the pytest version, whilepytest --version --version
displays more verbose information including plugins. This is more consistent with how other tools show--version
.#7133:
caplog.set_level()
will now override anylog_level
set via the CLI or configuration file.#7159:
caplog.set_level()
andcaplog.at_level()
no longer affect the level of logs that are shown in the Captured log report report section.#7348: Improve recursive diff report for comparison asserts on dataclasses / attrs.
#7385:
--junitxml
now includes the exception cause in themessage
XML attribute for failures during setup and teardown.Previously:
<error message="test setup failure">
Now:
<error message="failed on setup with "ValueError: Some error during setup"">
Bug Fixes¶
#1120: Fix issue where directories from
tmpdir
are not removed properly when multiple instances of pytest are running in parallel.#4583: Prevent crashing and provide a user-friendly error when a marker expression (
-m
) invoking ofeval()
raises any exception.#4677: The path shown in the summary report for SKIPPED tests is now always relative. Previously it was sometimes absolute.
#5456: Fix a possible race condition when trying to remove lock files used to control access to folders created by
tmp_path
andtmpdir
.#6240: Fixes an issue where logging during collection step caused duplication of log messages to stderr.
#6428: Paths appearing in error messages are now correct in case the current working directory has changed since the start of the session.
#6755: Support deleting paths longer than 260 characters on windows created inside
tmpdir
.#6871: Fix crash with captured output when using
capsysbinary
.#6909: Revert the change introduced by #6330, which required all arguments to
@pytest.mark.parametrize
to be explicitly defined in the function signature.The intention of the original change was to remove what was expected to be an unintended/surprising behavior, but it turns out many people relied on it, so the restriction has been reverted.
#6910: Fix crash when plugins return an unknown stats while using the
--reportlog
option.#6924: Ensure a
unittest.IsolatedAsyncioTestCase
is actually awaited.#6925: Fix
TerminalRepr
instances to be hashable again.#6947: Fix regression where functions registered with
unittest.TestCase.addCleanup()
were not being called on test failures.#6951: Allow users to still set the deprecated
TerminalReporter.writer
attribute.#6956: Prevent pytest from printing
ConftestImportFailure
traceback to stdout.#6991: Fix regressions with
--lf
filtering too much since pytest 5.4.#6992: Revert “tmpdir: clean up indirection via config for factories” #6767 as it breaks pytest-xdist.
#7061: When a yielding fixture fails to yield a value, report a test setup error instead of crashing.
#7076: The path of file skipped by
@pytest.mark.skip
in the SKIPPED report is now relative to invocation directory. Previously it was relative to root directory.#7110: Fixed regression:
asyncbase.TestCase
tests are executed correctly again.#7126:
--setup-show
now doesn’t raise an error when a bytes value is used as aparametrize
parameter when Python is called with the-bb
flag.#7143: Fix
pytest.File.from_parent
so it forwards extra keyword arguments to the constructor.#7145: Classes with broken
__getattribute__
methods are displayed correctly during failures.#7150: Prevent hiding the underlying exception when
ConfTestImportFailure
is raised.#7180: Fix
_is_setup_py
for files encoded differently than locale.#7215: Fix regression where running with
--pdb
would callunittest.TestCase.tearDown()
for skipped tests.#7253: When using
pytest.fixture
on a function directly, as inpytest.fixture(func)
, if theautouse
orparams
arguments are also passed, the function is no longer ignored, but is marked as a fixture.#7360: Fix possibly incorrect evaluation of string expressions passed to
pytest.mark.skipif
andpytest.mark.xfail
, in rare circumstances where the exact same string is used but refers to different global values.#7383: Fixed exception causes all over the codebase, i.e. use
raise new_exception from old_exception
when wrapping an exception.
Improved Documentation¶
Trivial/Internal Changes¶
#7035: The
originalname
attribute of_pytest.python.Function
now defaults toname
if not provided explicitly, and is always set.#7264: The dependency on the
wcwidth
package has been removed.#7295:
src/_pytest/config/__init__.py
now uses thewarnings
module to report warnings instead ofsys.stderr.write
.#7356: Remove last internal uses of deprecated slave term from old
pytest-xdist
.#7357:
py
>=1.8.2 is now required.
pytest 5.4.3 (2020-06-02)¶
Bug Fixes¶
#6428: Paths appearing in error messages are now correct in case the current working directory has changed since the start of the session.
#6755: Support deleting paths longer than 260 characters on windows created inside tmpdir.
#6956: Prevent pytest from printing ConftestImportFailure traceback to stdout.
#7150: Prevent hiding the underlying exception when
ConfTestImportFailure
is raised.#7215: Fix regression where running with
--pdb
would call thetearDown
methods ofunittest.TestCase
subclasses for skipped tests.
pytest 5.4.2 (2020-05-08)¶
Bug Fixes¶
#6871: Fix crash with captured output when using the
capsysbinary fixture
.#6924: Ensure a
unittest.IsolatedAsyncioTestCase
is actually awaited.#6925: Fix TerminalRepr instances to be hashable again.
#6947: Fix regression where functions registered with
TestCase.addCleanup
were not being called on test failures.#6951: Allow users to still set the deprecated
TerminalReporter.writer
attribute.#6992: Revert “tmpdir: clean up indirection via config for factories” #6767 as it breaks pytest-xdist.
#7110: Fixed regression:
asyncbase.TestCase
tests are executed correctly again.#7143: Fix
File.from_parent
so it forwards extra keyword arguments to the constructor.#7145: Classes with broken
__getattribute__
methods are displayed correctly during failures.#7180: Fix
_is_setup_py
for files encoded differently than locale.
pytest 5.4.1 (2020-03-13)¶
Bug Fixes¶
#6909: Revert the change introduced by #6330, which required all arguments to
@pytest.mark.parametrize
to be explicitly defined in the function signature.The intention of the original change was to remove what was expected to be an unintended/surprising behavior, but it turns out many people relied on it, so the restriction has been reverted.
#6910: Fix crash when plugins return an unknown stats while using the
--reportlog
option.
pytest 5.4.0 (2020-03-12)¶
Breaking Changes¶
#6316: Matching of
-k EXPRESSION
to test names is now case-insensitive.#6443: Plugins specified with
-p
are now loaded after internal plugins, which results in their hooks being called before the internal ones.This makes the
-p
behavior consistent withPYTEST_PLUGINS
.#6637: Removed the long-deprecated
pytest_itemstart
hook.This hook has been marked as deprecated and not been even called by pytest for over 10 years now.
#6673: Reversed / fix meaning of “+/-” in error diffs. “-” means that something expected is missing in the result and “+” means that there are unexpected extras in the result.
#6737: The
cached_result
attribute ofFixtureDef
is now set toNone
when the result is unavailable, instead of being deleted.If your plugin performs checks like
hasattr(fixturedef, 'cached_result')
, for example in apytest_fixture_post_finalizer
hook implementation, replace it withfixturedef.cached_result is not None
. If youdel
the attribute, set it toNone
instead.
Deprecations¶
#3238: Option
--no-print-logs
is deprecated and meant to be removed in a future release. If you use--no-print-logs
, please try out--show-capture
and provide feedback.--show-capture
command-line option was added inpytest 3.5.0
and allows to specify how to display captured output when tests fail:no
,stdout
,stderr
,log
orall
(the default).#571: Deprecate the unused/broken
pytest_collect_directory
hook. It was misaligned since the removal of theDirectory
collector in 2010 and incorrect/unusable as soon as collection was split from test execution.#5975: Deprecate using direct constructors for
Nodes
.Instead they are now constructed via
Node.from_parent
.This transitional mechanism enables us to untangle the very intensely entangled
Node
relationships by enforcing more controlled creation/configuration patterns.As part of this change, session/config are already disallowed parameters and as we work on the details we might need disallow a few more as well.
Subclasses are expected to use
super().from_parent
if they intend to expand the creation ofNodes
.#6779: The
TerminalReporter.writer
attribute has been deprecated and should no longer be used. This was inadvertently exposed as part of the public API of that plugin and ties it too much withpy.io.TerminalWriter
.
Features¶
#4597: New –capture=tee-sys option to allow both live printing and capturing of test output.
#5712: Now all arguments to
@pytest.mark.parametrize
need to be explicitly declared in the function signature or viaindirect
. Previously it was possible to omit an argument if a fixture with the same name existed, which was just an accident of implementation and was not meant to be a part of the API.#6454: Changed default for
-r
tofE
, which displays failures and errors in the short test summary.-rN
can be used to disable it (the old behavior).#6469: New options have been added to the
junit_logging
option:log
,out-err
, andall
.#6834: Excess warning summaries are now collapsed per file to ensure readable display of warning summaries.
Improvements¶
#1857:
pytest.mark.parametrize
accepts integers forids
again, converting it to strings.#449: Use “yellow” main color with any XPASSED tests.
#4639: Revert “A warning is now issued when assertions are made for
None
”.The warning proved to be less useful than initially expected and had quite a few false positive cases.
#5686:
tmpdir_factory.mktemp
now fails when given absolute and non-normalized paths.#5984: The
pytest_warning_captured
hook now receives alocation
parameter with the code location that generated the warning.#6213: pytester: the
testdir
fixture respects environment settings from themonkeypatch
fixture for inner runs.#6247:
--fulltrace
is honored with collection errors.#6384: Make
--showlocals
work also with--tb=short
.#6653: Add support for matching lines consecutively with
LineMatcher
’sfnmatch_lines()
andre_match_lines()
.#6658: Code is now highlighted in tracebacks when
pygments
is installed.Users are encouraged to install
pygments
into their environment and provide feedback, because the plan is to makepygments
a regular dependency in the future.#6795: Import usage error message with invalid
-o
option.#759:
pytest.mark.parametrize
supports iterators and generators forids
.
Bug Fixes¶
#310: Add support for calling
pytest.xfail()
andpytest.importorskip()
with doctests.#3823:
--trace
now works with unittests.#4445: Fixed some warning reports produced by pytest to point to the correct location of the warning in the user’s code.
#5301: Fix
--last-failed
to collect new tests from files with known failures.#5928: Report
PytestUnknownMarkWarning
at the level of the user’s code, notpytest
’s.#5991: Fix interaction with
--pdb
and unittests: do not use unittest’sTestCase.debug()
.#6334: Fix summary entries appearing twice when
f/F
ands/S
report chars were used at the same time in the-r
command-line option (for example-rFf
).The upper case variants were never documented and the preferred form should be the lower case.
#6409: Fallback to green (instead of yellow) for non-last items without previous passes with colored terminal progress indicator.
#6454:
--disable-warnings
is honored with-ra
and-rA
.#6497: Fix bug in the comparison of request key with cached key in fixture.
A construct
if key == cached_key:
can fail either because==
is explicitly disallowed, or for, e.g., NumPy arrays, where the result ofa == b
cannot generally be converted tobool
. The implemented fix replaces==
withis
.#6557: Make capture output streams
.write()
method return the same return value from original streams.#6566: Fix
EncodedFile.writelines
to call the underlying buffer’swritelines
method.#6575: Fix internal crash when
faulthandler
starts initialized (for example withPYTHONFAULTHANDLER=1
environment variable set) andfaulthandler_timeout
defined in the configuration file.#6597: Fix node ids which contain a parametrized empty-string variable.
#6646: Assertion rewriting hooks are (re)stored for the current item, which fixes them being still used after e.g. pytester’s
testdir.runpytest
etc.#6660:
pytest.exit()
is handled when emitted from thepytest_sessionfinish
hook. This includes quitting from a debugger.#6752: When
pytest.raises()
is used as a function (as opposed to a context manager), amatch
keyword argument is now passed through to the tested function. Previously it was swallowed and ignored (regression in pytest 5.1.0).#6801: Do not display empty lines in between traceback for unexpected exceptions with doctests.
#6802: The
testdir fixture
works within doctests now.
Improved Documentation¶
Trivial/Internal Changes¶
#6404: Remove usage of
parser
module, deprecated in Python 3.9.
pytest 5.3.5 (2020-01-29)¶
Bug Fixes¶
#6517: Fix regression in pytest 5.3.4 causing an INTERNALERROR due to a wrong assertion.
pytest 5.3.4 (2020-01-20)¶
Bug Fixes¶
pytest 5.3.3 (2020-01-16)¶
Bug Fixes¶
#2780: Captured output during teardown is shown with
-rP
.#5971: Fix a
pytest-xdist
crash when dealing with exceptions raised in subprocesses created by themultiprocessing
module.#6436:
FixtureDef
objects now properly register their finalizers with autouse and parameterized fixtures that execute before them in the fixture stack so they are torn down at the right times, and in the right order.#6532: Fix parsing of outcomes containing multiple errors with
testdir
results (regression in 5.3.0).
Trivial/Internal Changes¶
#6350: Optimized automatic renaming of test parameter IDs.
pytest 5.3.2 (2019-12-13)¶
Improvements¶
#4639: Revert “A warning is now issued when assertions are made for
None
”.The warning proved to be less useful than initially expected and had quite a few false positive cases.
Bug Fixes¶
#5430: junitxml: Logs for failed test are now passed to junit report in case the test fails during call phase.
#6290: The supporting files in the
.pytest_cache
directory are kept with--cache-clear
, which only clears cached values now.#6301: Fix assertion rewriting for egg-based distributions and
editable
installs (pip install --editable
).
pytest 5.3.1 (2019-11-25)¶
Improvements¶
#6231: Improve check for misspelling of pytest.mark.parametrize.
#6257: Handle
pytest.exit()
being used viapytest_internalerror
, e.g. when quitting pdb from post mortem.
Bug Fixes¶
#5914: pytester: fix
no_fnmatch_line()
when used after positive matching.#6082: Fix line detection for doctest samples inside
property
docstrings, as a workaround to python/cpython#61648.#6254: Fix compatibility with pytest-parallel (regression in pytest 5.3.0).
#6255: Clear the
sys.last_traceback
,sys.last_type
andsys.last_value
attributes by deleting them instead of setting them toNone
. This better matches the behaviour of the Python standard library.
pytest 5.3.0 (2019-11-19)¶
Deprecations¶
#6179: The default value of
junit_family
option will change to"xunit2"
in pytest 6.0, given that this is the version supported by default in modern tools that manipulate this type of file.In order to smooth the transition, pytest will issue a warning in case the
--junitxml
option is given in the command line butjunit_family
is not explicitly configured inpytest.ini
.For more information, see the docs.
Features¶
#4488: The pytest team has created the pytest-reportlog plugin, which provides a new
--report-log=FILE
option that writes report logs into a file as the test session executes.Each line of the report log contains a self contained JSON object corresponding to a testing event, such as a collection or a test result report. The file is guaranteed to be flushed after writing each line, so systems can read and process events in real-time.
The plugin is meant to replace the
--resultlog
option, which is deprecated and meant to be removed in a future release. If you use--resultlog
, please try outpytest-reportlog
and provide feedback.#4730: When
sys.pycache_prefix
(Python 3.8+) is set, it will be used by pytest to cache test files changed by the assertion rewriting mechanism.This makes it easier to benefit of cached
.pyc
files even on file systems without permissions.#5515: Allow selective auto-indentation of multiline log messages.
Adds command line option
--log-auto-indent
, config optionlog_auto_indent
and support for per-entry configuration of indentation behavior on calls tologging.log()
.Alters the default for auto-indention from
"on"
to"off"
. This restores the older behavior that existed prior to v4.6.0. This reversion to earlier behavior was done because it is better to activate new features that may lead to broken tests explicitly rather than implicitly.#5914:
testdir
learned two new functions,no_fnmatch_line()
andno_re_match_line()
.The functions are used to ensure the captured text does not match the given pattern.
The previous idiom was to use
re.match()
:result = testdir.runpytest() assert re.match(pat, result.stdout.str()) is None
Or the
in
operator:result = testdir.runpytest() assert text in result.stdout.str()
But the new functions produce best output on failure.
#6057: Added tolerances to complex values when printing
pytest.approx
.For example,
repr(pytest.approx(3+4j))
returns(3+4j) ± 5e-06 ∠ ±180°
. This is polar notation indicating a circle around the expected value, with a radius of 5e-06. Forapprox
comparisons to returnTrue
, the actual value should fall within this circle.#6061: Added the pluginmanager as an argument to
pytest_addoption
so that hooks can be invoked when setting up command line options. This is useful for having one plugin communicate things to another plugin, such as default values or which set of command line options to add.
Improvements¶
#5061: Use multiple colors with terminal summary statistics.
#5630: Quitting from debuggers is now properly handled in
doctest
items.#5924: Improved verbose diff output with sequences.
Before:
E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...] E Right contains 3 more items, first extra item: ' ' E Full diff: E - ['version', 'version_info', 'sys.version', 'sys.version_info'] E + ['version', E + 'version_info', E + 'sys.version', E + 'sys.version_info', E + ' ', E + 'sys.version', E + 'sys.version_info']
After:
E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...] E Right contains 3 more items, first extra item: ' ' E Full diff: E [ E 'version', E 'version_info', E 'sys.version', E 'sys.version_info', E + ' ', E + 'sys.version', E + 'sys.version_info', E ]
#5934:
repr
ofExceptionInfo
objects has been improved to honor the__repr__
method of the underlying exception.#5936: Display untruncated assertion message with
-vv
.#5990: Fixed plurality mismatch in test summary (e.g. display “1 error” instead of “1 errors”).
#6008:
Config.InvocationParams.args
is now always atuple
to better convey that it should be immutable and avoid accidental modifications.#6023:
pytest.main
returns apytest.ExitCode
instance now, except for when custom exit codes are used (where it returnsint
then still).#6026: Align prefixes in output of pytester’s
LineMatcher
.#6059: Collection errors are reported as errors (and not failures like before) in the terminal’s short test summary.
#6069:
pytester.spawn
does not skip/xfail tests on FreeBSD anymore unconditionally.#6097: The “[…%]” indicator in the test summary is now colored according to the final (new) multi-colored line’s main color.
#6116: Added
--co
as a synonym to--collect-only
.#6148:
atomicwrites
is now only used on Windows, fixing a performance regression with assertion rewriting on Unix.#6152: Now parametrization will use the
__name__
attribute of any object for the id, if present. Previously it would only use__name__
for functions and classes.#6176: Improved failure reporting with pytester’s
Hookrecorder.assertoutcome
.#6181: The reason for a stopped session, e.g. with
--maxfail
/-x
, now gets reported in the test summary.#6206: Improved
cache.set
robustness and performance.
Bug Fixes¶
#2049: Fixed
--setup-plan
showing inaccurate information about fixture lifetimes.#2548: Fixed line offset mismatch of skipped tests in terminal summary.
#6039: The
PytestDoctestRunner
is now properly invalidated when unconfiguring the doctest plugin.This is important when used with
pytester
’srunpytest_inprocess
.#6047: BaseExceptions are now handled in
saferepr
, which includespytest.fail.Exception
etc.#6074: pytester: fixed order of arguments in
rm_rf
warning when cleaning up temporary directories, and do not emit warnings for errors withos.open
.#6189: Fixed result of
getmodpath
method.
Trivial/Internal Changes¶
#4901:
RunResult
frompytester
now displays the mnemonic of theret
attribute when it is a validpytest.ExitCode
value.
pytest 5.2.4 (2019-11-15)¶
Bug Fixes¶
pytest 5.2.3 (2019-11-14)¶
Bug Fixes¶
pytest 5.2.2 (2019-10-24)¶
Bug Fixes¶
#5206: Fix
--nf
to not forget about known nodeids with partial test selection.#5906: Fix crash with
KeyboardInterrupt
during--setup-show
.#5946: Fixed issue when parametrizing fixtures with numpy arrays (and possibly other sequence-like types).
#6044: Properly ignore
FileNotFoundError
exceptions when trying to remove old temporary directories, for instance when multiple processes try to remove the same directory (common withpytest-xdist
for example).
pytest 5.2.1 (2019-10-06)¶
Bug Fixes¶
#5902: Fix warnings about deprecated
cmp
attribute inattrs>=19.2
.
pytest 5.2.0 (2019-09-28)¶
Deprecations¶
#1682: Passing arguments to pytest.fixture() as positional arguments is deprecated - pass them as a keyword argument instead.
Features¶
#1682: The
scope
parameter of@pytest.fixture
can now be a callable that receives the fixture name and theconfig
object as keyword-only parameters. See the docs for more information.#5764: New behavior of the
--pastebin
option: failures to connect to the pastebin server are reported, without failing the pytest run
Bug Fixes¶
Trivial/Internal Changes¶
#5056: The HelpFormatter uses
py.io.get_terminal_width
for better width detection.
pytest 5.1.3 (2019-09-18)¶
Bug Fixes¶
pytest 5.1.2 (2019-08-30)¶
Bug Fixes¶
#2270: Fixed
self
reference in function-scoped fixtures defined plugin classes: previouslyself
would be a reference to a test class, not the plugin class.#570: Fixed long standing issue where fixture scope was not respected when indirect fixtures were used during parametrization.
#5782: Fix decoding error when printing an error response from
--pastebin
.#5786: Chained exceptions in test and collection reports are now correctly serialized, allowing plugins like
pytest-xdist
to display them properly.#5792: Windows: Fix error that occurs in certain circumstances when loading
conftest.py
from a working directory that has casing other than the one stored in the filesystem (e.g.,c:\test
instead ofC:\test
).
pytest 5.1.1 (2019-08-20)¶
Bug Fixes¶
#5751: Fixed
TypeError
when importing pytest on Python 3.5.0 and 3.5.1.
pytest 5.1.0 (2019-08-15)¶
Removals¶
#5180: As per our policy, the following features have been deprecated in the 4.X series and are now removed:
Request.getfuncargvalue
: useRequest.getfixturevalue
instead.pytest.raises
andpytest.warns
no longer support strings as the second argument.message
parameter ofpytest.raises
.pytest.raises
,pytest.warns
andParameterSet.param
now use native keyword-only syntax. This might change the exception message from previous versions, but they still raiseTypeError
on unknown keyword arguments as before.pytest.config
global variable.tmpdir_factory.ensuretemp
method.pytest_logwarning
hook.RemovedInPytest4Warning
warning type.request
is now a reserved name for fixtures.
For more information consult Deprecations and Removals in the docs.
#5565: Removed unused support code for unittest2.
The
unittest2
backport module is no longer necessary since Python 3.3+, and the small amount of code in pytest to support it also doesn’t seem to be used: after removed, all tests still pass unchanged.Although our policy is to introduce a deprecation period before removing any features or support for third party libraries, because this code is apparently not used at all (even if
unittest2
is used by a test suite executed by pytest), it was decided to remove it in this release.If you experience a regression because of this, please file an issue.
#5615:
pytest.fail
,pytest.xfail
andpytest.skip
no longer support bytes for the message argument.This was supported for Python 2 where it was tempting to use
"message"
instead ofu"message"
.Python 3 code is unlikely to pass
bytes
to these functions. If you do, please decode it to anstr
beforehand.
Features¶
Improvements¶
Bug Fixes¶
#4344: Fix RuntimeError/StopIteration when trying to collect package with “__init__.py” only.
#5115: Warnings issued during
pytest_configure
are explicitly not treated as errors, even if configured as such, because it otherwise completely breaks pytest.#5477: The XML file produced by
--junitxml
now correctly contain a<testsuites>
root element.#5524: Fix issue where
tmp_path
andtmpdir
would not remove directories containing files marked as read-only, which could lead to pytest crashing when executed a second time with the--basetemp
option.#5537: Replace
importlib_metadata
backport withimportlib.metadata
from the standard library on Python 3.8+.#5578: Improve type checking for some exception-raising functions (
pytest.xfail
,pytest.skip
, etc) so they provide better error messages when users meant to use marks (for example@pytest.xfail
instead of@pytest.mark.xfail
).#5606: Fixed internal error when test functions were patched with objects that cannot be compared for truth values against others, like
numpy
arrays.#5634:
pytest.exit
is now correctly handled inunittest
cases. This makesunittest
cases handlequit
from pytest’s pdb correctly.#5650: Improved output when parsing an ini configuration file fails.
#5701: Fix collection of
staticmethod
objects defined withfunctools.partial
.#5734: Skip async generator test functions, and update the warning message to refer to
async def
functions.
Improved Documentation¶
#5669: Add docstring for
Testdir.copy_example
.
Trivial/Internal Changes¶
#5095: XML files of the
xunit2
family are now validated against the schema by pytest’s own test suite to avoid future regressions.#5516: Cache node splitting function which can improve collection performance in very large test suites.
#5603: Simplified internal
SafeRepr
class and removed some dead code.#5664: When invoking pytest’s own testsuite with
PYTHONDONTWRITEBYTECODE=1
, thetest_xfail_handling
test no longer fails.#5684: Replace manual handling of
OSError.errno
in the codebase by newOSError
subclasses (PermissionError
,FileNotFoundError
, etc.).
pytest 5.0.1 (2019-07-04)¶
Bug Fixes¶
Improved Documentation¶
#5517: Improve “Declaring new hooks” section in chapter “Writing Plugins”
pytest 5.0.0 (2019-06-28)¶
Important¶
This release is a Python3.5+ only release.
For more details, see our Python 2.7 and 3.4 support plan.
Removals¶
#1149: Pytest no longer accepts prefixes of command-line arguments, for example typing
pytest --doctest-mod
inplace of--doctest-modules
. This was previously allowed where theArgumentParser
thought it was unambiguous, but this could be incorrect due to delayed parsing of options for plugins. See for example issues #1149, #3413, and #4009.#5402: PytestDeprecationWarning are now errors by default.
Following our plan to remove deprecated features with as little disruption as possible, all warnings of type
PytestDeprecationWarning
now generate errors instead of warning messages.The affected features will be effectively removed in pytest 5.1, so please consult the Deprecations and Removals section in the docs for directions on how to update existing code.
In the pytest
5.0.X
series, it is possible to change the errors back into warnings as a stop gap measure by adding this to yourpytest.ini
file:[pytest] filterwarnings = ignore::pytest.PytestDeprecationWarning
But this will stop working when pytest
5.1
is released.If you have concerns about the removal of a specific feature, please add a comment to #5402.
#5412:
ExceptionInfo
objects (returned bypytest.raises
) now have the samestr
representation asrepr
, which avoids some confusion when users useprint(e)
to inspect the object.This means code like:
with pytest.raises(SomeException) as e: ... assert "some message" in str(e)
Needs to be changed to:
with pytest.raises(SomeException) as e: ... assert "some message" in str(e.value)
Deprecations¶
Features¶
#3457: New
pytest_assertion_pass
hook, called with context information when an assertion passes.This hook is still experimental so use it with caution.
#5440: The
faulthandler
standard library module is now enabled by default to help users diagnose crashes in C modules.This functionality was provided by integrating the external pytest-faulthandler plugin into the core, so users should remove that plugin from their requirements if used.
For more information see the docs: Fault Handler.
#5452: When warnings are configured as errors, pytest warnings now appear as originating from
pytest.
instead of the internal_pytest.warning_types.
module.#5125:
Session.exitcode
values are now coded inpytest.ExitCode
, anIntEnum
. This makes the exit code available for consumer code and are more explicit other than just documentation. User defined exit codes are still valid, but should be used with caution.The team doesn’t expect this change to break test suites or plugins in general, except in esoteric/specific scenarios.
pytest-xdist users should upgrade to
1.29.0
or later, aspytest-xdist
required a compatibility fix because of this change.
Bug Fixes¶
#1403: Switch from
imp
toimportlib
.#1671: The name of the
.pyc
files cached by the assertion writer now includes the pytest version to avoid stale caches.#5078: Test module is no longer double-imported when using
--pyargs
.#5260: Improved comparison of byte strings.
When comparing bytes, the assertion message used to show the byte numeric value when showing the differences:
def test(): > assert b'spam' == b'eggs' E AssertionError: assert b'spam' == b'eggs' E At index 0 diff: 115 != 101 E Use -v to get the full diff
It now shows the actual ascii representation instead, which is often more useful:
def test(): > assert b'spam' == b'eggs' E AssertionError: assert b'spam' == b'eggs' E At index 0 diff: b's' != b'e' E Use -v to get the full diff
#5335: Colorize level names when the level in the logging format is formatted using ‘%(levelname).Xs’ (truncated fixed width alignment), where X is an integer.
#5354: Fix
pytest.mark.parametrize
when the argvalues is an iterator.#5370: Revert unrolling of
all()
to fixNameError
on nested comprehensions.#5371: Revert unrolling of
all()
to fix incorrect handling of generators withif
.#5372: Revert unrolling of
all()
to fix incorrect assertion when usingall()
in an expression.#5383:
-q
has again an impact on the style of the collected items (--collect-only
) when--log-cli-level
is used.#5389: Fix regressions of #5063 for
importlib_metadata.PathDistribution
which have theirfiles
attribute beingNone
.#5390: Fix regression where the
obj
attribute ofTestCase
items was no longer bound to methods.#5404: Emit a warning when attempting to unwrap a broken object raises an exception, for easier debugging (#5080).
#5432: Prevent “already imported” warnings from assertion rewriter when invoking pytest in-process multiple times.
#5433: Fix assertion rewriting in packages (
__init__.py
).#5444: Fix
--stepwise
mode when the first file passed on the command-line fails to collect.#5482: Fix bug introduced in 4.6.0 causing collection errors when passing more than 2 positional arguments to
pytest.mark.parametrize
.#5505: Fix crash when discovery fails while using
-p no:terminal
.
Improved Documentation¶
pytest 4.6.11 (2020-06-04)¶
Bug Fixes¶
#6334: Fix summary entries appearing twice when
f/F
ands/S
report chars were used at the same time in the-r
command-line option (for example-rFf
).The upper case variants were never documented and the preferred form should be the lower case.
#7310: Fix
UnboundLocalError: local variable 'letter' referenced before assignment
in_pytest.terminal.pytest_report_teststatus()
when plugins return report objects in an unconventional state.This was making
pytest_report_teststatus()
skip entering if-block branches that declare theletter
variable.The fix was to set the initial value of the
letter
before the if-block cascade so that it always has a value.
pytest 4.6.10 (2020-05-08)¶
Features¶
#6870: New
Config.invocation_args
attribute containing the unchanged arguments passed topytest.main()
.Remark: while this is technically a new feature and according to our policy it should not have been backported, we have opened an exception in this particular case because it fixes a serious interaction with
pytest-xdist
, so it can also be considered a bugfix.
Trivial/Internal Changes¶
#6404: Remove usage of
parser
module, deprecated in Python 3.9.
pytest 4.6.9 (2020-01-04)¶
Bug Fixes¶
#6301: Fix assertion rewriting for egg-based distributions and
editable
installs (pip install --editable
).
pytest 4.6.8 (2019-12-19)¶
Features¶
#5471: JUnit XML now includes a timestamp and hostname in the testsuite tag.
Bug Fixes¶
#5430: junitxml: Logs for failed test are now passed to junit report in case the test fails during call phase.
Trivial/Internal Changes¶
#6345: Pin
colorama
to0.4.1
only for Python 3.4 so newer Python versions can still receive colorama updates.
pytest 4.6.7 (2019-12-05)¶
Bug Fixes¶
#5477: The XML file produced by
--junitxml
now correctly contain a<testsuites>
root element.#6044: Properly ignore
FileNotFoundError
(OSError.errno == NOENT
in Python 2) exceptions when trying to remove old temporary directories, for instance when multiple processes try to remove the same directory (common withpytest-xdist
for example).
pytest 4.6.6 (2019-10-11)¶
Bug Fixes¶
#5523: Fixed using multiple short options together in the command-line (for example
-vs
) in Python 3.8+.#5537: Replace
importlib_metadata
backport withimportlib.metadata
from the standard library on Python 3.8+.#5806: Fix “lexer” being used when uploading to bpaste.net from
--pastebin
to “text”.#5902: Fix warnings about deprecated
cmp
attribute inattrs>=19.2
.
Trivial/Internal Changes¶
#5801: Fixes python version checks (detected by
flake8-2020
) in case python4 becomes a thing.
pytest 4.6.5 (2019-08-05)¶
Bug Fixes¶
#4344: Fix RuntimeError/StopIteration when trying to collect package with “__init__.py” only.
#5478: Fix encode error when using unicode strings in exceptions with
pytest.raises
.#5524: Fix issue where
tmp_path
andtmpdir
would not remove directories containing files marked as read-only, which could lead to pytest crashing when executed a second time with the--basetemp
option.#5547:
--step-wise
now handlesxfail(strict=True)
markers properly.#5650: Improved output when parsing an ini configuration file fails.
pytest 4.6.4 (2019-06-28)¶
Bug Fixes¶
#5404: Emit a warning when attempting to unwrap a broken object raises an exception, for easier debugging (#5080).
#5444: Fix
--stepwise
mode when the first file passed on the command-line fails to collect.#5482: Fix bug introduced in 4.6.0 causing collection errors when passing more than 2 positional arguments to
pytest.mark.parametrize
.#5505: Fix crash when discovery fails while using
-p no:terminal
.
pytest 4.6.3 (2019-06-11)¶
Bug Fixes¶
#5383:
-q
has again an impact on the style of the collected items (--collect-only
) when--log-cli-level
is used.#5389: Fix regressions of #5063 for
importlib_metadata.PathDistribution
which have theirfiles
attribute beingNone
.#5390: Fix regression where the
obj
attribute ofTestCase
items was no longer bound to methods.
pytest 4.6.2 (2019-06-03)¶
Bug Fixes¶
pytest 4.6.1 (2019-06-02)¶
Bug Fixes¶
pytest 4.6.0 (2019-05-31)¶
Important¶
The 4.6.X
series will be the last series to support Python 2 and Python 3.4.
For more details, see our Python 2.7 and 3.4 support plan.
Features¶
#4559: Added the
junit_log_passing_tests
ini value which can be used to enable or disable logging of passing test output in the Junit XML file.#4956: pytester’s
testdir.spawn
usestmpdir
as HOME/USERPROFILE directory.#5062: Unroll calls to
all
to full for-loops with assertion rewriting for better failure messages, especially when using Generator Expressions.#5063: Switch from
pkg_resources
toimportlib-metadata
for entrypoint detection for improved performance and import time.#5091: The output for ini options in
--help
has been improved.#5269:
pytest.importorskip
includes theImportError
now in the defaultreason
.#5311: Captured logs that are output for each failing test are formatted using the ColoredLevelFormatter.
#5312: Improved formatting of multiline log messages in Python 3.
Bug Fixes¶
#2064: The debugging plugin imports the wrapped
Pdb
class (--pdbcls
) on-demand now.#4908: The
pytest_enter_pdb
hook gets called with post-mortem (--pdb
).#5036: Fix issue where fixtures dependent on other parametrized fixtures would be erroneously parametrized.
#5256: Handle internal error due to a lone surrogate unicode character not being representable in Jython.
#5257: Ensure that
sys.stdout.mode
does not include'b'
as it is a text stream.#5278: Pytest’s internal python plugin can be disabled using
-p no:python
again.#5286: Fix issue with
disable_test_id_escaping_and_forfeit_all_rights_to_community_support
option not working when using a list of test IDs in parametrized tests.#5330: Show the test module being collected when emitting
PytestCollectionWarning
messages for test classes with__init__
and__new__
methods to make it easier to pin down the problem.#5333: Fix regression in 4.5.0 with
--lf
not re-running all tests with known failures from non-selected tests.
Improved Documentation¶
#5250: Expand docs on use of
setenv
anddelenv
withmonkeypatch
.
pytest 4.5.0 (2019-05-11)¶
Features¶
#4826: A warning is now emitted when unknown marks are used as a decorator. This is often due to a typo, which can lead to silently broken tests.
#4907: Show XFail reason as part of JUnitXML message field.
#5013: Messages from crash reports are displayed within test summaries now, truncated to the terminal width.
#5023: New flag
--strict-markers
that triggers an error when unknown markers (e.g. those not registered using themarkers
option in the configuration file) are used in the test suite.The existing
--strict
option has the same behavior currently, but can be augmented in the future for additional checks.#5026: Assertion failure messages for sequences and dicts contain the number of different items now.
#5034: Improve reporting with
--lf
and--ff
(run-last-failure).#5035: The
--cache-show
option/action accepts an optional glob to show only matching cache entries.#5059: Standard input (stdin) can be given to pytester’s
Testdir.run()
andTestdir.popen()
.#5068: The
-r
option learnt aboutA
to display all reports (including passed ones) in the short test summary.#5108: The short test summary is displayed after passes with output (
-rP
).#5172: The
--last-failed
(--lf
) option got smarter and will now skip entire files if all tests of that test file have passed in previous runs, greatly speeding up collection.#5177: Introduce new specific warning
PytestWarning
subclasses to make it easier to filter warnings based on the class, rather than on the message. The new subclasses are:PytestAssertRewriteWarning
PytestCacheWarning
PytestCollectionWarning
PytestConfigWarning
PytestUnhandledCoroutineWarning
PytestUnknownMarkWarning
#5202: New
record_testsuite_property
session-scoped fixture allows users to log<property>
tags at thetestsuite
level with thejunitxml
plugin.The generated XML is compatible with the latest xunit standard, contrary to the properties recorded by
record_property
andrecord_xml_attribute
.#5214: The default logging format has been changed to improve readability. Here is an example of a previous logging message:
test_log_cli_enabled_disabled.py 3 CRITICAL critical message logged by test
This has now become:
CRITICAL root:test_log_cli_enabled_disabled.py:3 critical message logged by test
The formatting can be changed through the
log_format
configuration option.#5220:
--fixtures
now also shows fixture scope for scopes other than"function"
.
Bug Fixes¶
#5113: Deselected items from plugins using
pytest_collect_modifyitems
as a hookwrapper are correctly reported now.#5144: With usage errors
exitstatus
is set toEXIT_USAGEERROR
in thepytest_sessionfinish
hook now as expected.#5235:
outcome.exit
is not used withEOF
in the pdb wrapper anymore, but only withquit
.
Improved Documentation¶
#4935: Expand docs on registering marks and the effect of
--strict
.
Trivial/Internal Changes¶
#4942:
logging.raiseExceptions
is not set toFalse
anymore.#5013: pytest now depends on wcwidth to properly track unicode character sizes for more precise terminal output.
#5059: pytester’s
Testdir.popen()
usesstdout
andstderr
via keyword arguments with defaults now (subprocess.PIPE
).#5069: The code for the short test summary in the terminal was moved to the terminal plugin.
#5082: Improved validation of kwargs for various methods in the pytester plugin.
#5202:
record_property
now emits aPytestWarning
when used withjunit_family=xunit2
: the fixture generatesproperty
tags as children oftestcase
, which is not permitted according to the most recent schema.#5239: Pin
pluggy
to< 1.0
so we don’t update to1.0
automatically when it gets released: there are planned breaking changes, and we want to ensure pytest properly supportspluggy 1.0
.
pytest 4.4.2 (2019-05-08)¶
Bug Fixes¶
#5089: Fix crash caused by error in
__repr__
function with bothshowlocals
and verbose output enabled.#5139: Eliminate core dependency on ‘terminal’ plugin.
#5229: Require
pluggy>=0.11.0
which reverts a dependency toimportlib-metadata
added in0.10.0
. Theimportlib-metadata
package cannot be imported when installed as an egg and causes issues when relying onsetup.py
to install test dependencies.
Improved Documentation¶
Trivial/Internal Changes¶
#5182: Removed internal and unused
_pytest.deprecated.MARK_INFO_ATTRIBUTE
.
pytest 4.4.1 (2019-04-15)¶
Bug Fixes¶
#5031: Environment variables are properly restored when using pytester’s
testdir
fixture.#5039: Fix regression with
--pdbcls
, which stopped working with local modules in 4.0.0.#5092: Produce a warning when unknown keywords are passed to
pytest.param(...)
.#5098: Invalidate import caches with
monkeypatch.syspath_prepend
, which is required with namespace packages being used.
pytest 4.4.0 (2019-03-29)¶
Features¶
#2224:
async
test functions are skipped and a warning is emitted when a suitable async plugin is not installed (such aspytest-asyncio
orpytest-trio
).Previously
async
functions would not execute at all but still be marked as “passed”.#2482: Include new
disable_test_id_escaping_and_forfeit_all_rights_to_community_support
option to disable ascii-escaping in parametrized values. This may cause a series of problems and as the name makes clear, use at your own risk.#4718: The
-p
option can now be used to early-load plugins also by entry-point name, instead of just by module name.This makes it possible to early load external plugins like
pytest-cov
in the command-line:pytest -p pytest_cov
#4855: The
--pdbcls
option handles classes via module attributes now (e.g.pdb:pdb.Pdb
with pdbpp), and its validation was improved.#4875: The
testpaths
configuration option is now displayed next to therootdir
andinifile
lines in the pytest header if the option is in effect, i.e., directories or file names were not explicitly passed in the command line.Also,
inifile
is only displayed if there’s a configuration file, instead of an emptyinifile:
string.#4911: Doctests can be skipped now dynamically using
pytest.skip()
.#4920: Internal refactorings have been made in order to make the implementation of the pytest-subtests plugin possible, which adds unittest sub-test support and a new
subtests
fixture as discussed in #1367.For details on the internal refactorings, please see the details on the related PR.
#4931: pytester’s
LineMatcher
asserts that the passed lines are a sequence.#4936: Handle
-p plug
after-p no:plug
.This can be used to override a blocked plugin (e.g. in “addopts”) from the command line etc.
#4951: Output capturing is handled correctly when only capturing via fixtures (capsys, capfs) with
pdb.set_trace()
.#4956:
pytester
sets$HOME
and$USERPROFILE
to the temporary directory during test runs.This ensures to not load configuration files from the real user’s home directory.
#4980: Namespace packages are handled better with
monkeypatch.syspath_prepend
andtestdir.syspathinsert
(viapkg_resources.fixup_namespace_packages
).#4993: The stepwise plugin reports status information now.
#5008: If a
setup.cfg
file contains[tool:pytest]
and also the no longer supported[pytest]
section, pytest will use[tool:pytest]
ignoring[pytest]
. Previously it would unconditionally error out.This makes it simpler for plugins to support old pytest versions.
Bug Fixes¶
#1895: Fix bug where fixtures requested dynamically via
request.getfixturevalue()
might be teardown before the requesting fixture.#4851: pytester unsets
PYTEST_ADDOPTS
now to not use outer options withtestdir.runpytest()
.#4903: Use the correct modified time for years after 2038 in rewritten
.pyc
files.#4928: Fix line offsets with
ScopeMismatch
errors.#4957:
-p no:plugin
is handled correctly for default (internal) plugins now, e.g. with-p no:capture
.Previously they were loaded (imported) always, making e.g. the
capfd
fixture available.#4968: The pdb
quit
command is handled properly when used after thedebug
command with pdbpp.#4975: Fix the interpretation of
-qq
option where it was being considered as-v
instead.#4978:
outcomes.Exit
is not swallowed inassertrepr_compare
anymore.#4988: Close logging’s file handler explicitly when the session finishes.
#5003: Fix line offset with mark collection error (off by one).
Improved Documentation¶
#4974: Update docs for
pytest_cmdline_parse
hook to note availability limitations
Trivial/Internal Changes¶
#4718:
pluggy>=0.9
is now required.#4815:
funcsigs>=1.0
is now required for Python 2.7.#4829: Some left-over internal code related to
yield
tests has been removed.#4890: Remove internally unused
anypython
fixture from the pytester plugin.#4912: Remove deprecated Sphinx directive,
add_description_unit()
, pin sphinx-removed-in to >= 0.2.0 to support Sphinx 2.0.#4913: Fix pytest tests invocation with custom
PYTHONPATH
.#4965: New
pytest_report_to_serializable
andpytest_report_from_serializable
experimental hooks.These hooks will be used by
pytest-xdist
,pytest-subtests
, and the replacement for resultlog to serialize and customize reports.They are experimental, meaning that their details might change or even be removed completely in future patch releases without warning.
Feedback is welcome from plugin authors and users alike.
#4987:
Collector.repr_failure
respects the--tb
option, but only defaults toshort
now (withauto
).
pytest 4.3.1 (2019-03-11)¶
Bug Fixes¶
#4810: Logging messages inside
pytest_runtest_logreport()
are now properly captured and displayed.#4861: Improve validation of contents written to captured output so it behaves the same as when capture is disabled.
#4898: Fix
AttributeError: FixtureRequest has no 'confg' attribute
bug intestdir.copy_example
.
Trivial/Internal Changes¶
#4768: Avoid pkg_resources import at the top-level.
pytest 4.3.0 (2019-02-16)¶
Deprecations¶
#4724:
pytest.warns()
now emits a warning when it receives unknown keyword arguments.This will be changed into an error in the future.
Features¶
#2753: Usage errors from argparse are mapped to pytest’s
UsageError
.#3711: Add the
--ignore-glob
parameter to exclude test-modules with Unix shell-style wildcards. Add thecollect_ignore_glob
forconftest.py
to exclude test-modules with Unix shell-style wildcards.#4698: The warning about Python 2.7 and 3.4 not being supported in pytest 5.0 has been removed.
In the end it was considered to be more of a nuisance than actual utility and users of those Python versions shouldn’t have problems as
pip
will not install pytest 5.0 on those interpreters.#4707: With the help of new
set_log_path()
method there is a way to setlog_file
paths from hooks.
Bug Fixes¶
pytest 4.2.1 (2019-02-12)¶
Bug Fixes¶
#2895: The
pytest_report_collectionfinish
hook now is also called with--collect-only
.#3899: Do not raise
UsageError
when an imported package has apytest_plugins.py
child module.#4347: Fix output capturing when using pdb++ with recursive debugging.
#4592: Fix handling of
collect_ignore
via parentconftest.py
.#4700: Fix regression where
setUpClass
would always be called in subclasses even if all tests were skipped by aunittest.skip()
decorator applied in the subclass.#4739: Fix
parametrize(... ids=<function>)
when the function returns non-strings.#4745: Fix/improve collection of args when passing in
__init__.py
and a test file.#4770:
more_itertools
is now constrained to <6.0.0 when required for Python 2.7 compatibility.#526: Fix “ValueError: Plugin already registered” exceptions when running in build directories that symlink to actual source.
Improved Documentation¶
#3899: Add note to
plugins.rst
thatpytest_plugins
should not be used as a name for a user module containing plugins.#4324: Document how to use
raises
anddoes_not_raise
to write parametrized tests with conditional raises.#4709: Document how to customize test failure messages when using
pytest.warns
.
Trivial/Internal Changes¶
#4741: Some verbosity related attributes of the TerminalReporter plugin are now read only properties.
pytest 4.2.0 (2019-01-30)¶
Features¶
#3094: Classic xunit-style functions and methods now obey the scope of autouse fixtures.
This fixes a number of surprising issues like
setup_method
being called before session-scoped autouse fixtures (see #517 for an example).#4627: Display a message at the end of the test session when running under Python 2.7 and 3.4 that pytest 5.0 will no longer support those Python versions.
#4660: The number of selected tests now are also displayed when the
-k
or-m
flags are used.#4688:
pytest_report_teststatus
hook now can also receive aconfig
parameter.#4691:
pytest_terminal_summary
hook now can also receive aconfig
parameter.
Bug Fixes¶
#3547:
--junitxml
can emit XML compatible with Jenkins xUnit.junit_family
INI option acceptslegacy|xunit1
, which produces old style output, andxunit2
that conforms more strictly to https://github.com/jenkinsci/xunit-plugin/blob/xunit-2.3.2/src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd#4280: Improve quitting from pdb, especially with
--trace
.Using
q[quit]
afterpdb.set_trace()
will quit pytest also.#4402: Warning summary now groups warnings by message instead of by test id.
This makes the output more compact and better conveys the general idea of how much code is actually generating warnings, instead of how many tests call that code.
#4536:
monkeypatch.delattr
handles class descriptors likestaticmethod
/classmethod
.#4649: Restore marks being considered keywords for keyword expressions.
#4653:
tmp_path
fixture and other related ones provides resolved path (a.k.a real path)#4667:
pytest_terminal_summary
uses result frompytest_report_teststatus
hook, rather than hardcoded strings.#4669: Correctly handle
unittest.SkipTest
exception containing non-ascii characters on Python 2.#4680: Ensure the
tmpdir
and thetmp_path
fixtures are the same folder.#4681: Ensure
tmp_path
is always a real path.
Trivial/Internal Changes¶
#4643: Use
a.item()
instead of the deprecatednp.asscalar(a)
inpytest.approx
.np.asscalar
has been deprecated innumpy 1.16.
.#4657: Copy saferepr from pylib
#4668: The verbose word for expected failures in the teststatus report changes from
xfail
toXFAIL
to be consistent with other test outcomes.
pytest 4.1.1 (2019-01-12)¶
Bug Fixes¶
Improved Documentation¶
#3375: Document that using
setup.cfg
may crash other tools or cause hard to track down problems because it uses a different parser thanpytest.ini
ortox.ini
files.
Trivial/Internal Changes¶
#4602: Uninstall
hypothesis
in regen tox env.
pytest 4.1.0 (2019-01-05)¶
Removals¶
#2169:
pytest.mark.parametrize
: in previous versions, errors raised by id functions were suppressed and changed into warnings. Now the exceptions are propagated, along with a pytest message informing the node, parameter value and index where the exception occurred.#3078: Remove legacy internal warnings system:
config.warn
,Node.warn
. Thepytest_logwarning
now issues a warning when implemented.See our docs on information on how to update your code.
#3079: Removed support for yield tests - they are fundamentally broken because they don’t support fixtures properly since collection and test execution were separated.
See our docs on information on how to update your code.
#3082: Removed support for applying marks directly to values in
@pytest.mark.parametrize
. Usepytest.param
instead.See our docs on information on how to update your code.
#3083: Removed
Metafunc.addcall
. This was the predecessor mechanism to@pytest.mark.parametrize
.See our docs on information on how to update your code.
#3085: Removed support for passing strings to
pytest.main
. Now, always pass a list of strings instead.See our docs on information on how to update your code.
#3086:
[pytest]
section in setup.cfg files is no longer supported, use[tool:pytest]
instead.setup.cfg
files are meant for use withdistutils
, and a section namedpytest
has notoriously been a source of conflicts and bugs.Note that for pytest.ini and tox.ini files the section remains
[pytest]
.#3616: Removed the deprecated compat properties for
node.Class/Function/Module
- usepytest.Class/Function/Module
now.See our docs on information on how to update your code.
#4421: Removed the implementation of the
pytest_namespace
hook.See our docs on information on how to update your code.
#4489: Removed
request.cached_setup
. This was the predecessor mechanism to modern fixtures.See our docs on information on how to update your code.
#4535: Removed the deprecated
PyCollector.makeitem
method. This method was made public by mistake a long time ago.#4543: Removed support to define fixtures using the
pytest_funcarg__
prefix. Use the@pytest.fixture
decorator instead.See our docs on information on how to update your code.
#4545: Calling fixtures directly is now always an error instead of a warning.
See our docs on information on how to update your code.
#4546: Remove
Node.get_marker(name)
the return value was not usable for more than an existence check.Use
Node.get_closest_marker(name)
as a replacement.#4547: The deprecated
record_xml_property
fixture has been removed, use the more genericrecord_property
instead.See our docs for more information.
#4548: An error is now raised if the
pytest_plugins
variable is defined in a non-top-levelconftest.py
file (i.e., not residing in therootdir
).See our docs for more information.
#891: Remove
testfunction.markername
attributes - useNode.iter_markers(name=None)
to iterate them.
Deprecations¶
#3050: Deprecated the
pytest.config
global.See pytest.config global for rationale.
#3974: Passing the
message
parameter ofpytest.raises
now issues aDeprecationWarning
.It is a common mistake to think this parameter will match the exception message, while in fact it only serves to provide a custom message in case the
pytest.raises
check fails. To avoid this mistake and because it is believed to be little used, pytest is deprecating it without providing an alternative for the moment.If you have concerns about this, please comment on #3974.
#4435: Deprecated
raises(..., 'code(as_a_string)')
andwarns(..., 'code(as_a_string)')
.See raises / warns with a string as the second argument for rationale and examples.
Features¶
#3191: A warning is now issued when assertions are made for
None
.This is a common source of confusion among new users, which write:
assert mocked_object.assert_called_with(3, 4, 5, key="value")
When they should write:
mocked_object.assert_called_with(3, 4, 5, key="value")
Because the
assert_called_with
method of mock objects already executes an assertion.This warning will not be issued when
None
is explicitly checked. An assertion like:assert variable is None
will not issue the warning.
#3632: Richer equality comparison introspection on
AssertionError
for objects created using attrs ordataclasses
(Python 3.7+, backported to 3.6).#4278:
CACHEDIR.TAG
files are now created inside cache directories.Those files are part of the Cache Directory Tagging Standard, and can be used by backup or synchronization programs to identify pytest’s cache directory as such.
#4292:
pytest.outcomes.Exit
is derived fromSystemExit
instead ofKeyboardInterrupt
. This allows us to better handlepdb
exiting.#4371: Updated the
--collect-only
option to display test descriptions when ran using--verbose
.#4386: Restructured
ExceptionInfo
object construction and ensure incomplete instances have arepr
/str
.#4416: pdb: added support for keyword arguments with
pdb.set_trace
.It handles
header
similar to Python 3.7 does it, and forwards any other keyword arguments to thePdb
constructor.This allows for
__import__("pdb").set_trace(skip=["foo.*"])
.#4483: Added ini parameter
junit_duration_report
to optionally report test call durations, excluding setup and teardown times.The JUnit XML specification and the default pytest behavior is to include setup and teardown times in the test duration report. You can include just the call durations instead (excluding setup and teardown) by adding this to your
pytest.ini
file:[pytest] junit_duration_report = call
#4532:
-ra
now will show errors and failures last, instead of as the first items in the summary.This makes it easier to obtain a list of errors and failures to run tests selectively.
#4599:
pytest.importorskip
now supports areason
parameter, which will be shown when the requested module cannot be imported.
Bug Fixes¶
#3532:
-p
now accepts its argument without a space between the value, for example-pmyplugin
.#4327:
approx
again works with more generic containers, more precisely instances ofIterable
andSized
instead of more restrictiveSequence
.#4397: Ensure that node ids are printable.
#4435: Fixed
raises(..., 'code(string)')
frame filename.#4458: Display actual test ids in
--collect-only
.
Improved Documentation¶
Trivial/Internal Changes¶
#4447: Changed the deprecation type of
--result-log
toPytestDeprecationWarning
.It was decided to remove this feature at the next major revision.
pytest 4.0.2 (2018-12-13)¶
Bug Fixes¶
#4265: Validate arguments from the
PYTEST_ADDOPTS
environment variable and theaddopts
ini option separately.#4435: Fix
raises(..., 'code(string)')
frame filename.#4500: When a fixture yields and a log call is made after the test runs, and, if the test is interrupted, capture attributes are
None
.#4538: Raise
TypeError
forwith raises(..., match=<non-None falsey value>)
.
Improved Documentation¶
#1495: Document common doctest fixture directory tree structure pitfalls
pytest 4.0.1 (2018-11-23)¶
Bug Fixes¶
#3952: Display warnings before “short test summary info” again, but still later warnings in the end.
#4386: Handle uninitialized exceptioninfo in repr/str.
#4393: Do not create
.gitignore
/README.md
files in existing cache directories.#4400: Rearrange warning handling for the yield test errors so the opt-out in 4.0.x correctly works.
#4405: Fix collection of testpaths with
--pyargs
.#4412: Fix assertion rewriting involving
Starred
+ side-effects.#4425: Ensure we resolve the absolute path when the given
--basetemp
is a relative path.
Trivial/Internal Changes¶
pytest 4.0.0 (2018-11-13)¶
Removals¶
#3737: RemovedInPytest4Warnings are now errors by default.
Following our plan to remove deprecated features with as little disruption as possible, all warnings of type
RemovedInPytest4Warnings
now generate errors instead of warning messages.The affected features will be effectively removed in pytest 4.1, so please consult the Deprecations and Removals section in the docs for directions on how to update existing code.
In the pytest
4.0.X
series, it is possible to change the errors back into warnings as a stop gap measure by adding this to yourpytest.ini
file:[pytest] filterwarnings = ignore::pytest.RemovedInPytest4Warning
But this will stop working when pytest
4.1
is released.If you have concerns about the removal of a specific feature, please add a comment to #4348.
#4358: Remove the
::()
notation to denote a test class instance in node ids.Previously, node ids that contain test instances would use
::()
to denote the instance like this:test_foo.py::Test::()::test_bar
The extra
::()
was puzzling to most users and has been removed, so that the test id becomes now:test_foo.py::Test::test_bar
This change could not accompany a deprecation period as is usual when user-facing functionality changes because it was not really possible to detect when the functionality was being used explicitly.
The extra
::()
might have been removed in some places internally already, which then led to confusion in places where it was expected, e.g. with--deselect
(#4127).Test class instances are also not listed with
--collect-only
anymore.
Features¶
#4270: The
cache_dir
option uses$TOX_ENV_DIR
as prefix (if set in the environment).This uses a different cache per tox environment by default.
Bug Fixes¶
#3554: Fix
CallInfo.__repr__
for when the call is not finished yet.
pytest 3.10.1 (2018-11-11)¶
Bug Fixes¶
#4287: Fix nested usage of debugging plugin (pdb), e.g. with pytester’s
testdir.runpytest
.#4304: Block the
stepwise
plugin ifcacheprovider
is also blocked, as one depends on the other.#4306: Parse
minversion
as an actual version and not as dot-separated strings.#4310: Fix duplicate collection due to multiple args matching the same packages.
#4321: Fix
item.nodeid
with resolved symlinks.#4325: Fix collection of direct symlinked files, where the target does not match
python_files
.#4329: Fix TypeError in report_collect with _collect_report_last_write.
Trivial/Internal Changes¶
#4305: Replace byte/unicode helpers in test_capture with python level syntax.
pytest 3.10.0 (2018-11-03)¶
Features¶
#2619: Resume capturing output after
continue
with__import__("pdb").set_trace()
.This also adds a new
pytest_leave_pdb
hook, and passes inpdb
to the existingpytest_enter_pdb
hook.#4147: Add
--sw
,--stepwise
as an alternative to--lf -x
for stopping at the first failure, but starting the next test invocation from that test. See the documentation for more info.#4188: Make
--color
emit colorful dots when not running in verbose mode. Earlier, it would only colorize the test-by-test output if--verbose
was also passed.#4225: Improve performance with collection reporting in non-quiet mode with terminals.
The “collecting …” message is only printed/updated every 0.5s.
Bug Fixes¶
#2701: Fix false
RemovedInPytest4Warning: usage of Session... is deprecated, please use pytest
warnings.#4046: Fix problems with running tests in package
__init__.py
files.#4260: Swallow warnings during anonymous compilation of source.
#4262: Fix access denied error when deleting stale directories created by
tmpdir
/tmp_path
.#611: Naming a fixture
request
will now raise a warning: therequest
fixture is internal and should not be overwritten as it will lead to internal errors.#4266: Handle (ignore) exceptions raised during collection, e.g. with Django’s LazySettings proxy class.
Improved Documentation¶
#4255: Added missing documentation about the fact that module names passed to filter warnings are not regex-escaped.
Trivial/Internal Changes¶
#4272: Display cachedir also in non-verbose mode if non-default.
#4277: pdb: improve message about output capturing with
set_trace
.Do not display “IO-capturing turned off/on” when
-s
is used to avoid confusion.#4279: Improve message and stack level of warnings issued by
monkeypatch.setenv
when the value of the environment variable is not astr
.
pytest 3.9.3 (2018-10-27)¶
Bug Fixes¶
#4174: Fix “ValueError: Plugin already registered” with conftest plugins via symlink.
#4181: Handle race condition between creation and deletion of temporary folders.
#4221: Fix bug where the warning summary at the end of the test session was not showing the test where the warning was originated.
#4243: Fix regression when
stacklevel
for warnings was passed as positional argument on python2.
Improved Documentation¶
#3851: Add reference to
empty_parameter_set_mark
ini option in documentation of@pytest.mark.parametrize
Trivial/Internal Changes¶
pytest 3.9.2 (2018-10-22)¶
Bug Fixes¶
#2909: Improve error message when a recursive dependency between fixtures is detected.
#3340: Fix logging messages not shown in hooks
pytest_sessionstart()
andpytest_sessionfinish()
.#3533: Fix unescaped XML raw objects in JUnit report for skipped tests
#3691: Python 2: safely format warning message about passing unicode strings to
warnings.warn
, which may cause surprisingMemoryError
exception when monkey patchingwarnings.warn
itself.#4026: Improve error message when it is not possible to determine a function’s signature.
#4177: Pin
setuptools>=40.0
to supportpy_modules
insetup.cfg
#4179: Restore the tmpdir behaviour of symlinking the current test run.
#4192: Fix filename reported by
warnings.warn
when usingrecwarn
under python2.
pytest 3.9.1 (2018-10-16)¶
Features¶
#4159: For test-suites containing test classes, the information about the subclassed module is now output only if a higher verbosity level is specified (at least “-vv”).
pytest 3.9.0 (2018-10-15 - not published due to a release automation bug)¶
Deprecations¶
#3616: The following accesses have been documented as deprecated for years, but are now actually emitting deprecation warnings.
Access of
Module
,Function
,Class
,Instance
,File
andItem
throughNode
instances. Now users will this warning:usage of Function.Module is deprecated, please use pytest.Module instead
Users should just
import pytest
and access those objects using thepytest
module.request.cached_setup
, this was the precursor of the setup/teardown mechanism available to fixtures. You can consult funcarg comparison section in the docs.Using objects named
"Class"
as a way to customize the type of nodes that are collected inCollector
subclasses has been deprecated. Users instead should usepytest_collect_make_item
to customize node types during collection.This issue should affect only advanced plugins who create new collection types, so if you see this warning message please contact the authors so they can change the code.
The warning that produces the message below has changed to
RemovedInPytest4Warning
:getfuncargvalue is deprecated, use getfixturevalue
#3988: Add a Deprecation warning for pytest.ensuretemp as it was deprecated since a while.
Features¶
#2293: Improve usage errors messages by hiding internal details which can be distracting and noisy.
This has the side effect that some error conditions that previously raised generic errors (such as
ValueError
for unregistered marks) are now raisingFailed
exceptions.#3332: Improve the error displayed when a
conftest.py
file could not be imported.In order to implement this, a new
chain
parameter was added toExceptionInfo.getrepr
to show or hide chained tracebacks in Python 3 (defaults toTrue
).#3849: Add
empty_parameter_set_mark=fail_at_collect
ini option for raising an exception when parametrize collects an empty set.#3964: Log messages generated in the collection phase are shown when live-logging is enabled and/or when they are logged to a file.
#3985: Introduce
tmp_path
as a fixture providing a Path object. Also introducetmp_path_factory
as a session-scoped fixture for creating arbitrary temporary directories from any other fixture or test.#4013: Deprecation warnings are now shown even if you customize the warnings filters yourself. In the previous version any customization would override pytest’s filters and deprecation warnings would fall back to being hidden by default.
#4073: Allow specification of timeout for
Testdir.runpytest_subprocess()
andTestdir.run()
.#4098: Add returncode argument to pytest.exit() to exit pytest with a specific return code.
#4102: Reimplement
pytest.deprecated_call
usingpytest.warns
so it supports thematch='...'
keyword argument.This has the side effect that
pytest.deprecated_call
now raisespytest.fail.Exception
instead ofAssertionError
.#4149: Require setuptools>=30.3 and move most of the metadata to
setup.cfg
.
Bug Fixes¶
#2535: Improve error message when test functions of
unittest.TestCase
subclasses use a parametrized fixture.#3057:
request.fixturenames
now correctly returns the name of fixtures created byrequest.getfixturevalue()
.#3946: Warning filters passed as command line options using
-W
now take precedence over filters defined inini
configuration files.#4066: Fix source reindenting by using
textwrap.dedent
directly.#4102:
pytest.warn
will capture previously-warned warnings in Python 2. Previously they were never raised.#4108: Resolve symbolic links for args.
This fixes running
pytest tests/test_foo.py::test_bar
, wheretests
is a symlink toproject/app/tests
: previouslyproject/app/conftest.py
would be ignored for fixtures then.#4132: Fix duplicate printing of internal errors when using
--pdb
.#4135: pathlib based tmpdir cleanup now correctly handles symlinks in the folder.
#4152: Display the filename when encountering
SyntaxWarning
.
Improved Documentation¶
#3713: Update usefixtures documentation to clarify that it can’t be used with fixture functions.
#4058: Update fixture documentation to specify that a fixture can be invoked twice in the scope it’s defined for.
#4064: According to unittest.rst, setUpModule and tearDownModule were not implemented, but it turns out they are. So updated the documentation for unittest.
#4151: Add tempir testing example to CONTRIBUTING.rst guide
Trivial/Internal Changes¶
pytest 3.8.2 (2018-10-02)¶
Deprecations and Removals¶
#4036: The
item
parameter ofpytest_warning_captured
hook is now documented as deprecated. We realized only after the3.8
release that this parameter is incompatible withpytest-xdist
.Our policy is to not deprecate features during bug-fix releases, but in this case we believe it makes sense as we are only documenting it as deprecated, without issuing warnings which might potentially break test suites. This will get the word out that hook implementers should not use this parameter at all.
In a future release
item
will always beNone
and will emit a proper warning when a hook implementation makes use of it.
Bug Fixes¶
#3539: Fix reload on assertion rewritten modules.
#4034: The
.user_properties
attribute ofTestReport
objects is a list of (name, value) tuples, but could sometimes be instantiated as a tuple of tuples. It is now always a list.#4039: No longer issue warnings about using
pytest_plugins
in non-top-level directories when using--pyargs
: the current--pyargs
mechanism is not reliable and might give false negatives.#4040: Exclude empty reports for passed tests when
-rP
option is used.#4051: Improve error message when an invalid Python expression is passed to the
-m
option.#4056:
MonkeyPatch.setenv
andMonkeyPatch.delenv
issue a warning if the environment variable name is notstr
on Python 2.In Python 2, adding
unicode
keys toos.environ
causes problems withsubprocess
(and possible other modules), making this a subtle bug specially susceptible when used withfrom __future__ import unicode_literals
.
Improved Documentation¶
#3928: Add possible values for fixture scope to docs.
pytest 3.8.1 (2018-09-22)¶
Bug Fixes¶
#3286:
.pytest_cache
directory is now automatically ignored by Git. Users who would like to contribute a solution for other SCMs please consult/comment on this issue.#3749: Fix the following error during collection of tests inside packages:
TypeError: object of type 'Package' has no len()
#3941: Fix bug where indirect parametrization would consider the scope of all fixtures used by the test function to determine the parametrization scope, and not only the scope of the fixtures being parametrized.
#3973: Fix crash of the assertion rewriter if a test changed the current working directory without restoring it afterwards.
#3998: Fix issue that prevented some caplog properties (for example
record_tuples
) from being available when entering the debugger with--pdb
.#3999: Fix
UnicodeDecodeError
in python2.x when a class returns a non-ascii binary__repr__
in an assertion which also contains non-ascii text.
Improved Documentation¶
#3996: New Deprecations and Removals page shows all currently deprecated features, the rationale to do so, and alternatives to update your code. It also list features removed from pytest in past major releases to help those with ancient pytest versions to upgrade.
Trivial/Internal Changes¶
pytest 3.8.0 (2018-09-05)¶
Deprecations and Removals¶
#2452:
Config.warn
andNode.warn
have been deprecated, see Config.warn and Node.warn for rationale and examples.#3936:
@pytest.mark.filterwarnings
second parameter is no longer regex-escaped, making it possible to actually use regular expressions to check the warning message.Note: regex-escaping the match string was an implementation oversight that might break test suites which depend on the old behavior.
Features¶
#2452: Internal pytest warnings are now issued using the standard
warnings
module, making it possible to use the standard warnings filters to manage those warnings. This introducesPytestWarning
,PytestDeprecationWarning
andRemovedInPytest4Warning
warning types as part of the public API.Consult the documentation for more info.
#2908:
DeprecationWarning
andPendingDeprecationWarning
are now shown by default if no other warning filter is configured. This makes pytest more compliant with PEP 506#recommended-filter-settings-for-test-runners. See the docs for more info.#3251: Warnings are now captured and displayed during test collection.
#3784:
PYTEST_DISABLE_PLUGIN_AUTOLOAD
environment variable disables plugin auto-loading when set.#3829: Added the
count
option toconsole_output_style
to enable displaying the progress as a count instead of a percentage.#3837: Added support for ‘xfailed’ and ‘xpassed’ outcomes to the
pytester.RunResult.assert_outcomes
signature.
Bug Fixes¶
Improved Documentation¶
Trivial/Internal Changes¶
#3853: Removed
"run all (no recorded failures)"
message printed with--failed-first
and--last-failed
when there are no failed tests.
pytest 3.7.4 (2018-08-29)¶
Bug Fixes¶
#3506: Fix possible infinite recursion when writing
.pyc
files.#3853: Cache plugin now obeys the
-q
flag when--last-failed
and--failed-first
flags are used.#3883: Fix bad console output when using
console_output_style=classic
.#3888: Fix macOS specific code using
capturemanager
plugin in doctests.
Improved Documentation¶
#3902: Fix pytest.org links
pytest 3.7.3 (2018-08-26)¶
Bug Fixes¶
#3033: Fixtures during teardown can again use
capsys
andcapfd
to inspect output captured during tests.#3773: Fix collection of tests from
__init__.py
files if they match thepython_files
configuration option.#3796: Fix issue where teardown of fixtures of consecutive sub-packages were executed once, at the end of the outer package.
#3816: Fix bug where
--show-capture=no
option would still show logs printed during fixture teardown.#3819: Fix
stdout/stderr
not getting captured when real-time cli logging is active.#3843: Fix collection error when specifying test functions directly in the command line using
test.py::test
syntax together with--doctest-modules
.#3848: Fix bugs where unicode arguments could not be passed to
testdir.runpytest
on Python 2.#3854: Fix double collection of tests within packages when the filename starts with a capital letter.
Improved Documentation¶
Trivial/Internal Changes¶
pytest 3.7.2 (2018-08-16)¶
Bug Fixes¶
#3671: Fix
filterwarnings
not being registered as a builtin mark.#3768, #3789: Fix test collection from packages mixed with normal directories.
#3771: Fix infinite recursion during collection if a
pytest_ignore_collect
hook returnsFalse
instead ofNone
.#3774: Fix bug where decorated fixtures would lose functionality (for example
@mock.patch
).#3775: Fix bug where importing modules or other objects with prefix
pytest_
prefix would raise aPluginValidationError
.#3788: Fix
AttributeError
during teardown ofTestCase
subclasses which raise an exception during__init__
.#3804: Fix traceback reporting for exceptions with
__cause__
cycles.
Improved Documentation¶
#3746: Add documentation for
metafunc.config
that had been mistakenly hidden.
pytest 3.7.1 (2018-08-02)¶
Bug Fixes¶
#3473: Raise immediately if
approx()
is given an expected value of a type it doesn’t understand (e.g. strings, nested dicts, etc.).#3712: Correctly represent the dimensions of a numpy array when calling
repr()
onapprox()
.#3742: Fix incompatibility with third party plugins during collection, which produced the error
object has no attribute '_collectfile'
.#3745: Display the absolute path if
cache_dir
is not relative to therootdir
instead of failing.#3747: Fix compatibility problem with plugins and the warning code issued by fixture functions when they are called directly.
#3748: Fix infinite recursion in
pytest.approx
with arrays innumpy<1.13
.#3757: Pin pathlib2 to
>=2.2.0
as we require__fspath__
support.#3763: Fix
TypeError
when the assertion message isbytes
in python 3.
pytest 3.7.0 (2018-07-30)¶
Deprecations and Removals¶
#2639:
pytest_namespace
has been deprecated.#3661: Calling a fixture function directly, as opposed to request them in a test function, now issues a
RemovedInPytest4Warning
. See the documentation for rationale and examples.
Features¶
#2283: New
package
fixture scope: fixtures are finalized when the last test of a package finishes. This feature is considered experimental, so use it sparingly.#3576:
Node.add_marker
now supports anappend=True/False
parameter to determine whether the mark comes last (default) or first.#3579: Fixture
caplog
now has amessages
property, providing convenient access to the format-interpolated log messages without the extra data provided by the formatter/handler.#3610: New
--trace
option to enter the debugger at the start of a test.#3623: Introduce
pytester.copy_example
as helper to do acceptance tests against examples from the project.
Bug Fixes¶
#2220: Fix a bug where fixtures overridden by direct parameters (for example parametrization) were being instantiated even if they were not being used by a test.
#3695: Fix
ApproxNumpy
initialisation argument mixup,abs
andrel
tolerances were flipped causing strange comparison results. Add tests to checkabs
andrel
tolerances fornp.array
and test for expectingnan
withnp.array()
#980: Fix truncated locals output in verbose mode.
Improved Documentation¶
#3295: Correct the usage documentation of
--last-failed-no-failures
by adding the missing--last-failed
argument in the presented examples, because they are misleading and lead to think that the missing argument is not needed.
Trivial/Internal Changes¶
#3519: Now a
README.md
file is created in.pytest_cache
to make it clear why the directory exists.
pytest 3.6.4 (2018-07-28)¶
Bug Fixes¶
Invoke pytest using
-mpytest
sosys.path
does not get polluted by packages installed insite-packages
. (#742)
Improved Documentation¶
Use
smtp_connection
instead ofsmtp
in fixtures documentation to avoid possible confusion. (#3592)
Trivial/Internal Changes¶
pytest 3.6.3 (2018-07-04)¶
Bug Fixes¶
Fix
ImportWarning
triggered by explicit relative imports in assertion-rewritten package modules. (#3061)Fix error in
pytest.approx
when dealing with 0-dimension numpy arrays. (#3593)No longer raise
ValueError
when using theget_marker
API. (#3605)Fix problem where log messages with non-ascii characters would not appear in the output log file. (#3630)
No longer raise
AttributeError
when legacy marks can’t be stored in functions. (#3631)
Improved Documentation¶
The description above the example for
@pytest.mark.skipif
now better matches the code. (#3611)
Trivial/Internal Changes¶
pytest 3.6.2 (2018-06-20)¶
Bug Fixes¶
Fix regression in
Node.add_marker
by extracting the mark object of aMarkDecorator
. (#3555)Warnings without
location
were reported asNone
. This is corrected to now report<undetermined location>
. (#3563)Continue to call finalizers in the stack when a finalizer in a former scope raises an exception. (#3569)
Fix encoding error with
print
statements in doctests (#3583)
Improved Documentation¶
Add documentation for the
--strict
flag. (#3549)
Trivial/Internal Changes¶
Update old quotation style to parens in fixture.rst documentation. (#3525)
Improve display of hint about
--fulltrace
withKeyboardInterrupt
. (#3545)pytest’s testsuite is no longer runnable through
python setup.py test
– instead invokepytest
ortox
directly. (#3552)Fix typo in documentation (#3567)
pytest 3.6.1 (2018-06-05)¶
Bug Fixes¶
Improved Documentation¶
Added a section on how to use fixtures as factories to the fixture documentation. (#3461)
Trivial/Internal Changes¶
Enable caching for pip/pre-commit in order to reduce build time on travis/appveyor. (#3502)
Switch pytest to the src/ layout as we already suggested it for good practice - now we implement it as well. (#3513)
Fix if in tests to support 3.7.0b5, where a docstring handling in AST got reverted. (#3530)
Remove some python2.5 compatibility code. (#3529)
pytest 3.6.0 (2018-05-23)¶
Features¶
Revamp the internals of the
pytest.mark
implementation with correct per node handling which fixes a number of long standing bugs caused by the old design. This introduces newNode.iter_markers(name)
andNode.get_closest_marker(name)
APIs. Users are strongly encouraged to read the reasons for the revamp in the docs, or jump over to details about updating existing code to use the new APIs. (#3317)Now when
@pytest.fixture
is applied more than once to the same function aValueError
is raised. This buggy behavior would cause surprising problems and if was working for a test suite it was mostly by accident. (#2334)Support for Python 3.7’s builtin
breakpoint()
method, see Using the builtin breakpoint function for details. (#3180)monkeypatch
now supports acontext()
function which acts as a context manager which undoes all patching done within thewith
block. (#3290)The
--pdb
option now causes KeyboardInterrupt to enter the debugger, instead of stopping the test session. On python 2.7, hitting CTRL+C again exits the debugger. On python 3.2 and higher, use CTRL+D. (#3299)pytest no longer changes the log level of the root logger when the
log-level
parameter has greater numeric value than that of the level of the root logger, which makes it play better with custom logging configuration in user code. (#3307)
Bug Fixes¶
A rare race-condition which might result in corrupted
.pyc
files on Windows has been hopefully solved. (#3008)Also use iter_marker for discovering the marks applying for marker expressions from the cli to avoid the bad data from the legacy mark storage. (#3441)
When showing diffs of failed assertions where the contents contain only whitespace, escape them using
repr()
first to make it easy to spot the differences. (#3443)
Improved Documentation¶
Change documentation copyright year to a range which auto-updates itself each time it is published. (#3303)
Trivial/Internal Changes¶
pytest
now depends on the python-atomicwrites library. (#3008)Update all pypi.python.org URLs to pypi.org. (#3431)
Detect
pytest_
prefixed hooks using the internal plugin manager sincepluggy
is deprecating theimplprefix
argument toPluginManager
. (#3487)Import
Mapping
andSequence
from_pytest.compat
instead of directly fromcollections
inpython_api.py::approx
. AddMapping
to_pytest.compat
, import it fromcollections
on python 2, but fromcollections.abc
on Python 3 to avoid aDeprecationWarning
on Python 3.7 or newer. (#3497)
pytest 3.5.1 (2018-04-23)¶
Bug Fixes¶
Reset
sys.last_type
,sys.last_value
andsys.last_traceback
before each test executes. Those attributes are added by pytest during the test run to aid debugging, but were never reset so they would create a leaking reference to the last failing test’s frame which in turn could never be reclaimed by the garbage collector. (#2798)pytest.raises
now raisesTypeError
when receiving an unknown keyword argument. (#3348)pytest.raises
now works with exception classes that look like iterables. (#3372)
Improved Documentation¶
Fix typo in
caplog
fixture documentation, which incorrectly identified certain attributes as methods. (#3406)
Trivial/Internal Changes¶
Added a more indicative error message when parametrizing a function whose argument takes a default value. (#3221)
Remove internal
_pytest.terminal.flatten
function in favor ofmore_itertools.collapse
. (#3330)Import some modules from
collections.abc
instead ofcollections
as the former modules triggerDeprecationWarning
in Python 3.7. (#3339)record_property is no longer experimental, removing the warnings was forgotten. (#3360)
Mention in documentation and CLI help that fixtures with leading
_
are printed bypytest --fixtures
only if the-v
option is added. (#3398)
pytest 3.5.0 (2018-03-21)¶
Deprecations and Removals¶
record_xml_property
fixture is now deprecated in favor of the more genericrecord_property
. (#2770)Defining
pytest_plugins
is now deprecated in non-top-level conftest.py files, because they “leak” to the entire directory tree. See the docs for the rationale behind this decision (#3084)
Features¶
New
--show-capture
command-line option that allows to specify how to display captured output when tests fail:no
,stdout
,stderr
,log
orall
(the default). (#1478)New
--rootdir
command-line option to override the rules for discovering the root directory. See customize in the documentation for details. (#1642)Fixtures are now instantiated based on their scopes, with higher-scoped fixtures (such as
session
) being instantiated first than lower-scoped fixtures (such asfunction
). The relative order of fixtures of the same scope is kept unchanged, based in their declaration order and their dependencies. (#2405)record_xml_property
renamed torecord_property
and is now compatible with xdist, markers and any reporter.record_xml_property
name is now deprecated. (#2770)New
--nf
,--new-first
options: run new tests first followed by the rest of the tests, in both cases tests are also sorted by the file modified time, with more recent files coming first. (#3034)New
--last-failed-no-failures
command-line option that allows to specify the behavior of the cache plugin’s`--last-failed
feature when no tests failed in the last run (or no cache was found):none
orall
(the default). (#3139)New
--doctest-continue-on-failure
command-line option to enable doctests to show multiple failures for each snippet, instead of stopping at the first failure. (#3149)Captured log messages are added to the
<system-out>
tag in the generated junit xml file if thejunit_logging
ini option is set tosystem-out
. If the value of this ini option issystem-err
, the logs are written to<system-err>
. The default value forjunit_logging
isno
, meaning captured logs are not written to the output file. (#3156)Allow the logging plugin to handle
pytest_runtest_logstart
andpytest_runtest_logfinish
hooks when live logs are enabled. (#3189)Passing
--log-cli-level
in the command-line now automatically activates live logging. (#3190)Add command line option
--deselect
to allow deselection of individual tests at collection time. (#3198)Captured logs are printed before entering pdb. (#3204)
Deselected item count is now shown before tests are run, e.g.
collected X items / Y deselected
. (#3213)The builtin module
platform
is now available for use in expressions inpytest.mark
. (#3236)The short test summary info section now is displayed after tracebacks and warnings in the terminal. (#3255)
New
--verbosity
flag to set verbosity level explicitly. (#3296)pytest.approx
now accepts comparing a numpy array with a scalar. (#3312)
Bug Fixes¶
Suppress
IOError
when closing the temporary file used for capturing streams in Python 2.7. (#2370)Fixed
clear()
method oncaplog
fixture which clearedrecords
, but not thetext
property. (#3297)During test collection, when stdin is not allowed to be read, the
DontReadFromStdin
object still allow itself to be iterable and resolved to an iterator without crashing. (#3314)
Improved Documentation¶
Trivial/Internal Changes¶
Change minimum requirement of
attrs
to17.4.0
. (#3228)Renamed example directories so all tests pass when ran from the base directory. (#3245)
Internal
mark.py
module has been turned into a package. (#3250)pytest
now depends on the more-itertools package. (#3265)Added warning when
[pytest]
section is used in a.cfg
file passed with-c
(#3268)nodeids
can now be passed explicitly toFSCollector
andNode
constructors. (#3291)Internal refactoring of
FormattedExcinfo
to useattrs
facilities and remove old support code for legacy Python versions. (#3292)Refactoring to unify how verbosity is handled internally. (#3296)
Internal refactoring to better integrate with argparse. (#3304)
Fix a python example when calling a fixture in doc/en/usage.rst (#3308)
pytest 3.4.2 (2018-03-04)¶
Bug Fixes¶
Removed progress information when capture option is
no
. (#3203)Refactor check of bindir from
exists
toisdir
. (#3241)Fix
TypeError
issue when usingapprox
with aDecimal
value. (#3247)Fix reference cycle generated when using the
request
fixture. (#3249)[tool:pytest]
sections in*.cfg
files passed by the-c
option are now properly recognized. (#3260)
Improved Documentation¶
Add logging plugin to plugins list. (#3209)
Trivial/Internal Changes¶
Fix minor typo in fixture.rst (#3259)
pytest 3.4.1 (2018-02-20)¶
Bug Fixes¶
Move import of
doctest.UnexpectedException
to top-level to avoid possible errors when using--pdb
. (#1810)Added printing of captured stdout/stderr before entering pdb, and improved a test which was giving false negatives about output capturing. (#3052)
Fix ordering of tests using parametrized fixtures which can lead to fixtures being created more than necessary. (#3161)
Fix bug where logging happening at hooks outside of “test run” hooks would cause an internal error. (#3184)
Detect arguments injected by
unittest.mock.patch
decorator correctly when pypimock.patch
is installed and imported. (#3206)Errors shown when a
pytest.raises()
withmatch=
fails are now cleaner on what happened: When no exception was raised, the “matching ‘…’” part got removed as it falsely implies that an exception was raised but it didn’t match. When a wrong exception was raised, it’s now thrown (likepytest.raised()
withoutmatch=
would) instead of complaining about the unmatched text. (#3222)Fixed output capture handling in doctests on macOS. (#985)
Improved Documentation¶
Add Sphinx parameter docs for
match
andmessage
args topytest.raises
. (#3202)
Trivial/Internal Changes¶
pytest 3.4.0 (2018-01-30)¶
Deprecations and Removals¶
All pytest classes now subclass
object
for better Python 2/3 compatibility. This should not affect user code except in very rare edge cases. (#2147)
Features¶
Introduce
empty_parameter_set_mark
ini option to select which mark to apply when@pytest.mark.parametrize
is given an empty set of parameters. Valid options areskip
(default) andxfail
. Note that it is planned to change the default toxfail
in future releases as this is considered less error prone. (#2527)Incompatible change: after community feedback the logging functionality has undergone some changes. Please consult the logging documentation for details. (#3013)
Console output falls back to “classic” mode when capturing is disabled (
-s
), otherwise the output gets garbled to the point of being useless. (#3038)New
pytest_runtest_logfinish
hook which is called when a test item has finished executing, analogous topytest_runtest_logstart
. (#3101)Improve performance when collecting tests using many fixtures. (#3107)
New
caplog.get_records(when)
method which provides access to the captured records for the"setup"
,"call"
and"teardown"
testing stages. (#3117)New fixture
record_xml_attribute
that allows modifying and inserting attributes on the<testcase>
xml node in JUnit reports. (#3130)The default cache directory has been renamed from
.cache
to.pytest_cache
after community feedback that the name.cache
did not make it clear that it was used by pytest. (#3138)Colorize the levelname column in the live-log output. (#3142)
Bug Fixes¶
Fix hanging pexpect test on macOS by using flush() instead of wait(). (#2022)
Fix restoring Python state after in-process pytest runs with the
pytester
plugin; this may break tests using multiple inprocess pytest runs if later ones depend on earlier ones leaking global interpreter changes. (#3016)Fix skipping plugin reporting hook when test aborted before plugin setup hook. (#3074)
Fix progress percentage reported when tests fail during teardown. (#3088)
Incompatible change:
-o/--override
option no longer eats all the remaining options, which can lead to surprising behavior: for example,pytest -o foo=1 /path/to/test.py
would fail because/path/to/test.py
would be considered as part of the-o
command-line argument. One consequence of this is that now multiple configuration overrides need multiple-o
flags:pytest -o foo=1 -o bar=2
. (#3103)
Improved Documentation¶
Document hooks (defined with
historic=True
) which cannot be used withhookwrapper=True
. (#2423)Clarify that warning capturing doesn’t change the warning filter by default. (#2457)
Clarify a possible confusion when using pytest_fixture_setup with fixture functions that return None. (#2698)
Fix the wording of a sentence on doctest flags used in pytest. (#3076)
Prefer
https://*.readthedocs.io
overhttp://*.rtfd.org
for links in the documentation. (#3092)Improve readability (wording, grammar) of Getting Started guide (#3131)
Added note that calling pytest.main multiple times from the same process is not recommended because of import caching. (#3143)
Trivial/Internal Changes¶
Show a simple and easy error when keyword expressions trigger a syntax error (for example,
"-k foo and import"
will show an error that you cannot use theimport
keyword in expressions). (#2953)Change parametrized automatic test id generation to use the
__name__
attribute of functions instead of the fallback argument name plus counter. (#2976)Replace py.std with stdlib imports. (#3067)
Corrected ‘you’ to ‘your’ in logging docs. (#3129)
pytest 3.3.2 (2017-12-25)¶
Bug Fixes¶
pytester: ignore files used to obtain current user metadata in the fd leak detector. (#2784)
Fix memory leak where objects returned by fixtures were never destructed by the garbage collector. (#2981)
Fix conversion of pyargs to filename to not convert symlinks on Python 2. (#2985)
PYTEST_DONT_REWRITE
is now checked for plugins too rather than only for test modules. (#2995)
Improved Documentation¶
Add clarifying note about behavior of multiple parametrized arguments (#3001)
Trivial/Internal Changes¶
pytest 3.3.1 (2017-12-05)¶
Bug Fixes¶
Fix issue about
-p no:<plugin>
having no effect. (#2920)Fix regression with warnings that contained non-strings in their arguments in Python 2. (#2956)
Always escape null bytes when setting
PYTEST_CURRENT_TEST
. (#2957)Fix
ZeroDivisionError
when using thetestmon
plugin when no tests were actually collected. (#2971)Bring back
TerminalReporter.writer
as an alias toTerminalReporter._tw
. This alias was removed by accident in the3.3.0
release. (#2984)The
pytest-capturelog
plugin is now also blacklisted, avoiding errors when running pytest with it still installed. (#3004)
Improved Documentation¶
Fix broken link to plugin
pytest-localserver
. (#2963)
Trivial/Internal Changes¶
Update github “bugs” link in
CONTRIBUTING.rst
(#2949)
pytest 3.3.0 (2017-11-23)¶
Deprecations and Removals¶
pytest no longer supports Python 2.6 and 3.3. Those Python versions are EOL for some time now and incur maintenance and compatibility costs on the pytest core team, and following up with the rest of the community we decided that they will no longer be supported starting on this version. Users which still require those versions should pin pytest to
<3.3
. (#2812)Remove internal
_preloadplugins()
function. This removal is part of thepytest_namespace()
hook deprecation. (#2636)Internally change
CallSpec2
to have a list of marks instead of a broken mapping of keywords. This removes the keywords attribute of the internalCallSpec2
class. (#2672)Remove ParameterSet.deprecated_arg_dict - its not a public api and the lack of the underscore was a naming error. (#2675)
Remove the internal multi-typed attribute
Node._evalskip
and replace it with the booleanNode._skipped_by_mark
. (#2767)The
params
list passed topytest.fixture
is now for all effects considered immutable and frozen at the moment of thepytest.fixture
call. Previously the list could be changed before the first invocation of the fixture allowing for a form of dynamic parametrization (for example, updated from command-line options), but this was an unwanted implementation detail which complicated the internals and prevented some internal cleanup. See issue #2959 for details and a recommended workaround.
Features¶
pytest_fixture_post_finalizer
hook can now receive arequest
argument. (#2124)Replace the old introspection code in compat.py that determines the available arguments of fixtures with inspect.signature on Python 3 and funcsigs.signature on Python 2. This should respect
__signature__
declarations on functions. (#2267)Report tests with global
pytestmark
variable only once. (#2549)Now pytest displays the total progress percentage while running tests. The previous output style can be set by configuring the
console_output_style
setting toclassic
. (#2657)Match
warns
signature toraises
by addingmatch
keyword. (#2708)pytest now captures and displays output from the standard
logging
module. The user can control the logging level to be captured by specifying options inpytest.ini
, the command line and also during individual tests using markers. Also, acaplog
fixture is available that enables users to test the captured log during specific tests (similar tocapsys
for example). For more information, please see the logging docs. This feature was introduced by merging the popular pytest-catchlog plugin, thanks to @thisch. Be advised that during the merging the backward compatibility interface with the defunctpytest-capturelog
has been dropped. (#2794)Add
allow_module_level
kwarg topytest.skip()
, enabling to skip the whole module. (#2808)Allow setting
file_or_dir
,-c
, and-o
in PYTEST_ADDOPTS. (#2824)Return stdout/stderr capture results as a
namedtuple
, soout
anderr
can be accessed by attribute. (#2879)Add
capfdbinary
, a version ofcapfd
which returns bytes fromreadouterr()
. (#2923)Add
capsysbinary
a version ofcapsys
which returns bytes fromreadouterr()
. (#2934)Implement feature to skip
setup.py
files when run with--doctest-modules
. (#502)
Bug Fixes¶
Resume output capturing after
capsys/capfd.disabled()
context manager. (#1993)pytest_fixture_setup
andpytest_fixture_post_finalizer
hooks are now called for allconftest.py
files. (#2124)If an exception happens while loading a plugin, pytest no longer hides the original traceback. In Python 2 it will show the original traceback with a new message that explains in which plugin. In Python 3 it will show 2 canonized exceptions, the original exception while loading the plugin in addition to an exception that pytest throws about loading a plugin. (#2491)
capsys
andcapfd
can now be used by other fixtures. (#2709)Internal
pytester
plugin properly encodesbytes
arguments toutf-8
. (#2738)testdir
now uses use the same method used bytmpdir
to create its temporary directory. This changes the final structure of thetestdir
directory slightly, but should not affect usage in normal scenarios and avoids a number of potential problems. (#2751)pytest no longer complains about warnings with unicode messages being non-ascii compatible even for ascii-compatible messages. As a result of this, warnings with unicode messages are converted first to an ascii representation for safety. (#2809)
Change return value of pytest command when
--maxfail
is reached from2
(interrupted) to1
(failed). (#2845)Fix issue in assertion rewriting which could lead it to rewrite modules which should not be rewritten. (#2939)
Handle marks without description in
pytest.ini
. (#2942)
Trivial/Internal Changes¶
pytest now depends on attrs for internal structures to ease code maintainability. (#2641)
Refactored internal Python 2/3 compatibility code to use
six
. (#2642)Stop vendoring
pluggy
- we’re missing out on its latest changes for not much benefit (#2719)Internal refactor: simplify ascii string escaping by using the backslashreplace error handler in newer Python 3 versions. (#2734)
Remove unnecessary mark evaluator in unittest plugin (#2767)
Calls to
Metafunc.addcall
now emit a deprecation warning. This function is scheduled to be removed inpytest-4.0
. (#2876)Internal move of the parameterset extraction to a more maintainable place. (#2877)
Internal refactoring to simplify scope node lookup. (#2910)
Configure
pytest
to prevent pip from installing pytest in unsupported Python versions. (#2922)
pytest 3.2.5 (2017-11-15)¶
Bug Fixes¶
Remove
py<1.5
restriction frompytest
as this can cause version conflicts in some installations. (#2926)
pytest 3.2.4 (2017-11-13)¶
Bug Fixes¶
Fix the bug where running with
--pyargs
will result in items with emptyparent.nodeid
if run from a different root directory. (#2775)Fix issue with
@pytest.parametrize
if argnames was specified as keyword arguments. (#2819)Strip whitespace from marker names when reading them from INI config. (#2856)
Show full context of doctest source in the pytest output, if the line number of failed example in the docstring is < 9. (#2882)
Match fixture paths against actual path segments in order to avoid matching folders which share a prefix. (#2836)
Improved Documentation¶
Introduce a dedicated section about conftest.py. (#1505)
Explicitly mention
xpass
in the documentation ofxfail
. (#1997)Append example for pytest.param in the example/parametrize document. (#2658)
Clarify language of proposal for fixtures parameters (#2893)
List python 3.6 in the documented supported versions in the getting started document. (#2903)
Clarify the documentation of available fixture scopes. (#538)
Add documentation about the
python -m pytest
invocation adding the current directory to sys.path. (#911)
pytest 3.2.3 (2017-10-03)¶
Bug Fixes¶
Fix crash in tab completion when no prefix is given. (#2748)
The equality checking function (
__eq__
) ofMarkDecorator
returnsFalse
if one object is not an instance ofMarkDecorator
. (#2758)When running
pytest --fixtures-per-test
: don’t crash if an item has no _fixtureinfo attribute (e.g. doctests) (#2788)
Improved Documentation¶
Trivial/Internal Changes¶
pytest 3.2.2 (2017-09-06)¶
Bug Fixes¶
Calling the deprecated
request.getfuncargvalue()
now shows the source of the call. (#2681)Allow tests declared as
@staticmethod
to use fixtures. (#2699)Fixed edge-case during collection: attributes which raised
pytest.fail
when accessed would abort the entire collection. (#2707)Fix
ReprFuncArgs
with mixed unicode and UTF-8 args. (#2731)
Improved Documentation¶
In examples on working with custom markers, add examples demonstrating the usage of
pytest.mark.MARKER_NAME.with_args
in comparison withpytest.mark.MARKER_NAME.__call__
(#2604)In one of the simple examples, use
pytest_collection_modifyitems()
to skip tests based on a command-line option, allowing its sharing while preventing a user error when accessingpytest.config
before the argument parsing. (#2653)
Trivial/Internal Changes¶
pytest 3.2.1 (2017-08-08)¶
Bug Fixes¶
Fixed small terminal glitch when collecting a single test item. (#2579)
Correctly consider
/
as the file separator to automatically mark plugin files for rewrite on Windows. (#2591)Properly escape test names when setting
PYTEST_CURRENT_TEST
environment variable. (#2644)Fix error on Windows and Python 3.6+ when
sys.stdout
has been replaced with a stream-like object which does not implement the fullio
module buffer protocol. In particular this affectspytest-xdist
users on the aforementioned platform. (#2666)
Improved Documentation¶
Explicitly document which pytest features work with
unittest
. (#2626)
pytest 3.2.0 (2017-07-30)¶
Deprecations and Removals¶
pytest.approx
no longer supports>
,>=
,<
and<=
operators to avoid surprising/inconsistent behavior. See theapprox()
docs for more information. (#2003)All old-style specific behavior in current classes in the pytest’s API is considered deprecated at this point and will be removed in a future release. This affects Python 2 users only and in rare situations. (#2147)
A deprecation warning is now raised when using marks for parameters in
pytest.mark.parametrize
. Usepytest.param
to apply marks to parameters instead. (#2427)
Features¶
Add support for numpy arrays (and dicts) to approx. (#1994)
Now test function objects have a
pytestmark
attribute containing a list of marks applied directly to the test function, as opposed to marks inherited from parent classes or modules. (#2516)Collection ignores local virtualenvs by default;
--collect-in-virtualenv
overrides this behavior. (#2518)Allow class methods decorated as
@staticmethod
to be candidates for collection as a test function. (Only for Python 2.7 and above. Python 2.6 will still ignore static methods.) (#2528)Introduce
mark.with_args
in order to allow passing functions/classes as sole argument to marks. (#2540)New
cache_dir
ini option: sets the directory where the contents of the cache plugin are stored. Directory may be relative or absolute path: if relative path, then directory is created relative torootdir
, otherwise it is used as is. Additionally path may contain environment variables which are expanded during runtime. (#2543)Introduce the
PYTEST_CURRENT_TEST
environment variable that is set with thenodeid
and stage (setup
,call
andteardown
) of the test being currently executed. See the documentation for more info. (#2583)Introduced
@pytest.mark.filterwarnings
mark which allows overwriting the warnings filter on a per test, class or module level. See the docs for more information. (#2598)--last-failed
now remembers forever when a test has failed and only forgets it if it passes again. This makes it easy to fix a test suite by selectively running files and fixing tests incrementally. (#2621)New
pytest_report_collectionfinish
hook which allows plugins to add messages to the terminal reporting after collection has been finished successfully. (#2622)Added support for PEP 415's
Exception.__suppress_context__
. Now if araise exception from None
is caught by pytest, pytest will no longer chain the context in the test report. The behavior now matches Python’s traceback behavior. (#2631)Exceptions raised by
pytest.fail
,pytest.skip
andpytest.xfail
now subclass BaseException, making them harder to be caught unintentionally by normal code. (#580)
Bug Fixes¶
Set
stdin
to a closedPIPE
inpytester.py.Testdir.popen()
for avoid unwanted interactivepdb
(#2023)Add missing
encoding
attribute tosys.std*
streams when usingcapsys
capture mode. (#2375)Fix terminal color changing to black on Windows if
colorama
is imported in aconftest.py
file. (#2510)Fix line number when reporting summary of skipped tests. (#2548)
capture: ensure that EncodedFile.name is a string. (#2555)
The options
--fixtures
and--fixtures-per-test
will now keep indentation within docstrings. (#2574)doctests line numbers are now reported correctly, fixing pytest-sugar#122. (#2610)
Fix non-determinism in order of fixture collection. Adds new dependency (ordereddict) for Python 2.6. (#920)
Improved Documentation¶
Trivial/Internal Changes¶
Update help message for
--strict
to make it clear it only deals with unregistered markers, not warnings. (#2444)Internal code move: move code for pytest.approx/pytest.raises to own files in order to cut down the size of python.py (#2489)
Renamed the utility function
_pytest.compat._escape_strings
to_ascii_escaped
to better communicate the function’s purpose. (#2533)Improve error message for CollectError with skip/skipif. (#2546)
Emit warning about
yield
tests being deprecated only once per generator. (#2562)Ensure final collected line doesn’t include artifacts of previous write. (#2571)
Fixed all flake8 errors and warnings. (#2581)
Added
fix-lint
tox environment to run automatic pep8 fixes on the code. (#2582)Turn warnings into errors in pytest’s own test suite in order to catch regressions due to deprecations more promptly. (#2588)
Show multiple issue links in CHANGELOG entries. (#2620)
pytest 3.1.3 (2017-07-03)¶
Bug Fixes¶
Fix decode error in Python 2 for doctests in docstrings. (#2434)
Exceptions raised during teardown by finalizers are now suppressed until all finalizers are called, with the initial exception reraised. (#2440)
Fix incorrect “collected items” report when specifying tests on the command- line. (#2464)
deprecated_call
in context-manager form now captures deprecation warnings even if the same warning has already been raised. Also,deprecated_call
will always produce the same error message (previously it would produce different messages in context-manager vs. function-call mode). (#2469)Fix issue where paths collected by pytest could have triple leading
/
characters. (#2475)Fix internal error when trying to detect the start of a recursive traceback. (#2486)
Improved Documentation¶
Explicitly state for which hooks the calls stop after the first non-None result. (#2493)
Trivial/Internal Changes¶
pytest 3.1.2 (2017-06-08)¶
Bug Fixes¶
Required options added via
pytest_addoption
will no longer prevent using –help without passing them. (#1999)Respect
python_files
in assertion rewriting. (#2121)Fix recursion error detection when frames in the traceback contain objects that can’t be compared (like
numpy
arrays). (#2459)UnicodeWarning
is issued from the internal pytest warnings plugin only when the message contains non-ascii unicode (Python 2 only). (#2463)Added a workaround for Python 3.6
WindowsConsoleIO
breaking due to Pytests’sFDCapture
. Other code using console handles might still be affected by the very same issue and might require further workarounds/fixes, i.e.colorama
. (#2467)
Improved Documentation¶
Fix internal API links to
pluggy
objects. (#2331)Make it clear that
pytest.xfail
stops test execution at the calling point and improve overall flow of theskipping
docs. (#810)
pytest 3.1.1 (2017-05-30)¶
Bug Fixes¶
pytest warning capture no longer overrides existing warning filters. The previous behaviour would override all filters and caused regressions in test suites which configure warning filters to match their needs. Note that as a side-effect of this is that
DeprecationWarning
andPendingDeprecationWarning
are no longer shown by default. (#2430)Fix issue with non-ascii contents in doctest text files. (#2434)
Fix encoding errors for unicode warnings in Python 2. (#2436)
pytest.deprecated_call
now capturesPendingDeprecationWarning
in context manager form. (#2441)
Improved Documentation¶
Addition of towncrier for changelog management. (#2390)
3.1.0 (2017-05-22)¶
New Features¶
The
pytest-warnings
plugin has been integrated into the core and nowpytest
automatically captures and displays warnings at the end of the test session.Warning
This feature may disrupt test suites which apply and treat warnings themselves, and can be disabled in your
pytest.ini
:[pytest] addopts = -p no:warnings
See the warnings documentation page for more information.
Thanks @nicoddemus for the PR.
Added
junit_suite_name
ini option to specify root<testsuite>
name for JUnit XML reports (#533).Added an ini option
doctest_encoding
to specify which encoding to use for doctest files. Thanks @wheerd for the PR (#2101).pytest.warns
now checks for subclass relationship rather than class equality. Thanks @lesteve for the PR (#2166)pytest.raises
now asserts that the error message matches a text or regex with thematch
keyword argument. Thanks @Kriechi for the PR.pytest.param
can be used to declare test parameter sets with marks and test ids. Thanks @RonnyPfannschmidt for the PR.
Changes¶
remove all internal uses of pytest_namespace hooks, this is to prepare the removal of preloadconfig in pytest 4.0 Thanks to @RonnyPfannschmidt for the PR.
pytest now warns when a callable ids raises in a parametrized test. Thanks @fogo for the PR.
It is now possible to skip test classes from being collected by setting a
__test__
attribute toFalse
in the class body (#2007). Thanks to @syre for the report and @lwm for the PR.Change junitxml.py to produce reports that comply with Junitxml schema. If the same test fails with failure in call and then errors in teardown we split testcase element into two, one containing the error and the other the failure. (#2228) Thanks to @kkoukiou for the PR.
Testcase reports with a
url
attribute will now properly write this to junitxml. Thanks @fushi for the PR (#1874).Remove common items from dict comparison output when verbosity=1. Also update the truncation message to make it clearer that pytest truncates all assertion messages if verbosity < 2 (#1512). Thanks @mattduck for the PR
--pdbcls
no longer implies--pdb
. This makes it possible to useaddopts=--pdbcls=module.SomeClass
onpytest.ini
. Thanks @davidszotten for the PR (#1952).fix #2013: turn RecordedWarning into
namedtuple
, to give it a comprehensible repr while preventing unwarranted modification.fix #2208: ensure an iteration limit for
_pytest.compat.get_real_func
. Thanks @RonnyPfannschmidt for the report and PR.Hooks are now verified after collection is complete, rather than right after loading installed plugins. This makes it easy to write hooks for plugins which will be loaded during collection, for example using the
pytest_plugins
special variable (#1821). Thanks @nicoddemus for the PR.Modify
pytest_make_parametrize_id()
hook to acceptargname
as an additional parameter. Thanks @unsignedint for the PR.Add
venv
to the defaultnorecursedirs
setting. Thanks @The-Compiler for the PR.PluginManager.import_plugin
now accepts unicode plugin names in Python 2. Thanks @reutsharabani for the PR.fix #2308: When using both
--lf
and--ff
, only the last failed tests are run. Thanks @ojii for the PR.Replace minor/patch level version numbers in the documentation with placeholders. This significantly reduces change-noise as different contributors regenerate the documentation on different platforms. Thanks @RonnyPfannschmidt for the PR.
fix #2391: consider pytest_plugins on all plugin modules Thanks @RonnyPfannschmidt for the PR.
Bug Fixes¶
Fix
AttributeError
onsys.stdout.buffer
/sys.stderr.buffer
while usingcapsys
fixture in python 3. (#1407). Thanks to @asottile.Change capture.py’s
DontReadFromInput
class to throwio.UnsupportedOperation
errors rather than ValueErrors in thefileno
method (#2276). Thanks @metasyn and @vlad-dragos for the PR.Fix exception formatting while importing modules when the exception message contains non-ascii characters (#2336). Thanks @fabioz for the report and @nicoddemus for the PR.
Added documentation related to issue (#1937) Thanks @skylarjhdownes for the PR.
Allow collecting files with any file extension as Python modules (#2369). Thanks @Kodiologist for the PR.
Show the correct error message when collect “parametrize” func with wrong args (#2383). Thanks @The-Compiler for the report and @robin0371 for the PR.
3.0.7 (2017-03-14)¶
Fix issue in assertion rewriting breaking due to modules silently discarding other modules when importing fails Notably, importing the
anydbm
module is fixed. (#2248). Thanks @pfhayes for the PR.junitxml: Fix problematic case where system-out tag occurred twice per testcase element in the XML report. Thanks @kkoukiou for the PR.
Fix regression, pytest now skips unittest correctly if run with
--pdb
(#2137). Thanks to @gst for the report and @mbyt for the PR.Ignore exceptions raised from descriptors (e.g. properties) during Python test collection (#2234). Thanks to @bluetech.
--override-ini
now correctly overrides some fundamental options likepython_files
(#2238). Thanks @sirex for the report and @nicoddemus for the PR.Replace
raise StopIteration
usages in the code by simplereturns
to finish generators, in accordance to PEP 479 (#2160). Thanks to @nicoddemus for the PR.Fix internal errors when an unprintable
AssertionError
is raised inside a test. Thanks @omerhadari for the PR.Skipping plugin now also works with test items generated by custom collectors (#2231). Thanks to @vidartf.
Fix trailing whitespace in console output if no .ini file presented (#2281). Thanks @fbjorn for the PR.
Conditionless
xfail
markers no longer rely on the underlying test item being an instance ofPyobjMixin
, and can therefore apply to tests not collected by the built-in python test collector. Thanks @barneygale for the PR.
3.0.6 (2017-01-22)¶
pytest no longer generates
PendingDeprecationWarning
from its own operations, which was introduced by mistake in version3.0.5
(#2118). Thanks to @nicoddemus for the report and @RonnyPfannschmidt for the PR.pytest no longer recognizes coroutine functions as yield tests (#2129). Thanks to @malinoff for the PR.
Plugins loaded by the
PYTEST_PLUGINS
environment variable are now automatically considered for assertion rewriting (#2185). Thanks @nicoddemus for the PR.Improve error message when pytest.warns fails (#2150). The type(s) of the expected warnings and the list of caught warnings is added to the error message. Thanks @lesteve for the PR.
Fix
pytester
internal plugin to work correctly with latest versions ofzope.interface
(#1989). Thanks @nicoddemus for the PR.Assert statements of the
pytester
plugin again benefit from assertion rewriting (#1920). Thanks @RonnyPfannschmidt for the report and @nicoddemus for the PR.Specifying tests with colons like
test_foo.py::test_bar
for tests in subdirectories with ini configuration files now uses the correct ini file (#2148). Thanks @pelme.Fail
testdir.runpytest().assert_outcomes()
explicitly if the pytest terminal output it relies on is missing. Thanks to @eli-b for the PR.
3.0.5 (2016-12-05)¶
Add warning when not passing
option=value
correctly to-o/--override-ini
(#2105). Also improved the help documentation. Thanks to @mbukatov for the report and @lwm for the PR.Now
--confcutdir
and--junit-xml
are properly validated if they are directories and filenames, respectively (#2089 and #2078). Thanks to @lwm for the PR.Add hint to error message hinting possible missing
__init__.py
(#478). Thanks @DuncanBetts.More accurately describe when fixture finalization occurs in documentation (#687). Thanks @DuncanBetts.
Provide
:ref:
targets forrecwarn.rst
so we can use intersphinx referencing. Thanks to @dupuy for the report and @lwm for the PR.In Python 2, use a simple
+-
ASCII string in the string representation ofpytest.approx
(for example"4 +- 4.0e-06"
) because it is brittle to handle that in different contexts and representations internally in pytest which can result in bugs such as #2111. In Python 3, the representation still uses±
(for example4 ± 4.0e-06
). Thanks @kerrick-lyft for the report and @nicoddemus for the PR.Using
item.Function
,item.Module
, etc., is now issuing deprecation warnings, preferpytest.Function
,pytest.Module
, etc., instead (#2034). Thanks @nmundar for the PR.Fix error message using
approx
with complex numbers (#2082). Thanks @adler-j for the report and @nicoddemus for the PR.Fixed false-positives warnings from assertion rewrite hook for modules imported more than once by the
pytest_plugins
mechanism. Thanks @nicoddemus for the PR.Remove an internal cache which could cause hooks from
conftest.py
files in sub-directories to be called in other directories incorrectly (#2016). Thanks @d-b-w for the report and @nicoddemus for the PR.Remove internal code meant to support earlier Python 3 versions that produced the side effect of leaving
None
insys.modules
when expressions were evaluated by pytest (for example passing a condition as a string topytest.mark.skipif
)(#2103). Thanks @jaraco for the report and @nicoddemus for the PR.Cope gracefully with a .pyc file with no matching .py file (#2038). Thanks @nedbat.
3.0.4 (2016-11-09)¶
Import errors when collecting test modules now display the full traceback (#1976). Thanks @cwitty for the report and @nicoddemus for the PR.
Fix confusing command-line help message for custom options with two or more
metavar
properties (#2004). Thanks @okulynyak and @davehunt for the report and @nicoddemus for the PR.When loading plugins, import errors which contain non-ascii messages are now properly handled in Python 2 (#1998). Thanks @nicoddemus for the PR.
Fixed cyclic reference when
pytest.raises
is used in context-manager form (#1965). Also as a result of this fix,sys.exc_info()
is left empty in both context-manager and function call usages. Previously,sys.exc_info
would contain the exception caught by the context manager, even when the expected exception occurred. Thanks @MSeifert04 for the report and the PR.Fixed false-positives warnings from assertion rewrite hook for modules that were rewritten but were later marked explicitly by
pytest.register_assert_rewrite
or implicitly as a plugin (#2005). Thanks @RonnyPfannschmidt for the report and @nicoddemus for the PR.Report teardown output on test failure (#442). Thanks @matclab for the PR.
Fix teardown error message in generated xUnit XML. Thanks @gdyuldin for the PR.
Properly handle exceptions in
multiprocessing
tasks (#1984). Thanks @adborden for the report and @nicoddemus for the PR.Clean up unittest TestCase objects after tests are complete (#1649). Thanks @d-b-w for the report and PR.
3.0.3 (2016-09-28)¶
The
ids
argument toparametrize
again acceptsunicode
strings in Python 2 (#1905). Thanks @philpep for the report and @nicoddemus for the PR.Assertions are now being rewritten for plugins in development mode (
pip install -e
) (#1934). Thanks @nicoddemus for the PR.Fix pkg_resources import error in Jython projects (#1853). Thanks @raquelalegre for the PR.
Got rid of
AttributeError: 'Module' object has no attribute '_obj'
exception in Python 3 (#1944). Thanks @axil for the PR.Explain a bad scope value passed to
@fixture
declarations or aMetaFunc.parametrize()
call.This version includes
pluggy-0.4.0
, which correctly handlesVersionConflict
errors in plugins (#704). Thanks @nicoddemus for the PR.
3.0.2 (2016-09-01)¶
Improve error message when passing non-string ids to
pytest.mark.parametrize
(#1857). Thanks @okken for the report and @nicoddemus for the PR.Add
buffer
attribute to stdin stub classpytest.capture.DontReadFromInput
Thanks @joguSD for the PR.Fix
UnicodeEncodeError
when string comparison with unicode has failed. (#1864) Thanks @AiOO for the PR.pytest_plugins
is now handled correctly if defined as a string (as opposed as a sequence of strings) when modules are considered for assertion rewriting. Due to this bug, much more modules were being rewritten than necessary if a test suite usespytest_plugins
to load internal plugins (#1888). Thanks @jaraco for the report and @nicoddemus for the PR (#1891).Do not call tearDown and cleanups when running tests from
unittest.TestCase
subclasses with--pdb
enabled. This allows proper post mortem debugging for all applications which have significant logic in their tearDown machinery (#1890). Thanks @mbyt for the PR.Fix use of deprecated
getfuncargvalue
method in the internal doctest plugin. Thanks @ViviCoder for the report (#1898).
3.0.1 (2016-08-23)¶
Fix regression when
importorskip
is used at module level (#1822). Thanks @jaraco and @The-Compiler for the report and @nicoddemus for the PR.Fix parametrization scope when session fixtures are used in conjunction with normal parameters in the same call (#1832). Thanks @The-Compiler for the report, @Kingdread and @nicoddemus for the PR.
Fix internal error when parametrizing tests or fixtures using an empty
ids
argument (#1849). Thanks @OPpuolitaival for the report and @nicoddemus for the PR.Fix loader error when running
pytest
embedded in a zipfile. Thanks @mbachry for the PR.
3.0.0 (2016-08-18)¶
Incompatible changes
A number of incompatible changes were made in this release, with the intent of removing features deprecated for a long time or change existing behaviors in order to make them less surprising/more useful.
Reinterpretation mode has now been removed. Only plain and rewrite mode are available, consequently the
--assert=reinterp
option is no longer available. This also means files imported from plugins orconftest.py
will not benefit from improved assertions by default, you should usepytest.register_assert_rewrite()
to explicitly turn on assertion rewriting for those files. Thanks @flub for the PR.The following deprecated commandline options were removed:
--genscript
: no longer supported;--no-assert
: use--assert=plain
instead;--nomagic
: use--assert=plain
instead;--report
: use-r
instead;
Thanks to @RedBeardCode for the PR (#1664).
ImportErrors in plugins now are a fatal error instead of issuing a pytest warning (#1479). Thanks to @The-Compiler for the PR.
Removed support code for Python 3 versions < 3.3 (#1627).
Removed all
py.test-X*
entry points. The versioned, suffixed entry points were never documented and a leftover from a pre-virtualenv era. These entry points also created broken entry points in wheels, so removing them also removes a source of confusion for users (#1632). Thanks @obestwalter for the PR.pytest.skip()
now raises an error when used to decorate a test function, as opposed to its original intent (to imperatively skip a test inside a test function). Previously this usage would cause the entire module to be skipped (#607). Thanks @omarkohl for the complete PR (#1519).Exit tests if a collection error occurs. A poll indicated most users will hit CTRL-C anyway as soon as they see collection errors, so pytest might as well make that the default behavior (#1421). A
--continue-on-collection-errors
option has been added to restore the previous behaviour. Thanks @olegpidsadnyi and @omarkohl for the complete PR (#1628).Renamed the pytest
pdb
module (plugin) intodebugging
to avoid clashes with the builtinpdb
module.Raise a helpful failure message when requesting a parametrized fixture at runtime, e.g. with
request.getfixturevalue
. Previously these parameters were simply never defined, so a fixture decorated like@pytest.fixture(params=[0, 1, 2])
only ran once (#460). Thanks to @nikratio for the bug report, @RedBeardCode and @tomviner for the PR._pytest.monkeypatch.monkeypatch
class has been renamed to_pytest.monkeypatch.MonkeyPatch
so it doesn’t conflict with themonkeypatch
fixture.--exitfirst / -x
can now be overridden by a following--maxfail=N
and is just a synonym for--maxfail=1
.
New Features
Support nose-style
__test__
attribute on methods of classes, including unittest-style Classes. If set toFalse
, the test will not be collected.New
doctest_namespace
fixture for injecting names into the namespace in which doctests run. Thanks @milliams for the complete PR (#1428).New
--doctest-report
option available to change the output format of diffs when running (failing) doctests (implements #1749). Thanks @hartym for the PR.New
name
argument topytest.fixture
decorator which allows a custom name for a fixture (to solve the funcarg-shadowing-fixture problem). Thanks @novas0x2a for the complete PR (#1444).New
approx()
function for easily comparing floating-point numbers in tests. Thanks @kalekundert for the complete PR (#1441).Ability to add global properties in the final xunit output file by accessing the internal
junitxml
plugin (experimental). Thanks @tareqalayan for the complete PR #1454).New
ExceptionInfo.match()
method to match a regular expression on the string representation of an exception (#372). Thanks @omarkohl for the complete PR (#1502).__tracebackhide__
can now also be set to a callable which then can decide whether to filter the traceback based on theExceptionInfo
object passed to it. Thanks @The-Compiler for the complete PR (#1526).New
pytest_make_parametrize_id(config, val)
hook which can be used by plugins to provide friendly strings for custom types. Thanks @palaviv for the PR.capsys
andcapfd
now have adisabled()
context-manager method, which can be used to temporarily disable capture within a test. Thanks @nicoddemus for the PR.New cli flag
--fixtures-per-test
: shows which fixtures are being used for each selected test item. Features doc strings of fixtures by default. Can also show where fixtures are defined if combined with-v
. Thanks @hackebrot for the PR.Introduce
pytest
command as recommended entry point. Note thatpy.test
still works and is not scheduled for removal. Closes proposal #1629. Thanks @obestwalter and @davehunt for the complete PR (#1633).New cli flags:
--setup-plan
: performs normal collection and reports the potential setup and teardown and does not execute any fixtures and tests;--setup-only
: performs normal collection, executes setup and teardown of fixtures and reports them;--setup-show
: performs normal test execution and additionally shows setup and teardown of fixtures;--keep-duplicates
: py.test now ignores duplicated paths given in the command line. To retain the previous behavior where the same test could be run multiple times by specifying it in the command-line multiple times, pass the--keep-duplicates
argument (#1609);
Thanks @d6e, @kvas-it, @sallner, @ioggstream and @omarkohl for the PRs.
New CLI flag
--override-ini
/-o
: overrides values from the ini file. For example:"-o xfail_strict=True"
’. Thanks @blueyed and @fengxx for the PR.New hooks:
pytest_fixture_setup(fixturedef, request)
: executes fixture setup;pytest_fixture_post_finalizer(fixturedef)
: called after the fixture’s finalizer and has access to the fixture’s result cache.
Issue warnings for asserts whose test is a tuple literal. Such asserts will never fail because tuples are always truthy and are usually a mistake (see #1562). Thanks @kvas-it, for the PR.
Allow passing a custom debugger class (e.g.
--pdbcls=IPython.core.debugger:Pdb
). Thanks to @anntzer for the PR.
Changes
Plugins now benefit from assertion rewriting. Thanks @sober7, @nicoddemus and @flub for the PR.
Change
report.outcome
forxpassed
tests to"passed"
in non-strict mode and"failed"
in strict mode. Thanks to @hackebrot for the PR (#1795) and @gprasad84 for report (#1546).Tests marked with
xfail(strict=False)
(the default) now appear in JUnitXML reports as passing tests instead of skipped. Thanks to @hackebrot for the PR (#1795).Highlight path of the file location in the error report to make it easier to copy/paste. Thanks @suzaku for the PR (#1778).
Fixtures marked with
@pytest.fixture
can now useyield
statements exactly like those marked with the@pytest.yield_fixture
decorator. This change renders@pytest.yield_fixture
deprecated and makes@pytest.fixture
withyield
statements the preferred way to write teardown code (#1461). Thanks @csaftoiu for bringing this to attention and @nicoddemus for the PR.Explicitly passed parametrize ids do not get escaped to ascii (#1351). Thanks @ceridwen for the PR.
Fixtures are now sorted in the error message displayed when an unknown fixture is declared in a test function. Thanks @nicoddemus for the PR.
pytest_terminal_summary
hook now receives theexitstatus
of the test session as argument. Thanks @blueyed for the PR (#1809).Parametrize ids can accept
None
as specific test id, in which case the automatically generated id for that argument will be used. Thanks @palaviv for the complete PR (#1468).The parameter to xunit-style setup/teardown methods (
setup_method
,setup_module
, etc.) is now optional and may be omitted. Thanks @okken for bringing this to attention and @nicoddemus for the PR.Improved automatic id generation selection in case of duplicate ids in parametrize. Thanks @palaviv for the complete PR (#1474).
Now pytest warnings summary is shown up by default. Added a new flag
--disable-pytest-warnings
to explicitly disable the warnings summary (#1668).Make ImportError during collection more explicit by reminding the user to check the name of the test module/package(s) (#1426). Thanks @omarkohl for the complete PR (#1520).
Add
build/
anddist/
to the default--norecursedirs
list. Thanks @mikofski for the report and @tomviner for the PR (#1544).pytest.raises
in the context manager form accepts a custommessage
to raise when no exception occurred. Thanks @palaviv for the complete PR (#1616).conftest.py
files now benefit from assertion rewriting; previously it was only available for test modules. Thanks @flub, @sober7 and @nicoddemus for the PR (#1619).Text documents without any doctests no longer appear as “skipped”. Thanks @graingert for reporting and providing a full PR (#1580).
Ensure that a module within a namespace package can be found when it is specified on the command line together with the
--pyargs
option. Thanks to @taschini for the PR (#1597).Always include full assertion explanation during assertion rewriting. The previous behaviour was hiding sub-expressions that happened to be
False
, assuming this was redundant information. Thanks @bagerard for reporting (#1503). Thanks to @davehunt and @tomviner for the PR.OptionGroup.addoption()
now checks if option names were already added before, to make it easier to track down issues like #1618. Before, you only got exceptions later fromargparse
library, giving no clue about the actual reason for double-added options.yield
-based tests are considered deprecated and will be removed in pytest-4.0. Thanks @nicoddemus for the PR.[pytest]
sections insetup.cfg
files should now be named[tool:pytest]
to avoid conflicts with other distutils commands (see #567).[pytest]
sections inpytest.ini
ortox.ini
files are supported and unchanged. Thanks @nicoddemus for the PR.Using
pytest_funcarg__
prefix to declare fixtures is considered deprecated and will be removed in pytest-4.0 (#1684). Thanks @nicoddemus for the PR.Passing a command-line string to
pytest.main()
is considered deprecated and scheduled for removal in pytest-4.0. It is recommended to pass a list of arguments instead (#1723).Rename
getfuncargvalue
togetfixturevalue
.getfuncargvalue
is still present but is now considered deprecated. Thanks to @RedBeardCode and @tomviner for the PR (#1626).optparse
type usage now triggers DeprecationWarnings (#1740).optparse
backward compatibility supports float/complex types (#457).Refined logic for determining the
rootdir
, considering only valid paths which fixes a number of issues: #1594, #1435 and #1471. Updated the documentation according to current behavior. Thanks to @blueyed, @davehunt and @matthiasha for the PR.Always include full assertion explanation. The previous behaviour was hiding sub-expressions that happened to be False, assuming this was redundant information. Thanks @bagerard for reporting (#1503). Thanks to @davehunt and @tomviner for PR.
Better message in case of not using parametrized variable (see #1539). Thanks to @tramwaj29 for the PR.
Updated docstrings with a more uniform style.
Add stderr write for
pytest.exit(msg)
during startup. Previously the message was never shown. Thanks @BeyondEvil for reporting #1210. Thanks to @jgsonesen and @tomviner for the PR.No longer display the incorrect test deselection reason (#1372). Thanks @ronnypfannschmidt for the PR.
The
--resultlog
command line option has been deprecated: it is little used and there are more modern and better alternatives (see #830). Thanks @nicoddemus for the PR.Improve error message with fixture lookup errors: add an ‘E’ to the first line and ‘>’ to the rest. Fixes #717. Thanks @blueyed for reporting and a PR, @eolo999 for the initial PR and @tomviner for his guidance during EuroPython2016 sprint.
Bug Fixes
Parametrize now correctly handles duplicated test ids.
Fix internal error issue when the
method
argument is missing forteardown_method()
(#1605).Fix exception visualization in case the current working directory (CWD) gets deleted during testing (#1235). Thanks @bukzor for reporting. PR by @marscher.
Improve test output for logical expression with brackets (#925). Thanks @DRMacIver for reporting and @RedBeardCode for the PR.
Create correct diff for strings ending with newlines (#1553). Thanks @Vogtinator for reporting and @RedBeardCode and @tomviner for the PR.
ConftestImportFailure
now shows the traceback making it easier to identify bugs inconftest.py
files (#1516). Thanks @txomon for the PR.Text documents without any doctests no longer appear as “skipped”. Thanks @graingert for reporting and providing a full PR (#1580).
Fixed collection of classes with custom
__new__
method. Fixes #1579. Thanks to @Stranger6667 for the PR.Fixed scope overriding inside metafunc.parametrize (#634). Thanks to @Stranger6667 for the PR.
Fixed the total tests tally in junit xml output (#1798). Thanks to @cboelsen for the PR.
Fixed off-by-one error with lines from
request.node.warn
. Thanks to @blueyed for the PR.
2.9.2 (2016-05-31)¶
Bug Fixes
fix #510: skip tests where one parameterize dimension was empty thanks Alex Stapleton for the Report and @RonnyPfannschmidt for the PR
Fix Xfail does not work with condition keyword argument. Thanks @astraw38 for reporting the issue (#1496) and @tomviner for PR the (#1524).
Fix win32 path issue when putting custom config file with absolute path in
pytest.main("-c your_absolute_path")
.Fix maximum recursion depth detection when raised error class is not aware of unicode/encoded bytes. Thanks @prusse-martin for the PR (#1506).
Fix
pytest.mark.skip
mark when used in strict mode. Thanks @pquentin for the PR and @RonnyPfannschmidt for showing how to fix the bug.Minor improvements and fixes to the documentation. Thanks @omarkohl for the PR.
Fix
--fixtures
to show all fixture definitions as opposed to just one per fixture name. Thanks to @hackebrot for the PR.
2.9.1 (2016-03-17)¶
Bug Fixes
Improve error message when a plugin fails to load. Thanks @nicoddemus for the PR.
Fix (#1178):
pytest.fail
with non-ascii characters raises an internal pytest error. Thanks @nicoddemus for the PR.Fix (#469): junit parses report.nodeid incorrectly, when params IDs contain
::
. Thanks @tomviner for the PR (#1431).Fix (#578): SyntaxErrors containing non-ascii lines at the point of failure generated an internal py.test error. Thanks @asottile for the report and @nicoddemus for the PR.
Fix (#1437): When passing in a bytestring regex pattern to parameterize attempt to decode it as utf-8 ignoring errors.
Fix (#649): parametrized test nodes cannot be specified to run on the command line.
Fix (#138): better reporting for python 3.3+ chained exceptions
2.9.0 (2016-02-29)¶
New Features
New
pytest.mark.skip
mark, which unconditionally skips marked tests. Thanks @MichaelAquilina for the complete PR (#1040).--doctest-glob
may now be passed multiple times in the command-line. Thanks @jab and @nicoddemus for the PR.New
-rp
and-rP
reporting options give the summary and full output of passing tests, respectively. Thanks to @codewarrior0 for the PR.pytest.mark.xfail
now has astrict
option, which makesXPASS
tests to fail the test suite (defaulting toFalse
). There’s also axfail_strict
ini option that can be used to configure it project-wise. Thanks @rabbbit for the request and @nicoddemus for the PR (#1355).Parser.addini
now supports options of typebool
. Thanks @nicoddemus for the PR.New
ALLOW_BYTES
doctest option. This stripsb
prefixes from byte strings in doctest output (similar toALLOW_UNICODE
). Thanks @jaraco for the request and @nicoddemus for the PR (#1287).Give a hint on
KeyboardInterrupt
to use the--fulltrace
option to show the errors. Fixes #1366. Thanks to @hpk42 for the report and @RonnyPfannschmidt for the PR.Catch
IndexError
exceptions when getting exception source location. Fixes a pytest internal error for dynamically generated code (fixtures and tests) where source lines are fake by intention.
Changes
Important: py.code has been merged into the
pytest
repository aspytest._code
. This decision was made becausepy.code
had very few uses outsidepytest
and the fact that it was in a different repository made it difficult to fix bugs on its code in a timely manner. The team hopes with this to be able to better refactor out and improve that code. This change shouldn’t affect users, but it is useful to let users aware if they encounter any strange behavior.Keep in mind that the code for
pytest._code
is private and experimental, so you definitely should not import it explicitly!Please note that the original
py.code
is still available in pylib.pytest_enter_pdb
now optionally receives the pytest config object. Thanks @nicoddemus for the PR.Removed code and documentation for Python 2.5 or lower versions, including removal of the obsolete
_pytest.assertion.oldinterpret
module. Thanks @nicoddemus for the PR (#1226).Comparisons now always show up in full when
CI
orBUILD_NUMBER
is found in the environment, even when-vv
isn’t used. Thanks @The-Compiler for the PR.--lf
and--ff
now support long names:--last-failed
and--failed-first
respectively. Thanks @MichaelAquilina for the PR.Added expected exceptions to
pytest.raises
fail message.Collection only displays progress (“collecting X items”) when in a terminal. This avoids cluttering the output when using
--color=yes
to obtain colors in CI integrations systems (#1397).
Bug Fixes
The
-s
and-c
options should now work underxdist
;Config.fromdictargs
now represents its input much more faithfully. Thanks to @bukzor for the complete PR (#680).Fix (#1290): support Python 3.5’s
@
operator in assertion rewriting. Thanks @Shinkenjoe for report with test case and @tomviner for the PR.Fix formatting utf-8 explanation messages (#1379). Thanks @biern for the PR.
Fix traceback style docs to describe all of the available options (auto/long/short/line/native/no), with
auto
being the default since v2.6. Thanks @hackebrot for the PR.Fix (#1422): junit record_xml_property doesn’t allow multiple records with same name.
2.8.7 (2016-01-24)¶
fix #1338: use predictable object resolution for monkeypatch
2.8.6 (2016-01-21)¶
fix #1259: allow for double nodeids in junitxml, this was a regression failing plugins combinations like pytest-pep8 + pytest-flakes
Workaround for exception that occurs in pyreadline when using
--pdb
with standard I/O capture enabled. Thanks Erik M. Bray for the PR.fix #900: Better error message in case the target of a
monkeypatch
call raises anImportError
.fix #1292: monkeypatch calls (setattr, setenv, etc.) are now O(1). Thanks David R. MacIver for the report and Bruno Oliveira for the PR.
fix #1223: captured stdout and stderr are now properly displayed before entering pdb when
--pdb
is used instead of being thrown away. Thanks Cal Leeming for the PR.fix #1305: pytest warnings emitted during
pytest_terminal_summary
are now properly displayed. Thanks Ionel Maries Cristian for the report and Bruno Oliveira for the PR.fix #628: fixed internal UnicodeDecodeError when doctests contain unicode. Thanks Jason R. Coombs for the report and Bruno Oliveira for the PR.
fix #1334: Add captured stdout to jUnit XML report on setup error. Thanks Georgy Dyuldin for the PR.
2.8.5 (2015-12-11)¶
fix #1243: fixed issue where class attributes injected during collection could break pytest. PR by Alexei Kozlenok, thanks Ronny Pfannschmidt and Bruno Oliveira for the review and help.
fix #1074: precompute junitxml chunks instead of storing the whole tree in objects Thanks Bruno Oliveira for the report and Ronny Pfannschmidt for the PR
fix #1238: fix
pytest.deprecated_call()
receiving multiple arguments (Regression introduced in 2.8.4). Thanks Alex Gaynor for the report and Bruno Oliveira for the PR.
2.8.4 (2015-12-06)¶
fix #1190:
deprecated_call()
now works when the deprecated function has been already called by another test in the same module. Thanks Mikhail Chernykh for the report and Bruno Oliveira for the PR.fix #1198:
--pastebin
option now works on Python 3. Thanks Mehdy Khoshnoody for the PR.fix #1219:
--pastebin
now works correctly when captured output contains non-ascii characters. Thanks Bruno Oliveira for the PR.fix #1204: another error when collecting with a nasty __getattr__(). Thanks Florian Bruhin for the PR.
fix the summary printed when no tests did run. Thanks Florian Bruhin for the PR.
fix #1185 - ensure MANIFEST.in exactly matches what should go to a sdist
a number of documentation modernizations wrt good practices. Thanks Bruno Oliveira for the PR.
2.8.3 (2015-11-18)¶
fix #1169: add __name__ attribute to testcases in TestCaseFunction to support the @unittest.skip decorator on functions and methods. Thanks Lee Kamentsky for the PR.
fix #1035: collecting tests if test module level obj has __getattr__(). Thanks Suor for the report and Bruno Oliveira / Tom Viner for the PR.
fix #331: don’t collect tests if their failure cannot be reported correctly e.g. they are a callable instance of a class.
fix #1133: fixed internal error when filtering tracebacks where one entry belongs to a file which is no longer available. Thanks Bruno Oliveira for the PR.
enhancement made to highlight in red the name of the failing tests so they stand out in the output. Thanks Gabriel Reis for the PR.
add more talks to the documentation
extend documentation on the –ignore cli option
use pytest-runner for setuptools integration
minor fixes for interaction with OS X El Capitan system integrity protection (thanks Florian)
2.8.2 (2015-10-07)¶
fix #1085: proper handling of encoding errors when passing encoded byte strings to pytest.parametrize in Python 2. Thanks Themanwithoutaplan for the report and Bruno Oliveira for the PR.
fix #1087: handling SystemError when passing empty byte strings to pytest.parametrize in Python 3. Thanks Paul Kehrer for the report and Bruno Oliveira for the PR.
fix #995: fixed internal error when filtering tracebacks where one entry was generated by an exec() statement. Thanks Daniel Hahler, Ashley C Straw, Philippe Gauthier and Pavel Savchenko for contributing and Bruno Oliveira for the PR.
fix #1100 and #1057: errors when using autouse fixtures and doctest modules. Thanks Sergey B Kirpichev and Vital Kudzelka for contributing and Bruno Oliveira for the PR.
2.8.1 (2015-09-29)¶
fix #1034: Add missing nodeid on pytest_logwarning call in addhook. Thanks Simon Gomizelj for the PR.
‘deprecated_call’ is now only satisfied with a DeprecationWarning or PendingDeprecationWarning. Before 2.8.0, it accepted any warning, and 2.8.0 made it accept only DeprecationWarning (but not PendingDeprecationWarning). Thanks Alex Gaynor for the issue and Eric Hunsberger for the PR.
fix issue #1073: avoid calling __getattr__ on potential plugin objects. This fixes an incompatibility with pytest-django. Thanks Andreas Pelme, Bruno Oliveira and Ronny Pfannschmidt for contributing and Holger Krekel for the fix.
Fix issue #704: handle versionconflict during plugin loading more gracefully. Thanks Bruno Oliveira for the PR.
Fix issue #1064: “”–junitxml” regression when used with the “pytest-xdist” plugin, with test reports being assigned to the wrong tests. Thanks Daniel Grunwald for the report and Bruno Oliveira for the PR.
(experimental) adapt more SEMVER style versioning and change meaning of master branch in git repo: “master” branch now keeps the bug fixes, changes aimed for micro releases. “features” branch will only be released with minor or major pytest releases.
Fix issue #766 by removing documentation references to distutils. Thanks Russel Winder.
Fix issue #1030: now byte-strings are escaped to produce item node ids to make them always serializable. Thanks Andy Freeland for the report and Bruno Oliveira for the PR.
Python 2: if unicode parametrized values are convertible to ascii, their ascii representation is used for the node id.
Fix issue #411: Add __eq__ method to assertion comparison example. Thanks Ben Webb.
Fix issue #653: deprecated_call can be used as context manager.
fix issue 877: properly handle assertion explanations with non-ascii repr Thanks Mathieu Agopian for the report and Ronny Pfannschmidt for the PR.
fix issue 1029: transform errors when writing cache values into pytest-warnings
2.8.0 (2015-09-18)¶
new
--lf
and-ff
options to run only the last failing tests or “failing tests first” from the last run. This functionality is provided through porting the formerly external pytest-cache plugin into pytest core. BACKWARD INCOMPAT: if you used pytest-cache’s functionality to persist data between test runs be aware that we don’t serialize sets anymore. Thanks Ronny Pfannschmidt for most of the merging work.“-r” option now accepts “a” to include all possible reports, similar to passing “fEsxXw” explicitly (issue960). Thanks Abhijeet Kasurde for the PR.
avoid python3.5 deprecation warnings by introducing version specific inspection helpers, thanks Michael Droettboom.
fix issue562: @nose.tools.istest now fully respected.
fix issue934: when string comparison fails and a diff is too large to display without passing -vv, still show a few lines of the diff. Thanks Florian Bruhin for the report and Bruno Oliveira for the PR.
fix issue736: Fix a bug where fixture params would be discarded when combined with parametrization markers. Thanks to Markus Unterwaditzer for the PR.
fix issue710: introduce ALLOW_UNICODE doctest option: when enabled, the
u
prefix is stripped from unicode strings in expected doctest output. This allows doctests which use unicode to run in Python 2 and 3 unchanged. Thanks Jason R. Coombs for the report and Bruno Oliveira for the PR.parametrize now also generates meaningful test IDs for enum, regex and class objects (as opposed to class instances). Thanks to Florian Bruhin for the PR.
Add ‘warns’ to assert that warnings are thrown (like ‘raises’). Thanks to Eric Hunsberger for the PR.
Fix issue683: Do not apply an already applied mark. Thanks ojake for the PR.
Deal with capturing failures better so fewer exceptions get lost to /dev/null. Thanks David Szotten for the PR.
fix issue730: deprecate and warn about the –genscript option. Thanks Ronny Pfannschmidt for the report and Christian Pommranz for the PR.
fix issue751: multiple parametrize with ids bug if it parametrizes class with two or more test methods. Thanks Sergey Chipiga for reporting and Jan Bednarik for PR.
fix issue82: avoid loading conftest files from setup.cfg/pytest.ini/tox.ini files and upwards by default (–confcutdir can still be set to override this). Thanks Bruno Oliveira for the PR.
fix issue768: docstrings found in python modules were not setting up session fixtures. Thanks Jason R. Coombs for reporting and Bruno Oliveira for the PR.
added
tmpdir_factory
, a session-scoped fixture that can be used to create directories under the base temporary directory. Previously this object was installed as a_tmpdirhandler
attribute of theconfig
object, but now it is part of the official API and usingconfig._tmpdirhandler
is deprecated. Thanks Bruno Oliveira for the PR.fix issue808: pytest’s internal assertion rewrite hook now implements the optional PEP 302 get_data API so tests can access data files next to them. Thanks xmo-odoo for request and example and Bruno Oliveira for the PR.
rootdir and inifile are now displayed during usage errors to help users diagnose problems such as unexpected ini files which add unknown options being picked up by pytest. Thanks to Pavel Savchenko for bringing the problem to attention in #821 and Bruno Oliveira for the PR.
Summary bar now is colored yellow for warning situations such as: all tests either were skipped or xpass/xfailed, or no tests were run at all (this is a partial fix for issue500).
fix issue812: pytest now exits with status code 5 in situations where no tests were run at all, such as the directory given in the command line does not contain any tests or as result of a command line option filters all out all tests (-k for example). Thanks Eric Siegerman (issue812) and Bruno Oliveira for the PR.
Summary bar now is colored yellow for warning situations such as: all tests either were skipped or xpass/xfailed, or no tests were run at all (related to issue500). Thanks Eric Siegerman.
New
testpaths
ini option: list of directories to search for tests when executing pytest from the root directory. This can be used to speed up test collection when a project has well specified directories for tests, being usually more practical than configuring norecursedirs for all directories that do not contain tests. Thanks to Adrian for idea (#694) and Bruno Oliveira for the PR.fix issue713: JUnit XML reports for doctest failures. Thanks Punyashloka Biswal.
fix issue970: internal pytest warnings now appear as “pytest-warnings” in the terminal instead of “warnings”, so it is clear for users that those warnings are from pytest and not from the builtin “warnings” module. Thanks Bruno Oliveira.
Include setup and teardown in junitxml test durations. Thanks Janne Vanhala.
fix issue735: assertion failures on debug versions of Python 3.4+
new option
--import-mode
to allow to change test module importing behaviour to append to sys.path instead of prepending. This better allows to run test modules against installed versions of a package even if the package under test has the same import root. In this example:testing/__init__.py testing/test_pkg_under_test.py pkg_under_test/
the tests will run against the installed version of pkg_under_test when
--import-mode=append
is used whereas by default they would always pick up the local version. Thanks Holger Krekel.pytester: add method
TmpTestdir.delete_loaded_modules()
, and call it frominline_run()
to allow temporary modules to be reloaded. Thanks Eduardo Schettino.internally refactor pluginmanager API and code so that there is a clear distinction between a pytest-agnostic rather simple pluginmanager and the PytestPluginManager which adds a lot of behaviour, among it handling of the local conftest files. In terms of documented methods this is a backward compatible change but it might still break 3rd party plugins which relied on details like especially the pluginmanager.add_shutdown() API. Thanks Holger Krekel.
pluginmanagement: introduce
pytest.hookimpl
andpytest.hookspec
decorators for setting impl/spec specific parameters. This substitutes the previous now deprecated use ofpytest.mark
which is meant to contain markers for test functions only.write/refine docs for “writing plugins” which now have their own page and are separate from the “using/installing plugins`` page.
fix issue732: properly unregister plugins from any hook calling sites allowing to have temporary plugins during test execution.
deprecate and warn about
__multicall__
argument in hook implementations. Use thehookwrapper
mechanism instead already introduced with pytest-2.7.speed up pytest’s own test suite considerably by using inprocess tests by default (testrun can be modified with –runpytest=subprocess to create subprocesses in many places instead). The main APIs to run pytest in a test is “runpytest()” or “runpytest_subprocess” and “runpytest_inprocess” if you need a particular way of running the test. In all cases you get back a RunResult but the inprocess one will also have a “reprec” attribute with the recorded events/reports.
fix monkeypatch.setattr(“x.y”, raising=False) to actually not raise if “y” is not a preexisting attribute. Thanks Florian Bruhin.
fix issue741: make running output from testdir.run copy/pasteable Thanks Bruno Oliveira.
add a new
--noconftest
argument which ignores allconftest.py
files.add
file
andline
attributes to JUnit-XML output.fix issue890: changed extension of all documentation files from
txt
torst
. Thanks to Abhijeet for the PR.fix issue714: add ability to apply indirect=True parameter on particular argnames. Thanks Elizaveta239.
fix issue890: changed extension of all documentation files from
txt
torst
. Thanks to Abhijeet for the PR.fix issue957: “# doctest: SKIP” option will now register doctests as SKIPPED rather than PASSED. Thanks Thomas Grainger for the report and Bruno Oliveira for the PR.
issue951: add new record_xml_property fixture, that supports logging additional information on xml output. Thanks David Diaz for the PR.
issue949: paths after normal options (for example
-s
,-v
, etc) are now properly used to discoverrootdir
andini
files. Thanks Peter Lauri for the report and Bruno Oliveira for the PR.
2.7.3 (2015-09-15)¶
Allow ‘dev’, ‘rc’, or other non-integer version strings in
importorskip
. Thanks to Eric Hunsberger for the PR.fix issue856: consider –color parameter in all outputs (for example –fixtures). Thanks Barney Gale for the report and Bruno Oliveira for the PR.
fix issue855: passing str objects as
plugins
argument to pytest.main is now interpreted as a module name to be imported and registered as a plugin, instead of silently having no effect. Thanks xmo-odoo for the report and Bruno Oliveira for the PR.fix issue744: fix for ast.Call changes in Python 3.5+. Thanks Guido van Rossum, Matthias Bussonnier, Stefan Zimmermann and Thomas Kluyver.
fix issue842: applying markers in classes no longer propagate this markers to superclasses which also have markers. Thanks xmo-odoo for the report and Bruno Oliveira for the PR.
preserve warning functions after call to pytest.deprecated_call. Thanks Pieter Mulder for PR.
fix issue854: autouse yield_fixtures defined as class members of unittest.TestCase subclasses now work as expected. Thanks xmo-odoo for the report and Bruno Oliveira for the PR.
fix issue833: –fixtures now shows all fixtures of collected test files, instead of just the fixtures declared on the first one. Thanks Florian Bruhin for reporting and Bruno Oliveira for the PR.
fix issue863: skipped tests now report the correct reason when a skip/xfail condition is met when using multiple markers. Thanks Raphael Pierzina for reporting and Bruno Oliveira for the PR.
optimized tmpdir fixture initialization, which should make test sessions faster (specially when using pytest-xdist). The only visible effect is that now pytest uses a subdirectory in the $TEMP directory for all directories created by this fixture (defaults to $TEMP/pytest-$USER). Thanks Bruno Oliveira for the PR.
2.7.2 (2015-06-23)¶
fix issue767: pytest.raises value attribute does not contain the exception instance on Python 2.6. Thanks Eric Siegerman for providing the test case and Bruno Oliveira for PR.
Automatically create directory for junitxml and results log. Thanks Aron Curzon.
fix issue713: JUnit XML reports for doctest failures. Thanks Punyashloka Biswal.
fix issue735: assertion failures on debug versions of Python 3.4+ Thanks Benjamin Peterson.
fix issue114: skipif marker reports to internal skipping plugin; Thanks Floris Bruynooghe for reporting and Bruno Oliveira for the PR.
fix issue748: unittest.SkipTest reports to internal pytest unittest plugin. Thanks Thomas De Schampheleire for reporting and Bruno Oliveira for the PR.
fix issue718: failed to create representation of sets containing unsortable elements in python 2. Thanks Edison Gustavo Muenz.
fix issue756, fix issue752 (and similar issues): depend on py-1.4.29 which has a refined algorithm for traceback generation.
2.7.1 (2015-05-19)¶
fix issue731: do not get confused by the braces which may be present and unbalanced in an object’s repr while collapsing False explanations. Thanks Carl Meyer for the report and test case.
fix issue553: properly handling inspect.getsourcelines failures in FixtureLookupError which would lead to an internal error, obfuscating the original problem. Thanks talljosh for initial diagnose/patch and Bruno Oliveira for final patch.
fix issue660: properly report scope-mismatch-access errors independently from ordering of fixture arguments. Also avoid the pytest internal traceback which does not provide information to the user. Thanks Holger Krekel.
streamlined and documented release process. Also all versions (in setup.py and documentation generation) are now read from _pytest/__init__.py. Thanks Holger Krekel.
fixed docs to remove the notion that yield-fixtures are experimental. They are here to stay :) Thanks Bruno Oliveira.
Support building wheels by using environment markers for the requirements. Thanks Ionel Maries Cristian.
fixed regression to 2.6.4 which surfaced e.g. in lost stdout capture printing when tests raised SystemExit. Thanks Holger Krekel.
reintroduced _pytest fixture of the pytester plugin which is used at least by pytest-xdist.
2.7.0 (2015-03-26)¶
fix issue435: make reload() work when assert rewriting is active. Thanks Daniel Hahler.
fix issue616: conftest.py files and their contained fixtures are now properly considered for visibility, independently from the exact current working directory and test arguments that are used. Many thanks to Eric Siegerman and his PR235 which contains systematic tests for conftest visibility and now passes. This change also introduces the concept of a
rootdir
which is printed as a new pytest header and documented in the pytest customize web page.change reporting of “diverted” tests, i.e. tests that are collected in one file but actually come from another (e.g. when tests in a test class come from a base class in a different file). We now show the nodeid and indicate via a postfix the other file.
add ability to set command line options by environment variable PYTEST_ADDOPTS.
added documentation on the new pytest-dev teams on bitbucket and github. See https://pytest.org/en/stable/contributing.html . Thanks to Anatoly for pushing and initial work on this.
fix issue650: new option
--doctest-ignore-import-errors
which will turn import errors in doctests into skips. Thanks Charles Cloud for the complete PR.fix issue655: work around different ways that cause python2/3 to leak sys.exc_info into fixtures/tests causing failures in 3rd party code
fix issue615: assertion rewriting did not correctly escape % signs when formatting boolean operations, which tripped over mixing booleans with modulo operators. Thanks to Tom Viner for the report, triaging and fix.
implement issue351: add ability to specify parametrize ids as a callable to generate custom test ids. Thanks Brianna Laugher for the idea and implementation.
introduce and document new hookwrapper mechanism useful for plugins which want to wrap the execution of certain hooks for their purposes. This supersedes the undocumented
__multicall__
protocol which pytest itself and some external plugins use. Note that pytest-2.8 is scheduled to drop supporting the old__multicall__
and only support the hookwrapper protocol.majorly speed up invocation of plugin hooks
use hookwrapper mechanism in builtin pytest plugins.
add a doctest ini option for doctest flags, thanks Holger Peters.
add note to docs that if you want to mark a parameter and the parameter is a callable, you also need to pass in a reason to disambiguate it from the “decorator” case. Thanks Tom Viner.
“python_classes” and “python_functions” options now support glob-patterns for test discovery, as discussed in issue600. Thanks Ldiary Translations.
allow to override parametrized fixtures with non-parametrized ones and vice versa (bubenkoff).
fix issue463: raise specific error for ‘parameterize’ misspelling (pfctdayelise).
On failure, the
sys.last_value
,sys.last_type
andsys.last_traceback
are set, so that a user can inspect the error via postmortem debugging (almarklein).
2.6.4 (2014-10-24)¶
Improve assertion failure reporting on iterables, by using ndiff and pprint.
removed outdated japanese docs from source tree.
docs for “pytest_addhooks” hook. Thanks Bruno Oliveira.
updated plugin index docs. Thanks Bruno Oliveira.
fix issue557: with “-k” we only allow the old style “-” for negation at the beginning of strings and even that is deprecated. Use “not” instead. This should allow to pick parametrized tests where “-” appeared in the parameter.
fix issue604: Escape % character in the assertion message.
fix issue620: add explanation in the –genscript target about what the binary blob means. Thanks Dinu Gherman.
fix issue614: fixed pastebin support.
fix issue620: add explanation in the –genscript target about what the binary blob means. Thanks Dinu Gherman.
fix issue614: fixed pastebin support.
2.6.3 (2014-09-24)¶
fix issue575: xunit-xml was reporting collection errors as failures instead of errors, thanks Oleg Sinyavskiy.
fix issue582: fix setuptools example, thanks Laszlo Papp and Ronny Pfannschmidt.
Fix infinite recursion bug when pickling capture.EncodedFile, thanks Uwe Schmitt.
fix issue589: fix bad interaction with numpy and others when showing exceptions. Check for precise “maximum recursion depth exceed” exception instead of presuming any RuntimeError is that one (implemented in py dep). Thanks Charles Cloud for analysing the issue.
fix conftest related fixture visibility issue: when running with a CWD outside of a test package pytest would get fixture discovery wrong. Thanks to Wolfgang Schnerring for figuring out a reproducible example.
Introduce pytest_enter_pdb hook (needed e.g. by pytest_timeout to cancel the timeout when interactively entering pdb). Thanks Wolfgang Schnerring.
check xfail/skip also with non-python function test items. Thanks Floris Bruynooghe.
2.6.2 (2014-09-05)¶
Added function pytest.freeze_includes(), which makes it easy to embed pytest into executables using tools like cx_freeze. See docs for examples and rationale. Thanks Bruno Oliveira.
Improve assertion rewriting cache invalidation precision.
fixed issue561: adapt autouse fixture example for python3.
fixed issue453: assertion rewriting issue with __repr__ containing “n{”, “n}” and “n~”.
fix issue560: correctly display code if an “else:” or “finally:” is followed by statements on the same line.
Fix example in monkeypatch documentation, thanks t-8ch.
fix issue572: correct tmpdir doc example for python3.
Do not mark as universal wheel because Python 2.6 is different from other builds due to the extra argparse dependency. Fixes issue566. Thanks sontek.
Implement issue549: user-provided assertion messages now no longer replace the py.test introspection message but are shown in addition to them.
2.6.1 (2014-08-07)¶
No longer show line numbers in the –verbose output, the output is now purely the nodeid. The line number is still shown in failure reports. Thanks Floris Bruynooghe.
fix issue437 where assertion rewriting could cause pytest-xdist worker nodes to collect different tests. Thanks Bruno Oliveira.
fix issue555: add “errors” attribute to capture-streams to satisfy some distutils and possibly other code accessing sys.stdout.errors.
fix issue547 capsys/capfd also work when output capturing (“-s”) is disabled.
address issue170: allow pytest.mark.xfail(…) to specify expected exceptions via an optional “raises=EXC” argument where EXC can be a single exception or a tuple of exception classes. Thanks David Mohr for the complete PR.
fix integration of pytest with unittest.mock.patch decorator when it uses the “new” argument. Thanks Nicolas Delaby for test and PR.
fix issue with detecting conftest files if the arguments contain “::” node id specifications (copy pasted from “-v” output)
fix issue544 by only removing “@NUM” at the end of “::” separated parts and if the part has a “.py” extension
don’t use py.std import helper, rather import things directly. Thanks Bruno Oliveira.
2.6¶
Cache exceptions from fixtures according to their scope (issue 467).
fix issue537: Avoid importing old assertion reinterpretation code by default.
fix issue364: shorten and enhance tracebacks representation by default. The new “–tb=auto” option (default) will only display long tracebacks for the first and last entry. You can get the old behaviour of printing all entries as long entries with “–tb=long”. Also short entries by default are now printed very similarly to “–tb=native” ones.
fix issue514: teach assertion reinterpretation about private class attributes
change -v output to include full node IDs of tests. Users can copy a node ID from a test run, including line number, and use it as a positional argument in order to run only a single test.
fix issue 475: fail early and comprehensible if calling pytest.raises with wrong exception type.
fix issue516: tell in getting-started about current dependencies.
cleanup setup.py a bit and specify supported versions. Thanks Jurko Gospodnetic for the PR.
change XPASS colour to yellow rather than red when tests are run with -v.
fix issue473: work around mock putting an unbound method into a class dict when double-patching.
fix issue498: if a fixture finalizer fails, make sure that the fixture is still invalidated.
fix issue453: the result of the pytest_assertrepr_compare hook now gets it’s newlines escaped so that format_exception does not blow up.
internal new warning system: pytest will now produce warnings when it detects oddities in your test collection or execution. Warnings are ultimately sent to a new pytest_logwarning hook which is currently only implemented by the terminal plugin which displays warnings in the summary line and shows more details when -rw (report on warnings) is specified.
change skips into warnings for test classes with an __init__ and callables in test modules which look like a test but are not functions.
fix issue436: improved finding of initial conftest files from command line arguments by using the result of parse_known_args rather than the previous flaky heuristics. Thanks Marc Abramowitz for tests and initial fixing approaches in this area.
fix issue #479: properly handle nose/unittest(2) SkipTest exceptions during collection/loading of test modules. Thanks to Marc Schlaich for the complete PR.
fix issue490: include pytest_load_initial_conftests in documentation and improve docstring.
fix issue472: clarify that
pytest.config.getvalue()
cannot work if it’s triggered ahead of command line parsing.merge PR123: improved integration with mock.patch decorator on tests.
fix issue412: messing with stdout/stderr FD-level streams is now captured without crashes.
fix issue483: trial/py33 works now properly. Thanks Daniel Grana for PR.
improve example for pytest integration with “python setup.py test” which now has a generic “-a” or “–pytest-args” option where you can pass additional options as a quoted string. Thanks Trevor Bekolay.
simplified internal capturing mechanism and made it more robust against tests or setups changing FD1/FD2, also better integrated now with pytest.pdb() in single tests.
improvements to pytest’s own test-suite leakage detection, courtesy of PRs from Marc Abramowitz
fix issue492: avoid leak in test_writeorg. Thanks Marc Abramowitz.
fix issue493: don’t run tests in doc directory with
python setup.py test
(use tox -e doctesting for that)fix issue486: better reporting and handling of early conftest loading failures
some cleanup and simplification of internal conftest handling.
work a bit harder to break reference cycles when catching exceptions. Thanks Jurko Gospodnetic.
fix issue443: fix skip examples to use proper comparison. Thanks Alex Groenholm.
support nose-style
__test__
attribute on modules, classes and functions, including unittest-style Classes. If set to False, the test will not be collected.fix issue512: show “<notset>” for arguments which might not be set in monkeypatch plugin. Improves output in documentation.
2.5.2 (2014-01-29)¶
fix issue409 – better interoperate with cx_freeze by not trying to import from collections.abc which causes problems for py27/cx_freeze. Thanks Wolfgang L. for reporting and tracking it down.
fixed docs and code to use “pytest” instead of “py.test” almost everywhere. Thanks Jurko Gospodnetic for the complete PR.
fix issue425: mention at end of “py.test -h” that –markers and –fixtures work according to specified test path (or current dir)
fix issue413: exceptions with unicode attributes are now printed correctly also on python2 and with pytest-xdist runs. (the fix requires py-1.4.20)
copy, cleanup and integrate py.io capture from pylib 1.4.20.dev2 (rev 13d9af95547e)
address issue416: clarify docs as to conftest.py loading semantics
fix issue429: comparing byte strings with non-ascii chars in assert expressions now work better. Thanks Floris Bruynooghe.
make capfd/capsys.capture private, its unused and shouldn’t be exposed
2.5.1 (2013-12-17)¶
merge new documentation styling PR from Tobias Bieniek.
fix issue403: allow parametrize of multiple same-name functions within a collection node. Thanks Andreas Kloeckner and Alex Gaynor for reporting and analysis.
Allow parameterized fixtures to specify the ID of the parameters by adding an ids argument to pytest.fixture() and pytest.yield_fixture(). Thanks Floris Bruynooghe.
fix issue404 by always using the binary xml escape in the junitxml plugin. Thanks Ronny Pfannschmidt.
fix issue407: fix addoption docstring to point to argparse instead of optparse. Thanks Daniel D. Wright.
2.5.0 (2013-12-12)¶
dropped python2.5 from automated release testing of pytest itself which means it’s probably going to break soon (but still works with this release we believe).
simplified and fixed implementation for calling finalizers when parametrized fixtures or function arguments are involved. finalization is now performed lazily at setup time instead of in the “teardown phase”. While this might sound odd at first, it helps to ensure that we are correctly handling setup/teardown even in complex code. User-level code should not be affected unless it’s implementing the pytest_runtest_teardown hook and expecting certain fixture instances are torn down within (very unlikely and would have been unreliable anyway).
PR90: add –color=yes|no|auto option to force terminal coloring mode (“auto” is default). Thanks Marc Abramowitz.
fix issue319 - correctly show unicode in assertion errors. Many thanks to Floris Bruynooghe for the complete PR. Also means we depend on py>=1.4.19 now.
fix issue396 - correctly sort and finalize class-scoped parametrized tests independently from number of methods on the class.
refix issue323 in a better way – parametrization should now never cause Runtime Recursion errors because the underlying algorithm for re-ordering tests per-scope/per-fixture is not recursive anymore (it was tail-call recursive before which could lead to problems for more than >966 non-function scoped parameters).
fix issue290 - there is preliminary support now for parametrizing with repeated same values (sometimes useful to test if calling a second time works as with the first time).
close issue240 - document precisely how pytest module importing works, discuss the two common test directory layouts, and how it interacts with PEP 420-namespace packages.
fix issue246 fix finalizer order to be LIFO on independent fixtures depending on a parametrized higher-than-function scoped fixture. (was quite some effort so please bear with the complexity of this sentence :) Thanks Ralph Schmitt for the precise failure example.
fix issue244 by implementing special index for parameters to only use indices for parametrized test ids
fix issue287 by running all finalizers but saving the exception from the first failing finalizer and re-raising it so teardown will still have failed. We reraise the first failing exception because it might be the cause for other finalizers to fail.
fix ordering when mock.patch or other standard decorator-wrappings are used with test methods. This fixes issue346 and should help with random “xdist” collection failures. Thanks to Ronny Pfannschmidt and Donald Stufft for helping to isolate it.
fix issue357 - special case “-k” expressions to allow for filtering with simple strings that are not valid python expressions. Examples: “-k 1.3” matches all tests parametrized with 1.3. “-k None” filters all tests that have “None” in their name and conversely “-k ‘not None’”. Previously these examples would raise syntax errors.
fix issue384 by removing the trial support code since the unittest compat enhancements allow trial to handle it on its own
don’t hide an ImportError when importing a plugin produces one. fixes issue375.
fix issue275 - allow usefixtures and autouse fixtures for running doctest text files.
fix issue380 by making –resultlog only rely on longrepr instead of the “reprcrash” attribute which only exists sometimes.
address issue122: allow @pytest.fixture(params=iterator) by exploding into a list early on.
fix pexpect-3.0 compatibility for pytest’s own tests. (fixes issue386)
allow nested parametrize-value markers, thanks James Lan for the PR.
fix unicode handling with new monkeypatch.setattr(import_path, value) API. Thanks Rob Dennis. Fixes issue371.
fix unicode handling with junitxml, fixes issue368.
In assertion rewriting mode on Python 2, fix the detection of coding cookies. See issue #330.
make “–runxfail” turn imperative pytest.xfail calls into no ops (it already did neutralize pytest.mark.xfail markers)
refine pytest / pkg_resources interactions: The AssertionRewritingHook PEP 302 compliant loader now registers itself with setuptools/pkg_resources properly so that the pkg_resources.resource_stream method works properly. Fixes issue366. Thanks for the investigations and full PR to Jason R. Coombs.
pytestconfig fixture is now session-scoped as it is the same object during the whole test run. Fixes issue370.
avoid one surprising case of marker malfunction/confusion:
@pytest.mark.some(lambda arg: ...) def test_function():
would not work correctly because pytest assumes @pytest.mark.some gets a function to be decorated already. We now at least detect if this arg is a lambda and thus the example will work. Thanks Alex Gaynor for bringing it up.
xfail a test on pypy that checks wrong encoding/ascii (pypy does not error out). fixes issue385.
internally make varnames() deal with classes’s __init__, although it’s not needed by pytest itself atm. Also fix caching. Fixes issue376.
fix issue221 - handle importing of namespace-package with no __init__.py properly.
refactor internal FixtureRequest handling to avoid monkeypatching. One of the positive user-facing effects is that the “request” object can now be used in closures.
fixed version comparison in pytest.importskip(modname, minverstring)
fix issue377 by clarifying in the nose-compat docs that pytest does not duplicate the unittest-API into the “plain” namespace.
fix verbose reporting for @mock’d test functions
2.4.2 (2013-10-04)¶
on Windows require colorama and a newer py lib so that py.io.TerminalWriter() now uses colorama instead of its own ctypes hacks. (fixes issue365) thanks Paul Moore for bringing it up.
fix “-k” matching of tests where “repr” and “attr” and other names would cause wrong matches because of an internal implementation quirk (don’t ask) which is now properly implemented. fixes issue345.
avoid tmpdir fixture to create too long filenames especially when parametrization is used (issue354)
fix pytest-pep8 and pytest-flakes / pytest interactions (collection names in mark plugin was assuming an item always has a function which is not true for those plugins etc.) Thanks Andi Zeidler.
introduce node.get_marker/node.add_marker API for plugins like pytest-pep8 and pytest-flakes to avoid the messy details of the node.keywords pseudo-dicts. Adapted docs.
remove attempt to “dup” stdout at startup as it’s icky. the normal capturing should catch enough possibilities of tests messing up standard FDs.
add pluginmanager.do_configure(config) as a link to config.do_configure() for plugin-compatibility
2.4.1 (2013-10-02)¶
When using parser.addoption() unicode arguments to the “type” keyword should also be converted to the respective types. thanks Floris Bruynooghe, @dnozay. (fixes issue360 and issue362)
fix dotted filename completion when using argcomplete thanks Anthon van der Neuth. (fixes issue361)
fix regression when a 1-tuple (“arg”,) is used for specifying parametrization (the values of the parametrization were passed nested in a tuple). Thanks Donald Stufft.
merge doc typo fixes, thanks Andy Dirnberger
2.4¶
known incompatibilities:
if calling –genscript from python2.7 or above, you only get a standalone script which works on python2.7 or above. Use Python2.6 to also get a python2.5 compatible version.
all xunit-style teardown methods (nose-style, pytest-style, unittest-style) will not be called if the corresponding setup method failed, see issue322 below.
the pytest_plugin_unregister hook wasn’t ever properly called and there is no known implementation of the hook - so it got removed.
pytest.fixture-decorated functions cannot be generators (i.e. use yield) anymore. This change might be reversed in 2.4.1 if it causes unforeseen real-life issues. However, you can always write and return an inner function/generator and change the fixture consumer to iterate over the returned generator. This change was done in lieu of the new
pytest.yield_fixture
decorator, see below.
new features:
experimentally introduce a new
pytest.yield_fixture
decorator which accepts exactly the same parameters as pytest.fixture but mandates ayield
statement instead of areturn statement
from fixture functions. This allows direct integration with “with-style” context managers in fixture functions and generally avoids registering of finalization callbacks in favour of treating the “after-yield” as teardown code. Thanks Andreas Pelme, Vladimir Keleshev, Floris Bruynooghe, Ronny Pfannschmidt and many others for discussions.allow boolean expression directly with skipif/xfail if a “reason” is also specified. Rework skipping documentation to recommend “condition as booleans” because it prevents surprises when importing markers between modules. Specifying conditions as strings will remain fully supported.
reporting: color the last line red or green depending if failures/errors occurred or everything passed. thanks Christian Theunert.
make “import pdb ; pdb.set_trace()” work natively wrt capturing (no “-s” needed anymore), making
pytest.set_trace()
a mere shortcut.fix issue181: –pdb now also works on collect errors (and on internal errors) . This was implemented by a slight internal refactoring and the introduction of a new hook
pytest_exception_interact
hook (see next item).fix issue341: introduce new experimental hook for IDEs/terminals to intercept debugging:
pytest_exception_interact(node, call, report)
.new monkeypatch.setattr() variant to provide a shorter invocation for patching out classes/functions from modules:
monkeypatch.setattr(“requests.get”, myfunc)
will replace the “get” function of the “requests” module with
myfunc
.fix issue322: tearDownClass is not run if setUpClass failed. Thanks Mathieu Agopian for the initial fix. Also make all of pytest/nose finalizer mimic the same generic behaviour: if a setupX exists and fails, don’t run teardownX. This internally introduces a new method “node.addfinalizer()” helper which can only be called during the setup phase of a node.
simplify pytest.mark.parametrize() signature: allow to pass a CSV-separated string to specify argnames. For example:
pytest.mark.parametrize("input,expected", [(1,2), (2,3)])
works as well as the previous:pytest.mark.parametrize(("input", "expected"), ...)
.add support for setUpModule/tearDownModule detection, thanks Brian Okken.
integrate tab-completion on options through use of “argcomplete”. Thanks Anthon van der Neut for the PR.
change option names to be hyphen-separated long options but keep the old spelling backward compatible. py.test -h will only show the hyphenated version, for example “–collect-only” but “–collectonly” will remain valid as well (for backward-compat reasons). Many thanks to Anthon van der Neut for the implementation and to Hynek Schlawack for pushing us.
fix issue 308 - allow to mark/xfail/skip individual parameter sets when parametrizing. Thanks Brianna Laugher.
call new experimental pytest_load_initial_conftests hook to allow 3rd party plugins to do something before a conftest is loaded.
Bug fixes:
fix issue358 - capturing options are now parsed more properly by using a new parser.parse_known_args method.
pytest now uses argparse instead of optparse (thanks Anthon) which means that “argparse” is added as a dependency if installing into python2.6 environments or below.
fix issue333: fix a case of bad unittest/pytest hook interaction.
PR27: correctly handle nose.SkipTest during collection. Thanks Antonio Cuni, Ronny Pfannschmidt.
fix issue355: junitxml puts name=”pytest” attribute to testsuite tag.
fix issue336: autouse fixture in plugins should work again.
fix issue279: improve object comparisons on assertion failure for standard datatypes and recognise collections.abc. Thanks to Brianna Laugher and Mathieu Agopian.
fix issue317: assertion rewriter support for the is_package method
fix issue335: document py.code.ExceptionInfo() object returned from pytest.raises(), thanks Mathieu Agopian.
remove implicit distribute_setup support from setup.py.
fix issue305: ignore any problems when writing pyc files.
SO-17664702: call fixture finalizers even if the fixture function partially failed (finalizers would not always be called before)
fix issue320 - fix class scope for fixtures when mixed with module-level functions. Thanks Anatoly Bubenkoff.
you can specify “-q” or “-qq” to get different levels of “quieter” reporting (thanks Katarzyna Jachim)
fix issue300 - Fix order of conftest loading when starting py.test in a subdirectory.
fix issue323 - sorting of many module-scoped arg parametrizations
make sessionfinish hooks execute with the same cwd-context as at session start (helps fix plugin behaviour which write output files with relative path such as pytest-cov)
fix issue316 - properly reference collection hooks in docs
fix issue 306 - cleanup of -k/-m options to only match markers/test names/keywords respectively. Thanks Wouter van Ackooy.
improved doctest counting for doctests in python modules – files without any doctest items will not show up anymore and doctest examples are counted as separate test items. thanks Danilo Bellini.
fix issue245 by depending on the released py-1.4.14 which fixes py.io.dupfile to work with files with no mode. Thanks Jason R. Coombs.
fix junitxml generation when test output contains control characters, addressing issue267, thanks Jaap Broekhuizen
fix issue338: honor –tb style for setup/teardown errors as well. Thanks Maho.
fix issue307 - use yaml.safe_load in example, thanks Mark Eichin.
better parametrize error messages, thanks Brianna Laugher
pytest_terminal_summary(terminalreporter) hooks can now use “.section(title)” and “.line(msg)” methods to print extra information at the end of a test run.
2.3.5 (2013-04-30)¶
fix issue169: respect –tb=style with setup/teardown errors as well.
never consider a fixture function for test function collection
allow re-running of test items / helps to fix pytest-reruntests plugin and also help to keep less fixture/resource references alive
put captured stdout/stderr into junitxml output even for passing tests (thanks Adam Goucher)
Issue 265 - integrate nose setup/teardown with setupstate so it doesn’t try to teardown if it did not setup
issue 271 - don’t write junitxml on worker nodes
Issue 274 - don’t try to show full doctest example when doctest does not know the example location
issue 280 - disable assertion rewriting on buggy CPython 2.6.0
inject “getfixture()” helper to retrieve fixtures from doctests, thanks Andreas Zeidler
issue 259 - when assertion rewriting, be consistent with the default source encoding of ASCII on Python 2
issue 251 - report a skip instead of ignoring classes with init
issue250 unicode/str mixes in parametrization names and values now works
issue257, assertion-triggered compilation of source ending in a comment line doesn’t blow up in python2.5 (fixed through py>=1.4.13.dev6)
fix –genscript option to generate standalone scripts that also work with python3.3 (importer ordering)
issue171 - in assertion rewriting, show the repr of some global variables
fix option help for “-k”
move long description of distribution into README.rst
improve docstring for metafunc.parametrize()
fix bug where using capsys with pytest.set_trace() in a test function would break when looking at capsys.readouterr()
allow to specify prefixes starting with “_” when customizing python_functions test discovery. (thanks Graham Horler)
improve PYTEST_DEBUG tracing output by putting extra data on a new lines with additional indent
ensure OutcomeExceptions like skip/fail have initialized exception attributes
issue 260 - don’t use nose special setup on plain unittest cases
fix issue134 - print the collect errors that prevent running specified test items
fix issue266 - accept unicode in MarkEvaluator expressions
2.3.4 (2012-11-20)¶
yielded test functions will now have autouse-fixtures active but cannot accept fixtures as funcargs - it’s anyway recommended to rather use the post-2.0 parametrize features instead of yield, see: http://pytest.org/en/stable/example/how-to/parametrize.html
fix autouse-issue where autouse-fixtures would not be discovered if defined in an a/conftest.py file and tests in a/tests/test_some.py
fix issue226 - LIFO ordering for fixture teardowns
fix issue224 - invocations with >256 char arguments now work
fix issue91 - add/discuss package/directory level setups in example
allow to dynamically define markers via item.keywords[…]=assignment integrating with “-m” option
make “-k” accept an expressions the same as with “-m” so that one can write: -k “name1 or name2” etc. This is a slight incompatibility if you used special syntax like “TestClass.test_method” which you now need to write as -k “TestClass and test_method” to match a certain method in a certain test class.
2.3.3 (2012-11-06)¶
fix issue214 - parse modules that contain special objects like e. g. flask’s request object which blows up on getattr access if no request is active. thanks Thomas Waldmann.
fix issue213 - allow to parametrize with values like numpy arrays that do not support an __eq__ operator
fix issue215 - split test_python.org into multiple files
fix issue148 - @unittest.skip on classes is now recognized and avoids calling setUpClass/tearDownClass, thanks Pavel Repin
fix issue209 - reintroduce python2.4 support by depending on newer pylib which re-introduced statement-finding for pre-AST interpreters
nose support: only call setup if it’s a callable, thanks Andrew Taumoefolau
fix issue219 - add py2.4-3.3 classifiers to TROVE list
in tracebacks ,* arg values are now shown next to normal arguments (thanks Manuel Jacob)
fix issue217 - support mock.patch with pytest’s fixtures - note that you need either mock-1.0.1 or the python3.3 builtin unittest.mock.
fix issue127 - improve documentation for pytest_addoption() and add a
config.getoption(name)
helper function for consistency.
2.3.2 (2012-10-25)¶
fix issue208 and fix issue29 use new py version to avoid long pauses when printing tracebacks in long modules
fix issue205 - conftests in subdirs customizing pytest_pycollect_makemodule and pytest_pycollect_makeitem now work properly
fix teardown-ordering for parametrized setups
fix issue127 - better documentation for pytest_addoption and related objects.
fix unittest behaviour: TestCase.runtest only called if there are test methods defined
improve trial support: don’t collect its empty unittest.TestCase.runTest() method
“python setup.py test” now works with pytest itself
fix/improve internal/packaging related bits:
exception message check of test_nose.py now passes on python33 as well
issue206 - fix test_assertrewrite.py to work when a global PYTHONDONTWRITEBYTECODE=1 is present
add tox.ini to pytest distribution so that ignore-dirs and others config bits are properly distributed for maintainers who run pytest-own tests
2.3.1 (2012-10-20)¶
fix issue202 - fix regression: using “self” from fixture functions now works as expected (it’s the same “self” instance that a test method which uses the fixture sees)
skip pexpect using tests (test_pdb.py mostly) on freebsd* systems due to pexpect not supporting it properly (hanging)
link to web pages from –markers output which provides help for pytest.mark.* usage.
2.3.0 (2012-10-19)¶
fix issue202 - better automatic names for parametrized test functions
fix issue139 - introduce @pytest.fixture which allows direct scoping and parametrization of funcarg factories.
fix issue198 - conftest fixtures were not found on windows32 in some circumstances with nested directory structures due to path manipulation issues
fix issue193 skip test functions with were parametrized with empty parameter sets
fix python3.3 compat, mostly reporting bits that previously depended on dict ordering
introduce re-ordering of tests by resource and parametrization setup which takes precedence to the usual file-ordering
fix issue185 monkeypatching time.time does not cause pytest to fail
fix issue172 duplicate call of pytest.fixture decoratored setup_module functions
fix junitxml=path construction so that if tests change the current working directory and the path is a relative path it is constructed correctly from the original current working dir.
fix “python setup.py test” example to cause a proper “errno” return
fix issue165 - fix broken doc links and mention stackoverflow for FAQ
catch unicode-issues when writing failure representations to terminal to prevent the whole session from crashing
fix xfail/skip confusion: a skip-mark or an imperative pytest.skip will now take precedence before xfail-markers because we can’t determine xfail/xpass status in case of a skip. see also: http://stackoverflow.com/questions/11105828/in-py-test-when-i-explicitly-skip-a-test-that-is-marked-as-xfail-how-can-i-get
always report installed 3rd party plugins in the header of a test run
fix issue160: a failing setup of an xfail-marked tests should be reported as xfail (not xpass)
fix issue128: show captured output when capsys/capfd are used
fix issue179: properly show the dependency chain of factories
pluginmanager.register(…) now raises ValueError if the plugin has been already registered or the name is taken
fix issue159: improve https://docs.pytest.org/en/6.0.1/faq.html especially with respect to the “magic” history, also mention pytest-django, trial and unittest integration.
make request.keywords and node.keywords writable. All descendant collection nodes will see keyword values. Keywords are dictionaries containing markers and other info.
fix issue 178: xml binary escapes are now wrapped in py.xml.raw
fix issue 176: correctly catch the builtin AssertionError even when we replaced AssertionError with a subclass on the python level
factory discovery no longer fails with magic global callables that provide no sane __code__ object (mock.call for example)
fix issue 182: testdir.inprocess_run now considers passed plugins
- fix issue 188: ensure sys.exc_info is clear on python2
before calling into a test
fix issue 191: add unittest TestCase runTest method support
fix issue 156: monkeypatch correctly handles class level descriptors
reporting refinements:
pytest_report_header now receives a “startdir” so that you can use startdir.bestrelpath(yourpath) to show nice relative path
allow plugins to implement both pytest_report_header and pytest_sessionstart (sessionstart is invoked first).
don’t show deselected reason line if there is none
py.test -vv will show all of assert comparisons instead of truncating
2.2.4 (2012-05-22)¶
fix error message for rewritten assertions involving the % operator
fix issue 126: correctly match all invalid xml characters for junitxml binary escape
fix issue with unittest: now @unittest.expectedFailure markers should be processed correctly (you can also use @pytest.mark markers)
document integration with the extended distribute/setuptools test commands
fix issue 140: properly get the real functions of bound classmethods for setup/teardown_class
fix issue #141: switch from the deceased paste.pocoo.org to bpaste.net
fix issue #143: call unconfigure/sessionfinish always when configure/sessionstart where called
fix issue #144: better mangle test ids to junitxml classnames
upgrade distribute_setup.py to 0.6.27
2.2.3 (2012-02-05)¶
fix uploaded package to only include necessary files
2.2.2 (2012-02-05)¶
fix issue101: wrong args to unittest.TestCase test function now produce better output
fix issue102: report more useful errors and hints for when a test directory was renamed and some pyc/__pycache__ remain
fix issue106: allow parametrize to be applied multiple times e.g. from module, class and at function level.
fix issue107: actually perform session scope finalization
don’t check in parametrize if indirect parameters are funcarg names
add chdir method to monkeypatch funcarg
fix crash resulting from calling monkeypatch undo a second time
fix issue115: make –collectonly robust against early failure (missing files/directories)
“-qq –collectonly” now shows only files and the number of tests in them
“-q –collectonly” now shows test ids
allow adding of attributes to test reports such that it also works with distributed testing (no upgrade of pytest-xdist needed)
2.2.1 (2011-12-16)¶
fix issue99 (in pytest and py) internallerrors with resultlog now produce better output - fixed by normalizing pytest_internalerror input arguments.
fix issue97 / traceback issues (in pytest and py) improve traceback output in conjunction with jinja2 and cython which hack tracebacks
fix issue93 (in pytest and pytest-xdist) avoid “delayed teardowns”: the final test in a test node will now run its teardown directly instead of waiting for the end of the session. Thanks Dave Hunt for the good reporting and feedback. The pytest_runtest_protocol as well as the pytest_runtest_teardown hooks now have “nextitem” available which will be None indicating the end of the test run.
fix collection crash due to unknown-source collected items, thanks to Ralf Schmitt (fixed by depending on a more recent pylib)
2.2.0 (2011-11-18)¶
fix issue90: introduce eager tearing down of test items so that teardown function are called earlier.
add an all-powerful metafunc.parametrize function which allows to parametrize test function arguments in multiple steps and therefore from independent plugins and places.
add a @pytest.mark.parametrize helper which allows to easily call a test function with different argument values
Add examples to the “parametrize” example page, including a quick port of Test scenarios and the new parametrize function and decorator.
introduce registration for “pytest.mark.*” helpers via ini-files or through plugin hooks. Also introduce a “–strict” option which will treat unregistered markers as errors allowing to avoid typos and maintain a well described set of markers for your test suite. See examples at http://pytest.org/en/stable/how-to/mark.html and its links.
issue50: introduce “-m marker” option to select tests based on markers (this is a stricter and more predictable version of ‘-k’ in that “-m” only matches complete markers and has more obvious rules for and/or semantics.
new feature to help optimizing the speed of your tests: –durations=N option for displaying N slowest test calls and setup/teardown methods.
fix issue87: –pastebin now works with python3
fix issue89: –pdb with unexpected exceptions in doctest work more sensibly
fix and cleanup pytest’s own test suite to not leak FDs
fix issue83: link to generated funcarg list
fix issue74: pyarg module names are now checked against imp.find_module false positives
fix compatibility with twisted/trial-11.1.0 use cases
simplify Node.listchain
simplify junitxml output code by relying on py.xml
add support for skip properties on unittest classes and functions
2.1.3 (2011-10-18)¶
fix issue79: assertion rewriting failed on some comparisons in boolops
correctly handle zero length arguments (a la pytest ‘’)
fix issue67 / junitxml now contains correct test durations, thanks ronny
fix issue75 / skipping test failure on jython
fix issue77 / Allow assertrepr_compare hook to apply to a subset of tests
2.1.2 (2011-09-24)¶
fix assertion rewriting on files with windows newlines on some Python versions
refine test discovery by package/module name (–pyargs), thanks Florian Mayer
fix issue69 / assertion rewriting fixed on some boolean operations
fix issue68 / packages now work with assertion rewriting
fix issue66: use different assertion rewriting caches when the -O option is passed
don’t try assertion rewriting on Jython, use reinterp
2.1.1¶
fix issue64 / pytest.set_trace now works within pytest_generate_tests hooks
fix issue60 / fix error conditions involving the creation of __pycache__
fix issue63 / assertion rewriting on inserts involving strings containing ‘%’
fix assertion rewriting on calls with a ** arg
don’t cache rewritten modules if bytecode generation is disabled
fix assertion rewriting in read-only directories
fix issue59: provide system-out/err tags for junitxml output
fix issue61: assertion rewriting on boolean operations with 3 or more operands
you can now build a man page with “cd doc ; make man”
2.1.0 (2011-07-09)¶
fix issue53 call nosestyle setup functions with correct ordering
fix issue58 and issue59: new assertion code fixes
merge Benjamin’s assertionrewrite branch: now assertions for test modules on python 2.6 and above are done by rewriting the AST and saving the pyc file before the test module is imported. see doc/assert.txt for more info.
fix issue43: improve doctests with better traceback reporting on unexpected exceptions
fix issue47: timing output in junitxml for test cases is now correct
fix issue48: typo in MarkInfo repr leading to exception
fix issue49: avoid confusing error when initialization partially fails
fix issue44: env/username expansion for junitxml file path
show releaselevel information in test runs for pypy
reworked doc pages for better navigation and PDF generation
report KeyboardInterrupt even if interrupted during session startup
fix issue 35 - provide PDF doc version and download link from index page
2.0.3 (2011-05-11)¶
fix issue38: nicer tracebacks on calls to hooks, particularly early configure/sessionstart ones
fix missing skip reason/meta information in junitxml files, reported via http://lists.idyll.org/pipermail/testing-in-python/2011-March/003928.html
fix issue34: avoid collection failure with “test” prefixed classes deriving from object.
don’t require zlib (and other libs) for genscript plugin without –genscript actually being used.
speed up skips (by not doing a full traceback representation internally)
fix issue37: avoid invalid characters in junitxml’s output
2.0.2 (2011-03-09)¶
tackle issue32 - speed up test runs of very quick test functions by reducing the relative overhead
fix issue30 - extended xfail/skipif handling and improved reporting. If you have a syntax error in your skip/xfail expressions you now get nice error reports.
Also you can now access module globals from xfail/skipif expressions so that this for example works now:
import pytest import mymodule @pytest.mark.skipif("mymodule.__version__[0] == "1") def test_function(): pass
This will not run the test function if the module’s version string does not start with a “1”. Note that specifying a string instead of a boolean expressions allows py.test to report meaningful information when summarizing a test run as to what conditions lead to skipping (or xfail-ing) tests.
fix issue28 - setup_method and pytest_generate_tests work together The setup_method fixture method now gets called also for test function invocations generated from the pytest_generate_tests hook.
fix issue27 - collectonly and keyword-selection (-k) now work together Also, if you do “py.test –collectonly -q” you now get a flat list of test ids that you can use to paste to the py.test commandline in order to execute a particular test.
fix issue25 avoid reported problems with –pdb and python3.2/encodings output
fix issue23 - tmpdir argument now works on Python3.2 and WindowsXP Starting with Python3.2 os.symlink may be supported. By requiring a newer py lib version the py.path.local() implementation acknowledges this.
fixed typos in the docs (thanks Victor Garcia, Brianna Laugher) and particular thanks to Laura Creighton who also reviewed parts of the documentation.
fix slightly wrong output of verbose progress reporting for classes (thanks Amaury)
more precise (avoiding of) deprecation warnings for node.Class|Function accesses
avoid std unittest assertion helper code in tracebacks (thanks Ronny)
2.0.1 (2011-02-07)¶
refine and unify initial capturing so that it works nicely even if the logging module is used on an early-loaded conftest.py file or plugin.
allow to omit “()” in test ids to allow for uniform test ids as produced by Alfredo’s nice pytest.vim plugin.
fix issue12 - show plugin versions with “–version” and “–traceconfig” and also document how to add extra information to reporting test header
fix issue17 (import-* reporting issue on python3) by requiring py>1.4.0 (1.4.1 is going to include it)
fix issue10 (numpy arrays truth checking) by refining assertion interpretation in py lib
fix issue15: make nose compatibility tests compatible with python3 (now that nose-1.0 supports python3)
remove somewhat surprising “same-conftest” detection because it ignores conftest.py when they appear in several subdirs.
improve assertions (“not in”), thanks Floris Bruynooghe
improve behaviour/warnings when running on top of “python -OO” (assertions and docstrings are turned off, leading to potential false positives)
introduce a pytest_cmdline_processargs(args) hook to allow dynamic computation of command line arguments. This fixes a regression because py.test prior to 2.0 allowed to set command line options from conftest.py files which so far pytest-2.0 only allowed from ini-files now.
fix issue7: assert failures in doctest modules. unexpected failures in doctests will not generally show nicer, i.e. within the doctest failing context.
fix issue9: setup/teardown functions for an xfail-marked test will report as xfail if they fail but report as normally passing (not xpassing) if they succeed. This only is true for “direct” setup/teardown invocations because teardown_class/ teardown_module cannot closely relate to a single test.
fix issue14: no logging errors at process exit
refinements to “collecting” output on non-ttys
refine internal plugin registration and –traceconfig output
introduce a mechanism to prevent/unregister plugins from the command line, see http://pytest.org/en/stable/how-to/plugins.html#cmdunregister
activate resultlog plugin by default
fix regression wrt yielded tests which due to the collection-before-running semantics were not setup as with pytest 1.3.4. Note, however, that the recommended and much cleaner way to do test parameterization remains the “pytest_generate_tests” mechanism, see the docs.
2.0.0 (2010-11-25)¶
pytest-2.0 is now its own package and depends on pylib-2.0
new ability: python -m pytest / python -m pytest.main ability
new python invocation: pytest.main(args, plugins) to load some custom plugins early.
try harder to run unittest test suites in a more compatible manner by deferring setup/teardown semantics to the unittest package. also work harder to run twisted/trial and Django tests which should now basically work by default.
introduce a new way to set config options via ini-style files, by default setup.cfg and tox.ini files are searched. The old ways (certain environment variables, dynamic conftest.py reading is removed).
add a new “-q” option which decreases verbosity and prints a more nose/unittest-style “dot” output.
fix issue135 - marks now work with unittest test cases as well
fix issue126 - introduce py.test.set_trace() to trace execution via PDB during the running of tests even if capturing is ongoing.
fix issue123 - new “python -m py.test” invocation for py.test (requires Python 2.5 or above)
fix issue124 - make reporting more resilient against tests opening files on filedescriptor 1 (stdout).
fix issue109 - sibling conftest.py files will not be loaded. (and Directory collectors cannot be customized anymore from a Directory’s conftest.py - this needs to happen at least one level up).
introduce (customizable) assertion failure representations and enhance output on assertion failures for comparisons and other cases (Floris Bruynooghe)
nose-plugin: pass through type-signature failures in setup/teardown functions instead of not calling them (Ed Singleton)
remove py.test.collect.Directory (follows from a major refactoring and simplification of the collection process)
majorly reduce py.test core code, shift function/python testing to own plugin
fix issue88 (finding custom test nodes from command line arg)
refine ‘tmpdir’ creation, will now create basenames better associated with test names (thanks Ronny)
“xpass” (unexpected pass) tests don’t cause exitcode!=0
fix issue131 / issue60 - importing doctests in __init__ files used as namespace packages
fix issue93 stdout/stderr is captured while importing conftest.py
fix bug: unittest collected functions now also can have “pytestmark” applied at class/module level
add ability to use “class” level for cached_setup helper
fix strangeness: mark.* objects are now immutable, create new instances
1.3.4 (2010-09-14)¶
fix issue111: improve install documentation for windows
fix issue119: fix custom collectability of __init__.py as a module
fix issue116: –doctestmodules work with __init__.py files as well
fix issue115: unify internal exception passthrough/catching/GeneratorExit
fix issue118: new –tb=native for presenting cpython-standard exceptions
1.3.3 (2010-07-30)¶
fix issue113: assertion representation problem with triple-quoted strings (and possibly other cases)
make conftest loading detect that a conftest file with the same content was already loaded, avoids surprises in nested directory structures which can be produced e.g. by Hudson. It probably removes the need to use –confcutdir in most cases.
fix terminal coloring for win32 (thanks Michael Foord for reporting)
fix weirdness: make terminal width detection work on stdout instead of stdin (thanks Armin Ronacher for reporting)
remove trailing whitespace in all py/text distribution files
1.3.2 (2010-07-08)¶
New features
fix issue103: introduce py.test.raises as context manager, examples:
with py.test.raises(ZeroDivisionError): x = 0 1 / x with py.test.raises(RuntimeError) as excinfo: call_something() # you may do extra checks on excinfo.value|type|traceback here
(thanks Ronny Pfannschmidt)
Funcarg factories can now dynamically apply a marker to a test invocation. This is for example useful if a factory provides parameters to a test which are expected-to-fail:
def pytest_funcarg__arg(request): request.applymarker(py.test.mark.xfail(reason="flaky config")) ... def test_function(arg): ...
improved error reporting on collection and import errors. This makes use of a more general mechanism, namely that for custom test item/collect nodes
node.repr_failure(excinfo)
is now uniformly called so that you can override it to return a string error representation of your choice which is going to be reported as a (red) string.introduce ‘–junitprefix=STR’ option to prepend a prefix to all reports in the junitxml file.
Bug fixes
make tests and the
pytest_recwarn
plugin in particular fully compatible to Python2.7 (if you use therecwarn
funcarg warnings will be enabled so that you can properly check for their existence in a cross-python manner).refine –pdb: ignore xfailed tests, unify its TB-reporting and don’t display failures again at the end.
fix assertion interpretation with the ** operator (thanks Benjamin Peterson)
fix issue105 assignment on the same line as a failing assertion (thanks Benjamin Peterson)
fix issue104 proper escaping for test names in junitxml plugin (thanks anonymous)
fix issue57 -f|–looponfail to work with xpassing tests (thanks Ronny)
fix issue92 collectonly reporter and –pastebin (thanks Benjamin Peterson)
fix py.code.compile(source) to generate unique filenames
fix assertion re-interp problems on PyPy, by deferring code compilation to the (overridable) Frame.eval class. (thanks Amaury Forgeot)
fix py.path.local.pyimport() to work with directories
streamline py.path.local.mkdtemp implementation and usage
don’t print empty lines when showing junitxml-filename
add optional boolean ignore_errors parameter to py.path.local.remove
fix terminal writing on win32/python2.4
py.process.cmdexec() now tries harder to return properly encoded unicode objects on all python versions
install plain py.test/py.which scripts also for Jython, this helps to get canonical script paths in virtualenv situations
make path.bestrelpath(path) return “.”, note that when calling X.bestrelpath the assumption is that X is a directory.
make initial conftest discovery ignore “–” prefixed arguments
fix resultlog plugin when used in a multicpu/multihost xdist situation (thanks Jakub Gustak)
perform distributed testing related reporting in the xdist-plugin rather than having dist-related code in the generic py.test distribution
fix homedir detection on Windows
ship distribute_setup.py version 0.6.13
1.3.1 (2010-05-25)¶
New features
issue91: introduce new py.test.xfail(reason) helper to imperatively mark a test as expected to fail. Can be used from within setup and test functions. This is useful especially for parametrized tests when certain configurations are expected-to-fail. In this case the declarative approach with the @py.test.mark.xfail cannot be used as it would mark all configurations as xfail.
issue102: introduce new –maxfail=NUM option to stop test runs after NUM failures. This is a generalization of the ‘-x’ or ‘–exitfirst’ option which is now equivalent to ‘–maxfail=1’. Both ‘-x’ and ‘–maxfail’ will now also print a line near the end indicating the Interruption.
issue89: allow py.test.mark decorators to be used on classes (class decorators were introduced with python2.6) and also allow to have multiple markers applied at class/module level by specifying a list.
improve and refine letter reporting in the progress bar: . pass f failed test s skipped tests (reminder: use for dependency/platform mismatch only) x xfailed test (test that was expected to fail) X xpassed test (test that was expected to fail but passed)
You can use any combination of ‘fsxX’ with the ‘-r’ extended reporting option. The xfail/xpass results will show up as skipped tests in the junitxml output - which also fixes issue99.
make py.test.cmdline.main() return the exitstatus instead of raising SystemExit and also allow it to be called multiple times. This of course requires that your application and tests are properly teared down and don’t have global state.
Bug Fixes
improved traceback presentation: - improved and unified reporting for “–tb=short” option - Errors during test module imports are much shorter, (using –tb=short style) - raises shows shorter more relevant tracebacks - –fulltrace now more systematically makes traces longer / inhibits cutting
improve support for raises and other dynamically compiled code by manipulating python’s linecache.cache instead of the previous rather hacky way of creating custom code objects. This makes it seamlessly work on Jython and PyPy where it previously didn’t.
fix issue96: make capturing more resilient against Control-C interruptions (involved somewhat substantial refactoring to the underlying capturing functionality to avoid race conditions).
fix chaining of conditional skipif/xfail decorators - so it works now as expected to use multiple @py.test.mark.skipif(condition) decorators, including specific reporting which of the conditions lead to skipping.
fix issue95: late-import zlib so that it’s not required for general py.test startup.
fix issue94: make reporting more robust against bogus source code (and internally be more careful when presenting unexpected byte sequences)
1.3.0 (2010-05-05)¶
deprecate –report option in favour of a new shorter and easier to remember -r option: it takes a string argument consisting of any combination of ‘xfsX’ characters. They relate to the single chars you see during the dotted progress printing and will print an extra line per test at the end of the test run. This extra line indicates the exact position or test ID that you directly paste to the py.test cmdline in order to re-run a particular test.
allow external plugins to register new hooks via the new pytest_addhooks(pluginmanager) hook. The new release of the pytest-xdist plugin for distributed and looponfailing testing requires this feature.
add a new pytest_ignore_collect(path, config) hook to allow projects and plugins to define exclusion behaviour for their directory structure - for example you may define in a conftest.py this method:
def pytest_ignore_collect(path): return path.check(link=1)
to prevent even a collection try of any tests in symlinked dirs.
new pytest_pycollect_makemodule(path, parent) hook for allowing customization of the Module collection object for a matching test module.
extend and refine xfail mechanism:
@py.test.mark.xfail(run=False)
do not run the decorated test@py.test.mark.xfail(reason="...")
prints the reason string in xfail summaries specifying--runxfail
on command line virtually ignores xfail markersexpose (previously internal) commonly useful methods: py.io.get_terminal_with() -> return terminal width py.io.ansi_print(…) -> print colored/bold text on linux/win32 py.io.saferepr(obj) -> return limited representation string
expose test outcome related exceptions as py.test.skip.Exception, py.test.raises.Exception etc., useful mostly for plugins doing special outcome interpretation/tweaking
(issue85) fix junitxml plugin to handle tests with non-ascii output
fix/refine python3 compatibility (thanks Benjamin Peterson)
fixes for making the jython/win32 combination work, note however: jython2.5.1/win32 does not provide a command line launcher, see https://bugs.jython.org/issue1491 . See pylib install documentation for how to work around.
fixes for handling of unicode exception values and unprintable objects
(issue87) fix unboundlocal error in assertionold code
(issue86) improve documentation for looponfailing
refine IO capturing: stdin-redirect pseudo-file now has a NOP close() method
ship distribute_setup.py version 0.6.10
added links to the new capturelog and coverage plugins
1.2.0 (2010-01-18)¶
refined usage and options for “py.cleanup”:
py.cleanup # remove "*.pyc" and "*$py.class" (jython) files py.cleanup -e .swp -e .cache # also remove files with these extensions py.cleanup -s # remove "build" and "dist" directory next to setup.py files py.cleanup -d # also remove empty directories py.cleanup -a # synonym for "-s -d -e 'pip-log.txt'" py.cleanup -n # dry run, only show what would be removed
add a new option “py.test –funcargs” which shows available funcargs and their help strings (docstrings on their respective factory function) for a given test path
display a short and concise traceback if a funcarg lookup fails
early-load “conftest.py” files in non-dot first-level sub directories. allows to conveniently keep and access test-related options in a
test
subdir and still add command line options.fix issue67: new super-short traceback-printing option: “–tb=line” will print a single line for each failing (python) test indicating its filename, lineno and the failure value
fix issue78: always call python-level teardown functions even if the according setup failed. This includes refinements for calling setup_module/class functions which will now only be called once instead of the previous behaviour where they’d be called multiple times if they raise an exception (including a Skipped exception). Any exception will be re-corded and associated with all tests in the according module/class scope.
fix issue63: assume <40 columns to be a bogus terminal width, default to 80
fix pdb debugging to be in the correct frame on raises-related errors
update apipkg.py to fix an issue where recursive imports might unnecessarily break importing
fix plugin links
1.1.1 (2009-11-24)¶
moved dist/looponfailing from py.test core into a new separately released pytest-xdist plugin.
new junitxml plugin: –junitxml=path will generate a junit style xml file which is processable e.g. by the Hudson CI system.
new option: –genscript=path will generate a standalone py.test script which will not need any libraries installed. thanks to Ralf Schmitt.
new option: –ignore will prevent specified path from collection. Can be specified multiple times.
new option: –confcutdir=dir will make py.test only consider conftest files that are relative to the specified dir.
new funcarg: “pytestconfig” is the pytest config object for access to command line args and can now be easily used in a test.
install
py.test
andpy.which
with a-$VERSION
suffix to disambiguate between Python3, python2.X, Jython and PyPy installed versions.new “pytestconfig” funcarg allows access to test config object
new “pytest_report_header” hook can return additional lines to be displayed at the header of a test run.
(experimental) allow “py.test path::name1::name2::…” for pointing to a test within a test collection directly. This might eventually evolve as a full substitute to “-k” specifications.
streamlined plugin loading: order is now as documented in customize.html: setuptools, ENV, commandline, conftest. also setuptools entry point names are turned to canonical names (“pytest_*”)
automatically skip tests that need ‘capfd’ but have no os.dup
allow pytest_generate_tests to be defined in classes as well
deprecate usage of ‘disabled’ attribute in favour of pytestmark
deprecate definition of Directory, Module, Class and Function nodes in conftest.py files. Use pytest collect hooks instead.
collection/item node specific runtest/collect hooks are only called exactly on matching conftest.py files, i.e. ones which are exactly below the filesystem path of an item
change: the first pytest_collect_directory hook to return something will now prevent further hooks to be called.
change: figleaf plugin now requires –figleaf to run. Also change its long command line options to be a bit shorter (see py.test -h).
change: pytest doctest plugin is now enabled by default and has a new option –doctest-glob to set a pattern for file matches.
change: remove internal py._* helper vars, only keep py._pydir
robustify capturing to survive if custom pytest_runtest_setup code failed and prevented the capturing setup code from running.
make py.test.* helpers provided by default plugins visible early - works transparently both for pydoc and for interactive sessions which will regularly see e.g. py.test.mark and py.test.importorskip.
simplify internal plugin manager machinery
simplify internal collection tree by introducing a RootCollector node
fix assert reinterpreation that sees a call containing “keyword=…”
fix issue66: invoke pytest_sessionstart and pytest_sessionfinish hooks on worker nodes during dist-testing, report module/session teardown hooks correctly.
fix issue65: properly handle dist-testing if no execnet/py lib installed remotely.
skip some install-tests if no execnet is available
fix docs, fix internal bin/ script generation
1.1.0 (2009-11-05)¶
introduce automatic plugin registration via ‘pytest11’ entrypoints via setuptools’ pkg_resources.iter_entry_points
fix py.test dist-testing to work with execnet >= 1.0.0b4
re-introduce py.test.cmdline.main() for better backward compatibility
svn paths: fix a bug with path.check(versioned=True) for svn paths, allow ‘%’ in svn paths, make svnwc.update() default to interactive mode like in 1.0.x and add svnwc.update(interactive=False) to inhibit interaction.
refine distributed tarball to contain test and no pyc files
try harder to have deprecation warnings for py.compat.* accesses report a correct location
1.0.3¶
adjust and improve docs
remove py.rest tool and internal namespace - it was never really advertised and can still be used with the old release if needed. If there is interest it could be revived into its own tool i guess.
fix issue48 and issue59: raise an Error if the module from an imported test file does not seem to come from the filepath - avoids “same-name” confusion that has been reported repeatedly
merged Ronny’s nose-compatibility hacks: now nose-style setup_module() and setup() functions are supported
introduce generalized py.test.mark function marking
reshuffle / refine command line grouping
deprecate parser.addgroup in favour of getgroup which creates option group
add –report command line option that allows to control showing of skipped/xfailed sections
generalized skipping: a new way to mark python functions with skipif or xfail at function, class and modules level based on platform or sys-module attributes.
extend py.test.mark decorator to allow for positional args
introduce and test “py.cleanup -d” to remove empty directories
fix issue #59 - robustify unittest test collection
make bpython/help interaction work by adding an __all__ attribute to ApiModule, cleanup initpkg
use MIT license for pylib, add some contributors
remove py.execnet code and substitute all usages with ‘execnet’ proper
fix issue50 - cached_setup now caches more to expectations for test functions with multiple arguments.
merge Jarko’s fixes, issue #45 and #46
add the ability to specify a path for py.lookup to search in
fix a funcarg cached_setup bug probably only occurring in distributed testing and “module” scope with teardown.
many fixes and changes for making the code base python3 compatible, many thanks to Benjamin Peterson for helping with this.
consolidate builtins implementation to be compatible with >=2.3, add helpers to ease keeping 2 and 3k compatible code
deprecate py.compat.doctest|subprocess|textwrap|optparse
deprecate py.magic.autopath, remove py/magic directory
move pytest assertion handling to py/code and a pytest_assertion plugin, add “–no-assert” option, deprecate py.magic namespaces in favour of (less) py.code ones.
consolidate and cleanup py/code classes and files
cleanup py/misc, move tests to bin-for-dist
introduce delattr/delitem/delenv methods to py.test’s monkeypatch funcarg
consolidate py.log implementation, remove old approach.
introduce py.io.TextIO and py.io.BytesIO for distinguishing between text/unicode and byte-streams (uses underlying standard lib io.* if available)
make py.unittest_convert helper script available which converts “unittest.py” style files into the simpler assert/direct-test-classes py.test/nosetests style. The script was written by Laura Creighton.
simplified internal localpath implementation
1.0.2 (2009-08-27)¶
fixing packaging issues, triggered by fedora redhat packaging, also added doc, examples and contrib dirs to the tarball.
added a documentation link to the new django plugin.
1.0.1 (2009-08-19)¶
added a ‘pytest_nose’ plugin which handles nose.SkipTest, nose-style function/method/generator setup/teardown and tries to report functions correctly.
capturing of unicode writes or encoded strings to sys.stdout/err work better, also terminalwriting was adapted and somewhat unified between windows and linux.
improved documentation layout and content a lot
added a “–help-config” option to show conftest.py / ENV-var names for all longopt cmdline options, and some special conftest.py variables. renamed ‘conf_capture’ conftest setting to ‘option_capture’ accordingly.
fix issue #27: better reporting on non-collectable items given on commandline (e.g. pyc files)
fix issue #33: added –version flag (thanks Benjamin Peterson)
fix issue #32: adding support for “incomplete” paths to wcpath.status()
“Test” prefixed classes are not collected by default anymore if they have an __init__ method
monkeypatch setenv() now accepts a “prepend” parameter
improved reporting of collection error tracebacks
simplified multicall mechanism and plugin architecture, renamed some internal methods and argnames
1.0.0 (2009-08-04)¶
more terse reporting try to show filesystem path relatively to current dir
improve xfail output a bit
1.0.0b9 (2009-07-31)¶
cleanly handle and report final teardown of test setup
fix svn-1.6 compat issue with py.path.svnwc().versioned() (thanks Wouter Vanden Hove)
setup/teardown or collection problems now show as ERRORs or with big “E“‘s in the progress lines. they are reported and counted separately.
dist-testing: properly handle test items that get locally collected but cannot be collected on the remote side - often due to platform/dependency reasons
simplified py.test.mark API - see keyword plugin documentation
integrate better with logging: capturing now by default captures test functions and their immediate setup/teardown in a single stream
capsys and capfd funcargs now have a readouterr() and a close() method (underlyingly py.io.StdCapture/FD objects are used which grew a readouterr() method as well to return snapshots of captured out/err)
make assert-reinterpretation work better with comparisons not returning bools (reported with numpy from thanks maciej fijalkowski)
reworked per-test output capturing into the pytest_iocapture.py plugin and thus removed capturing code from config object
item.repr_failure(excinfo) instead of item.repr_failure(excinfo, outerr)
1.0.0b8 (2009-07-22)¶
pytest_unittest-plugin is now enabled by default
introduced pytest_keyboardinterrupt hook and refined pytest_sessionfinish hooked, added tests.
workaround a buggy logging module interaction (“closing already closed files”). Thanks to Sridhar Ratnakumar for triggering.
if plugins use “py.test.importorskip” for importing a dependency only a warning will be issued instead of exiting the testing process.
many improvements to docs: - refined funcargs doc , use the term “factory” instead of “provider” - added a new talk/tutorial doc page - better download page - better plugin docstrings - added new plugins page and automatic doc generation script
fixed teardown problem related to partially failing funcarg setups (thanks MrTopf for reporting), “pytest_runtest_teardown” is now always invoked even if the “pytest_runtest_setup” failed.
tweaked doctest output for docstrings in py modules, thanks Radomir.
1.0.0b7¶
renamed py.test.xfail back to py.test.mark.xfail to avoid two ways to decorate for xfail
re-added py.test.mark decorator for setting keywords on functions (it was actually documented so removing it was not nice)
remove scope-argument from request.addfinalizer() because request.cached_setup has the scope arg. TOOWTDI.
perform setup finalization before reporting failures
apply modified patches from Andreas Kloeckner to allow test functions to have no func_code (#22) and to make “-k” and function keywords work (#20)
apply patch from Daniel Peolzleithner (issue #23)
resolve issue #18, multiprocessing.Manager() and redirection clash
make __name__ == “__channelexec__” for remote_exec code
1.0.0b3 (2009-06-19)¶
plugin classes are removed: one now defines hooks directly in conftest.py or global pytest_*.py files.
added new pytest_namespace(config) hook that allows to inject helpers directly to the py.test.* namespace.
documented and refined many hooks
added new style of generative tests via pytest_generate_tests hook that integrates well with function arguments.
1.0.0b1¶
introduced new “funcarg” setup method, see doc/test/funcarg.txt
introduced plugin architecture and many new py.test plugins, see doc/test/plugins.txt
teardown_method is now guaranteed to get called after a test method has run.
new method: py.test.importorskip(mod,minversion) will either import or call py.test.skip()
completely revised internal py.test architecture
new py.process.ForkedFunc object allowing to fork execution of a function to a sub process and getting a result back.
XXX lots of things missing here XXX
0.9.2¶
refined installation and metadata, created new setup.py, now based on setuptools/ez_setup (thanks to Ralf Schmitt for his support).
improved the way of making py.* scripts available in windows environments, they are now added to the Scripts directory as “.cmd” files.
py.path.svnwc.status() now is more complete and uses xml output from the ‘svn’ command if available (Guido Wesdorp)
fix for py.path.svn* to work with svn 1.5 (Chris Lamb)
fix path.relto(otherpath) method on windows to use normcase for checking if a path is relative.
py.test’s traceback is better parseable from editors (follows the filenames:LINENO: MSG convention) (thanks to Osmo Salomaa)
fix to javascript-generation, “py.test –runbrowser” should work more reliably now
removed previously accidentally added py.test.broken and py.test.notimplemented helpers.
there now is a py.__version__ attribute
0.9.1¶
This is a fairly complete list of v0.9.1, which can serve as a reference for developers.
allowing + signs in py.path.svn urls [39106]
fixed support for Failed exceptions without excinfo in py.test [39340]
added support for killing processes for Windows (as well as platforms that support os.kill) in py.misc.killproc [39655]
added setup/teardown for generative tests to py.test [40702]
added detection of FAILED TO LOAD MODULE to py.test [40703, 40738, 40739]
fixed problem with calling .remove() on wcpaths of non-versioned files in py.path [44248]
fixed some import and inheritance issues in py.test [41480, 44648, 44655]
fail to run greenlet tests when pypy is available, but without stackless [45294]
small fixes in rsession tests [45295]
fixed issue with 2.5 type representations in py.test [45483, 45484]
made that internal reporting issues displaying is done atomically in py.test [45518]
made that non-existing files are ignored by the py.lookup script [45519]
improved exception name creation in py.test [45535]
made that less threads are used in execnet [merge in 45539]
removed lock required for atomic reporting issue displaying in py.test [45545]
removed globals from execnet [45541, 45547]
refactored cleanup mechanics, made that setDaemon is set to 1 to make atexit get called in 2.5 (py.execnet) [45548]
fixed bug in joining threads in py.execnet’s servemain [45549]
refactored py.test.rsession tests to not rely on exact output format anymore [45646]
using repr() on test outcome [45647]
added ‘Reason’ classes for py.test.skip() [45648, 45649]
killed some unnecessary sanity check in py.test.collect [45655]
avoid using os.tmpfile() in py.io.fdcapture because on Windows it’s only usable by Administrators [45901]
added support for locking and non-recursive commits to py.path.svnwc [45994]
locking files in py.execnet to prevent CPython from segfaulting [46010]
added export() method to py.path.svnurl
fixed -d -x in py.test [47277]
fixed argument concatenation problem in py.path.svnwc [49423]
restore py.test behaviour that it exits with code 1 when there are failures [49974]
don’t fail on html files that don’t have an accompanying .txt file [50606]
fixed ‘utestconvert.py < input’ [50645]
small fix for code indentation in py.code.source [50755]
fix _docgen.py documentation building [51285]
improved checks for source representation of code blocks in py.test [51292]
added support for passing authentication to py.path.svn* objects [52000, 52001]
removed sorted() call for py.apigen tests in favour of [].sort() to support Python 2.3 [52481]