Configuration¶
cargo-nextest supports repository-specific configuration at the location .config/nextest.toml
from the Cargo workspace root. The location of the configuration file can be overridden with the --config-file
option.
The default configuration shipped with cargo-nextest is:
# This is the default config used by nextest. It is embedded in the binary at
# build time. It may be used as a template for .config/nextest.toml.
[store]
# The directory under the workspace root at which nextest-related files are
# written. Profile-specific storage is currently written to dir/<profile-name>.
dir = "target/nextest"
# This section defines the default nextest profile. Custom profiles are layered
# on top of the default profile.
[profile.default]
# "retries" defines the number of times a test should be retried. If set to a
# non-zero value, tests that succeed on a subsequent attempt will be marked as
# flaky. Can be overridden through the `--retries` option.
# Examples
# * retries = 3
# * retries = { backoff = "fixed", count = 2, delay = "1s" }
# * retries = { backoff = "exponential", count = 10, delay = "1s", jitter = true, max-delay = "10s" }
retries = 0
# The number of threads to run tests with. Supported values are either an integer or
# the string "num-cpus". Can be overridden through the `--test-threads` option.
test-threads = "num-cpus"
# The number of threads required for each test. This is generally used in overrides to
# mark certain tests as heavier than others. However, it can also be set as a global parameter.
threads-required = 1
# Show these test statuses in the output.
#
# The possible values this can take are:
# * none: no output
# * fail: show failed (including exec-failed) tests
# * retry: show flaky and retried tests
# * slow: show slow tests
# * pass: show passed tests
# * skip: show skipped tests (most useful for CI)
# * all: all of the above
#
# Each value includes all the values above it; for example, "slow" includes
# failed and retried tests.
#
# Can be overridden through the `--status-level` flag.
status-level = "pass"
# Similar to status-level, show these test statuses at the end of the run.
final-status-level = "flaky"
# "failure-output" defines when standard output and standard error for failing tests are produced.
# Accepted values are
# * "immediate": output failures as soon as they happen
# * "final": output failures at the end of the test run
# * "immediate-final": output failures as soon as they happen and at the end of
# the test run; combination of "immediate" and "final"
# * "never": don't output failures at all
#
# For large test suites and CI it is generally useful to use "immediate-final".
#
# Can be overridden through the `--failure-output` option.
failure-output = "immediate"
# "success-output" controls production of standard output and standard error on success. This should
# generally be set to "never".
success-output = "never"
# Cancel the test run on the first failure. For CI runs, consider setting this
# to false.
fail-fast = true
# Treat a test that takes longer than the configured 'period' as slow, and print a message.
# See <https://nexte.st/book/slow-tests> for more information.
#
# Optional: specify the parameter 'terminate-after' with a non-zero integer,
# which will cause slow tests to be terminated after the specified number of
# periods have passed.
# Example: slow-timeout = { period = "60s", terminate-after = 2 }
slow-timeout = { period = "60s" }
# Treat a test as leaky if after the process is shut down, standard output and standard error
# aren't closed within this duration.
#
# This usually happens in case of a test that creates a child process and lets it inherit those
# handles, but doesn't clean the child process up (especially when it fails).
#
# See <https://nexte.st/book/leaky-tests> for more information.
leak-timeout = "100ms"
# `nextest archive` automatically includes any build output required by a standard build.
# However sometimes extra non-standard files are required.
# To address this, "archive.include" specifies additional paths that will be included in the archive.
archive.include = [
# Examples:
#
# { path = "application-data", relative-to = "target" },
# { path = "data-from-some-dependency/file.txt", relative-to = "target" },
#
# In the above example:
# * the directory and its contents at "target/application-data" will be included recursively in the archive.
# * the file "target/data-from-some-dependency/file.txt" will be included in the archive.
]
[profile.default.junit]
# Output a JUnit report into the given file inside 'store.dir/<profile-name>'.
# If unspecified, JUnit is not written out.
# path = "junit.xml"
# The name of the top-level "report" element in JUnit report. If aggregating
# reports across different test runs, it may be useful to provide separate names
# for each report.
report-name = "nextest-run"
# Whether standard output and standard error for passing tests should be stored in the JUnit report.
# Output is stored in the <system-out> and <system-err> elements of the <testcase> element.
store-success-output = false
# Whether standard output and standard error for failing tests should be stored in the JUnit report.
# Output is stored in the <system-out> and <system-err> elements of the <testcase> element.
#
# Note that if a description can be extracted from the output, it is always stored in the
# <description> element.
store-failure-output = true
# This profile is activated if MIRI_SYSROOT is set.
[profile.default-miri]
Profiles¶
With cargo-nextest, local and CI runs often need to use different settings. For example, CI test runs should not be cancelled as soon as the first test failure is seen.
cargo-nextest supports multiple profiles, where each profile is a set of options for cargo-nextest. Profiles are selected on the command line with the -P
or --profile
option. Most individual configuration settings can also be overridden at the command line.
Here is a recommended profile for CI runs:
[profile.ci]
# Print out output for failing tests as soon as they fail, and also at the end
# of the run (for easy scrollability).
failure-output = "immediate-final"
# Do not cancel the test run on the first failure.
fail-fast = false
After checking the profile into .config/nextest.toml
, use cargo nextest --profile ci
in your CI runs.
Default profiles
Nextest's embedded configuration may define new profiles whose names start with default-
in the future. To avoid backwards compatibility issues, do not name custom profiles starting with default-
.
Tool-specific configuration¶
Some tools that integrate with nextest may wish to customize nextest's defaults. However, in most cases, command-line arguments and repository-specific configuration should still override those defaults.
To support these tools, nextest supports the --tool-config-file
argument. Values to this argument are specified in the form tool:/path/to/config.toml
. For example, if your tool my-tool
needs to call nextest with customized defaults, it should run:
cargo nextest run --tool-config-file my-tool:/path/to/my/config.toml
The --tool-config-file
argument may be specified multiple times. Config files specified earlier are higher priority than those that come later.
Hierarchical configuration¶
For this example:
Configuration is resolved in the following order:
- Command-line arguments. For example, if
--retries=3
is specified on the command line, failing tests are retried up to 3 times. - Environment variables. For example, if
NEXTEST_RETRIES=4
is specified on the command line, failing tests are retried up to 4 times. - Per-test overrides, if they're supported for this configuration variable.
-
If a profile is specified, profile-specific configuration in
.config/nextest.toml
. For example, if the repository-specific configuration looks like:[profile.ci] retries = 2
then, if
--profile ci
is selected, failing tests are retried up to 2 times. -
If a profile is specified, tool-specific configuration for the given profile.
- Repository-specific configuration for the
default
profile. For example, if the repository-specific configuration looks like:then failing tests are retried up to 5 times.[profile.default] retries = 5
- Tool-specific configuration for the
default
profile. - The default configuration listed above, which is that tests are never retried.