Skip to main content

@testplane/test-repeater

Overview​

Use the @testplane/test-repeater plugin to run the same test (or group of tests) the required number of times.

This plugin can be useful in cases when you need to make sure that the written tests are stable. The plugin guarantees that the tests will be run as many times as you set, regardless of the results of their run in each attempt. In addition, the plugin allows you to run tests every time in a new browser session. This eliminates the impact of browser degradation or any other side effects that could occur during repeated runs in the same browser session.

Install​

npm install -D @testplane/test-repeater

Setup​

Add the plugin to the plugins section of the testplane config:

module.exports = {
    plugins: {
        "@testplane/test-repeater": {
            enabled: true,
            repeat: 50,
            minRepeat: 10,
            maxRepeat: 100,
            uniqSession: true,
        },

        // other Testplane plugins...
    },

    // other Testplane settings...
};

Description of configuration parameters​

ParameterTypeDefault valueDescription
enabledBooleantrueEnable / disable the plugin.
repeatNumber0How many times you need to run the test, regardless of the result of its run.
minRepeatNumber0The minimum number of times the test can be run.
maxRepeatNumberInfinityThe maximum number of times the test can be run.
uniqSessionBooleantrueRun each test in a unique browser session.

Passing parameters via the CLI​

All plugin parameters that can be defined in the config can also be passed as command line options or through environment variables during Testplane startup. Use the prefix --test-repeater- for command line options and testplane_test_repeater_ for environment variables. For example:

npx testplane --test-repeater-repeat 5
testplane_test_repeater_repeat=5 npx testplane

Usage​

Option --repeat​

The plugin also adds a special --repeat option to Testplane's CLI, with which you can run the test the right number of times in a more convenient way. For example:

npx testplane --repeat 5

Usage Examples​

Below are examples of test runs, in which they were run 21 times (1 primary + 20 retries) to check their stability.

Example of Broken Tests​

As seen in the screenshot, the tests were run 21 times and never completed successfully:

Broken Test

Example of Stable Test​

Here, on the contrary, all runs were successful:

Stable Test

Example of Unstable Tests​

In the next screenshot, the first test is almost non-functional – out of 21 attempts, the test passed only once. The second test is quite stable, although 2 failures out of 21 is still not 100% stability. The developer may try to understand why the test fails occasionally.

Unstable Test

On this page