Explanation of the test types available as part of model concurrency testing.
Peak load test
This test is used for measuring the average response times and the throughput of the models at given numbers of concurrent users.
These scenarios are tested:
- Simulating the anticipated peak hour of the typical business day.
- Simulating the anticipated peak hour of an exceptional day, for example, many users completing their transactions at the end of a financial period.
User ramp-up period is 20 minutes.
Peak load duration is one hour.
Stress test
This test is used to observe other performance characteristics of the models, such as how well the model scales to a growing number of users.
The scenarios simulate loads beyond those of the peak load test, such as double, or triple the number of concurrent users.
Although we measure response times and throughput, the focus of the test is to explore how a model reacts to an increasing number of users. The results can show whether the model can handle an excess number of users. The results can also show the model concurrency limit — when the model becomes unreasonably slow.
User ramp-up period is three hours.
Peak load duration is 5 to 10 minutes.
Endurance test
This test observes characteristic changes in models over time. The main aim of this test is to find out if there are any long-term stability issues. For example, does the model grow over time, and are response times for the same actions affected?
By completing user journeys at an accelerated rate and for an extended time, we can observe trend results and extrapolate. We can begin to understand and build a picture of what a production model may be like in the future. For example, in 12 months' time.
When the model experiences load for an extended time through the endurance test, it simulates the near-future state of the model. The endurance test provides the size of the model from the start to the end of the test, in context of the number of actions processed. These findings can also be extrapolated to assess future risks.
User ramp-up period is one hour.
Peak load duration is eight hours or more.
