Load vs Stress Testing

Overview

Load Testing and Stress Testing both fall under the umbrella of Performance Testing. Their primary objectives are:

• Load Testing: tests the system under normal or expected load, up to the point where errors appear.
• Stress Testing: tests the system in an incremental manner to find the point where errors appear, or its breaking point.

For more details, continue reading. There is overlap between these two, and it is discussed in the Conclusion.

Comparison

Table template created by ChatGPT of OpenAI.

SLA: Service-Level Agreement. This has a broad meaning, but in the context of latency, throughput, and error rate, it is the rate that is communicated (in writing or verbally to customers or other client services of this server) as a guaranteed level of performance.

Load Testing

The purpose of Load Testing is to identify either:

• The system works as expected at normal load.
• The system’s peak load before an error rate above SLA is observed.

Reactively Scaled System

Reactively Scaled Systems are systems that define a minimum and maximum number of replicas for its microservices. When idle, the system’s services deploy the minimum number of replicas. However, this minimum is likely not enough to handle normal or peak loads.

Direct Request / Response Services

Testing reactively scaled systems can be a challenge especially for Direct Request / Response Services. Even if retry mechanisms are implemented to provide a larger window of time for the downstream service (the service receiving the requests) to handle requests, these retry mechanisms exist at the application layer. Requests can be dropped at the network infrastructure layer, before they ever reach the application.

How this problem is mitigated depends on the traffic pattern of the system, and its deployment and scaling strategy. If the traffic pattern makes sharp changes that are too fast for the service’s reactive scaling, then:

• Adjust the scaling parameters (to be more sensitive, or select parameters that are more relevant for your service).
• Create a scaling schedule.

Asynchronous Services with Persistent Intermediary

Testing Asynchronous Services with a Persistent Intermediary (e.g. a message broker or distributed storage) is less of a challenge than Direct Request / Response Services, because downstream services that consume from the Persistent Intermediary have time to scale up as messages accumulate. This can still pose a problem if SLAs need to be met. This can be mitigated using similar mechanisms as Direct Request / Response Services.

Stress Testing

All the considerations above also apply to Stress Testing. This is where you take a system to the point of failure, and then observe its recovery. The exact definition of failure is up to interpretation. It could be the point where errors are observed, or it could be the point where services begin to crash.

Conclusion

Stress Testing is typically taking the system one step beyond where Load Testing takes it. The exact definition of this extent is up to interpretation. There is overlap. If you take the system up to the point where errors appear, then this could be categorized as either Load Testing (because you’re measuring its peak), or it could also be categorized as Stress Testing (because you’ve pushed the system beyond what it can handle).

Diagrams to be added, as well as a deeper dive into scaling parameters.

Author

Sam Malayek works in Vancouver, using this space to fill in a few gaps. Opinions are his own.

• ← Concurrency vs Parallelism
• Symmetric vs Asymmetric Encryption →