Build for production with performance testing
Software testing is what I do for a living and an integral part of any engineering project I have worked on. It’s the backbone for the design, analysis, development and management of any sort of technical or software project, within any vertical. Software is susceptible to issues arising from increased demand. Performance testing is critical to the understanding and prevention of such problems. As a Test Automation Engineer for Cabiri, I design and run effective performance test strategies to answer and resolve these questions:
Can this software cope with increased load?
What is the maximum load that can be processed?
Where are the bottlenecks?
Will the software scale?
What types of scaling would be the most efficient?
What are the hosting costs likely to be?
Planning a strategy is complex, there are multiple testing types to target and most projects will have additional unique requirements. Let’s run through four core types typical of most performance testing strategies. Before getting started there are a couple of essential jobs to do. Job number one, define the specifics of the load being tested, for example, website visitors, number of transactions, applications, votes, views and so on. Job number two, use business data to predict expected stress levels.
TYPES OF TESTING METHODS
LOAD TEST - Most fundamental. Gives insight into how an application behaves under the specified load, with the objective to identify potential bottlenecks. It also provides information with regards to the infrastructure.
STRESS TEST - The objective of this test is to find breaking points by simulating the application under a higher than usual workload, usually well above expected limits. This puts a planned and measured strain on specified resources such as CPU, memory, disks, and network interfaces.
SPIKE TEST - This test verifies how the system performs under a sudden increase in load. It helps determine if performance will be affected and if the system will fail. An example of this could be an unexpected spike in sales following (marketing campaign) influencer coverage.
SOAK TEST - The purpose of this test is to spot any performance and reliability issues arising from a system being used for an extended period of time. Problems typically relate to resource leaks - memory, storage or database misconfiguration.
Key performance indicators (KPIs) also known as metrics are used to measure the effectiveness of each testing type. The analysis of metrics is a multidimensional exercise. Examples include CPU usage, memory and disk utilisation, response and load times, throughput, bandwidth, requests per second and error rate. Ensuring that an infrastructure runs on the desired specification is important and metrics analysis can help with this too. It is vital to assess if the current setup can meet requirements.
Operationally, a metric becomes a point that can be used to understand what's going on across the wider dataset, from the application being tested and beyond. An important aspect of load testing, especially in preparation for peak times. Does a load journey travel beyond the application being tested, to third party systems. We need confirmation that all parties meet performance requirements and are ready to handle the additional load.
A good example of this is the payment processor. Let's imagine a scenario where we have one million users engaged with an entertainment application and browsing events. What happens when 10% of these users decide that they are ready to check their baskets out the moment that tickets go on sale for a concert? This payment processor would receive 100,000 simultaneous requests, possibly more depending on the nature of the implementation. If the application isn’t ready for it and starts to fall over then some customers will undoubtedly have a negative experience. With a real and lasting impact on customer satisfaction. Site credibility will diminish and a lot more work will be created for the customer support team.
In applying these principles to composable commerce, we have learnt that engaging early will give the platform the best chance to succeed in both performance and cost of development. Working closely with SaaS application providers ensures testing yields representative and accurate results setting up our customers with the very best support for both day to day and peak trading periods.
