20 April 2012

Load testing .NET applications is (still) expensive

  Performance analysis is usually a late activity in the software development process but a vital one if you have to deploy your application in production.
After an intense performance analysis session during load testing I decided to share my experience  about the tools that I used.

  Redgate's ANTS Performance Profiler(RAPP) is the best tool for performance analysis when you are dealing with small scale application usage scenarios. Very easy to get it up and running and intuitive to use and read the profiling results.
  Visual Studio 2010 Performance Profiler(VSPP) is a good tool as well but is less intuitive to use (you need to spend some time reading about the various configuration options). It is a lot more expensive being part of Visual Studio 2010 Premium Edition which is a couple of thousand $/£/€ more compared to a couple of hundred for RAPP. For small scale usage scenario analysis I would use RAPP exclusively given its price and usability.

  The situation is different when you need to do load testing for a large number of users and when you want to reach the absolute limits of how much load your application can take. RAPP has a bigger overhead than VSPP and makes the load testing process very slow overall - I don't think RAPP is even meant to be used in this scenario. VSPP has a very small overhead and you can undertake performance analysis while bombarding your application with a lot of requests. It is like this tool has used a back door into the operating system and it just gets the data that it needs. You can download VSPP as  command line tool so you can use it on a computer that does not have Visual Studio installed. Although you can gather the test results for free in order to read the final report you need to have at least Visual Studio Premium edition. If you can generate the load test manually or using a third party tool your expenses stop here but if you want to use Visual Studio to generate the load test you have to buy Visual Studio Ultimate edition (which is a lot more expensive than the price of Visual Studio Premium).

  There is a lot more to be said on this subject but the takeaway is that if you your application needs support a couple of thousand users that perform frequent writes be prepared to invest a lot of many and a reasonable development effort or less money and a considerable amount of effort - load testing in .NET is not cheap.

No comments: