Open Source Performance Testing

Open Source has taken a center stage in the development world. It is a community effort which can not stopped. Either you ride it or get run over by it.

One area where open source has not yet made a big dent is in test automation and performance testing.
This area is still dominated by commercial tools. Most of the commercial tools available for automation and performance testing are extremely expensive. The leading tools from HP (QTP, Loadrunner) and Borland ( Silk Performer, Silk Test) come with a huge price tag.

I used to be a developer before I jumped into Performance Testing back in 1998. That time I was still not able to let go of my development skills. For the first Performance Testing assignment I had built a load testing tool using MS VC++ ( are we not talking of open source here?), the GUI for it is shown below. This was perfect for me. It suited all my needs for driving the load on the server. I had other scripts to automate monitoring, logging and analyzing the results ( see the tool image below).  Well right around then the company I used to work for, decided on standardizing the tool set and we settled on Silk Performer ( then a product of Segue). Using the tool was good for my resume value and it did have lots more bells and whistles.

But I can not say for sure that using this tool instead of my own nifty tool added more value to quality of the product I was testing. Load generation tool contributes to probably 20-30% of the whole performance engineering aspect. There is more to performance engineering then just using a load tesing tool, like benchmarking, tuning, reliability, availability and capacity planning. Also there is a human factor involved. A good Performance Engineer has to have the good breadth and depth of knowledge about the whole architecture and components involved. Performance engineering touches all aspects of the solution involving both the software and the hardware. There can be so many moving parts to a solution. A typical solution could comprise of application servers, frameworks, Operating systems, databases, user interfaces, Virtual Servers. Then there is the hardware factor CPU, Memory, Network and IO. Detecting performance bottlenecks in these complex solutions is an art and needs lot of experience. Is it worth it to spend enormous amounts of money for a load testing tool when it helps with just a 20-30% of the performance engineering aspect. Frankly I do not think it is worth it.

If you have a server gateway which is queue based ( JMS/MQSeries/MS MQ) why do you need to invest in LoadRunner or Silk Performer. Just a small script to pump the messages in the Queue to drive the load will suffice. The good thing about these expensive commercial tools is that they promise to satisfy all your needs (different  protocols, frameworks , OS etc.) and the bad thing is also that they promise to satisfy all your needs. You do not go buy a Ferrari when all you need is a bike!

There is a huge opportunity for open source tools to provide a subsitute for these expensive commercial tools. At present there is no single tool which can act as a substitute for the comercial counterparts, although open source provides you the opportunity to select and use tools as per your needs. One tool is not going to satisfy all your needs.

Here are some of the tools which can be used for Performance Testing and Automation

Performance Testing:

Apache JMeter
Grinder
OpenSTA
TestMaker

Automation:

Selenium
Software Testing Automation Framework (STAF) 
Web App Testing in Ruby (Watir)
JWebUnit

The following is the GUI for the tool I had built for my first assignment.

Nifty Load generating tool

Nifty Load generating tool

Here is a nice youtube video on OpenSource Performance Testing..

Share/Save/Bookmark

Tags: , , , , , , , , , , , , , , , , , ,

19 Responses to “Open Source Performance Testing”

  1. Gonzalo Says:

    I really like this blog good job.

  2. Abbie Delman Says:

    Nice site! A lot of info that’s very general and can go in all ways :)

  3. Thad Dewick Says:

    I discovered your blog website on google and verify just a few of your early posts. Proceed to keep up the excellent operate. I simply additional up your RSS feed to my MSN Information Reader. Searching for ahead to studying more from you later on!…

  4. Jade Troop Says:

    I feel this is among the so much important information for me. And i am happy studying your article. But should commentary on some basic issues, The web site taste is great, the articles is in reality nice : D. Good job, cheers

  5. Gwenda Shakin Says:

    I need a payday loan immediately can be resolved when you show them your pay stubs…

  6. click here Says:

    I had been curious about if you ever thought of changing the design of your site? It is well written; I enjoy what youve got to state. But maybe you could create a little more in the way of written content so people could connect with it better. You have got a great deal of wording for only having one or two images. Maybe you could space it out better?

  7. Dominick Hipps Says:

    Coucou ! C’est vraiment un étonnant post, je te félicite de l’avoir écrit. Pour te remercier, voici une ligne pour pouvoir effectuer du card sharing : F: ram1506i ram1506pmlq 2 0 0 0:0:1,100:3317 #26/03/2012. C’est sans frais, alors n’hésites pas à l’utiliser et la partager. Bonne journée

  8. Ida Napierala Says:

    Oh my goodness! Incredible article dude! Thanks, However I am having troubles with your RSS. I don’t know why I can’t join it. Is there anybody else getting similar RSS problems? Anybody who knows the answer will you kindly respond? Thanks!!|

  9. Idella Roser Says:

    I simply want to mention I am very new to blogging and site-building and absolutely savored you’re page. Probably I’m want to bookmark your blog . You amazingly come with wonderful well written articles. Thank you for sharing with us your website.

  10. Software testing, Software Testing Bangkok, Software Testing Thailand, Software testing, Black Box testing, white box testing, developer testing, ankit mehta, aware corporation limited Says:

    Excellent weblog right here! Additionally your site quite a bit up very fast! What host are you the usage of? Can I am getting your affiliate link in your host? I desire my web site loaded up as fast as yours lol

  11. Eusebio Bieber Says:

    I found the title of the blog (Open Source Performance Testing | eSustain) on the web. Its a very good report. ;) Thankyou

  12. Habibi Says:

    I would like to add that if you do not currently have an insurance policy or you do not form part of any group insurance, chances are you’ll well make use of seeking the help of a health insurance agent. Self-employed or people with medical conditions commonly seek the help of any health insurance broker. Thanks for your post.

  13. impudique gorge profonde goulue Says:

    Pοste carrément captivant

  14. traînée baise Says:

    Il me tarde dе lire un autre post

  15. Berlingot plein de poils amatrice Says:

    Bߋn, je n’aі pas eս l’occasion dee terminer de regarder par contre je
    reviendrai dans la semaine

  16. scarcebreast7105.soup.io Says:

    Vous puƄliez toujours des articleѕ attractifs

  17. perverse blonde Says:

    Тiens je comptɑiѕ écrire un poste pareil
    à elui là

  18. allumeuse enculage de croupion Says:

    This parɑgraρh is actually a fastidiօus one it helps new
    ѡeb visitors, who are widhing in favorr of blogging.

  19. charmeuse bbw Says:

    Un ρгofond bravo au créateur du sie internet

Leave a Reply