You use load test tools to break application software you are testing. What if the tool itself breaks and the applications under test do not even have a dent?

This is exactly what happened when using JMS protocol in HPPC ( Loadrunner).  There is a memory leak in HPPC when using JMS. The mmdrv.exe process invokes JVM (in-process) during use of one of the jms functions (jms_send_message_queue).

During one of our tests we started getting out of memory error. Of course if you increase the JVM heap ( using -Xmx ) the error will be delayed but the fact remains it is leaking memory. It was very embarrassing to tell the application team that our tool is breaking during load testing.

We have been trying desperately to get HP to admit and fix it but they are making us go around in circle. We  used JConsole to monitor the JVM and prove to them that there is a leak but they still do not get it. The only thing that is left is for us to now fix the code for them.

In order to use JConsole with loadrunner JMS, you need to add the following in the -Dcom.sun.management.jmxremote in the “Additional VM Parameters” in order for JConsole to connect to the JVM.  Once you do that you can see the JVM memory keep on climbing inspite of you manually invoking GC.

Are you listening HP?

Open Source has taken a center stage in the development world. It is a community effort which can not stopped. Either you ride it or get run over by it.

One area where open source has not yet made a big dent is in test automation and performance testing.
This area is still dominated by commercial tools. Most of the commercial tools available for automation and performance testing are extremely expensive. The leading tools from HP (QTP, Loadrunner) and Borland ( Silk Performer, Silk Test) come with a huge price tag.

I used to be a developer before I jumped into Performance Testing back in 1998. That time I was still not able to let go of my development skills. For the first Performance Testing assignment I had built a load testing tool using MS VC++ ( are we not talking of open source here?), the GUI for it is shown below. This was perfect for me. It suited all my needs for driving the load on the server. I had other scripts to automate monitoring, logging and analyzing the results ( see the tool image below).  Well right around then the company I used to work for, decided on standardizing the tool set and we settled on Silk Performer ( then a product of Segue). Using the tool was good for my resume value and it did have lots more bells and whistles.
(more…)