Archive

Archive for the ‘LoadRunner’ Category

How to debug LoadRunner script

December 16th, 2010
Comments Off

From time to time there are situations where you work on a script, and it looks fine, compiles with no issues but simply doesn’t work as you expect. In such case there is no other option than just knuckle down and get a little bit dirty. That’s the right time for debugging.

And here some tips from me that (hopefully) will help you making this step less painful.

1) Increase log level
By default, VuGen report only standard information. If you want to see more verbose details, then go to Vuser -> Run-Time settings -> Log and enable “Extended log” option.

lr_prop

2) Use HTTP Proxy
Another option is to find by checking what exactly LoadRunner sends to the server. Personally I use WebScarab and Fidler (from time to time) to sniff network traffic and to check what data my script generate. You can setup proxy in VuGen by going to Vuser -> Run-Time settings -> Proxy.

Below is a setup for WebScarab running locally:

lr_webscarab_setup

and sample traffic as seen in WebScarab:

webscarab_sample_session

3) Use text tools to compare HTTP requests from LR and from real browser
Lets say you sniffed LR HTTP call and browser HTTP call e.g. with local proxy and you want to compare both to search for any differences. Sometimes your HTTP call fails only because of e.g. missing dot somewhere. And such bugs are even harder to find because there is no obvious “Hello, I’m here” error type of message. Try to use Notepad+ or ConTEXT (http://www.contexteditor.org).

Sample results of HTTP requests comparison in ConTEXT editor:

compare_results

4) Use breakpoint
Breakpoint is a place in script where LR will pause running so you can see all runtime values. Howto setup a breakpoint? Simply right click line where you would like to pause and select “Toggle Breakpoint”. This will put red dot at the beginning of line. It looks like that:

breakpoint

Now, simply start the script, and wait until script reaches this breakpoint. When it’s done, in output window you can switch to “RunTime Data” and see detailed information like which iteration is running currently, which action, line number and (most interesting) all parameter with their current values:

breakpoint_pause

To resume running, simply hit run button once again. To disable a break point, also right click the line and select the same “Toggle Breakpoint”.

5) Run script step by step
Running step by step means that instead of going from one call to another automatically, you need to hit F10 in order to perform next call. This is actually helpful when running with Browser preview or when you have lots of if/while/do…while/for loops and you want to see exactly what LR is doing and where.
To run scrip in step by step mode you just simply hit F10 or select Vuser -> Run Step by Step.

6) Force script to run with specific parameters values
This is actually a trick, definitely not something described in VuGen manual. Lets say your script use parameters very heavily and depends on them as well. Now, you have e.g. parameter USERNAME with lots of accounts to use but you want to use one very specific account. One method is to update/change the parameter so it will pick up correct account. But in such case you need to remember to undo your changes after you are done with the test. Another method is to leave parameter unchanged, and just override it’s value during runtime. For that you can use lr_save_string() function and place it somewhere at the beginning.

One could say that it’s even more dangerous than first method since it always overrides param value so when left unchanged could fail every future load test. But even for than there is a solution. Trick that I use looks like this:

  1. int id;
  2. lr_whoami(&id, NULL, NULL);
  3.  
  4. if(id < 0)
  5. {
  6.         lr_save_string("Pete", "USERNAME");
  7. }

And basically you can leave it in your script because LR will reach this lr_save_string call only in VuGen. When running in LR Controller, lr_save_string() won’t be reached (lr_whoami called in VuGen always return -1).

LoadRunner , , , ,

My LoadRunner Tips and Tricks

July 12th, 2010
Comments Off

Few tips&tricks to make working with LoadRunner easy:

- Store URL of your application in one single parameter. In case you need to change it, it requires a single change in one place, not tons of them for every single web request.

- If you don’t know why your HTTP/HTTPS request is failing, use proxy to sniff it and them compare it with the same request but generated from real browser. As a proxy you can use WebScarab or Fiddler.

- Separate load logic using actions, don’t store whole load in a huge single action that LR created be default at the beginning.

- Create web_reg_find() checkpoint for every HTTP request that you make. You don’t want to pass failing transations.

- On parameter setup, beware using “Update value” set to “Once”. Rather use “Every iteration”. If you mess it up somehow, at least only current iteration will fail, not all of them.

- For scripts that handle UI actions (like Web Click&Script or AJAX Click&Script) use “Run every user as a process” instead of “Run every user as a thread”. LR is not very good at memory management for such scripts so it’s better to run each vuser separately.

- Don’t use extended log level while running test in a Controller. One more time LR is not very good at handling it. Rather use basic level and if there is an issue switch to VuGen to digg for it.

- In LR Analysis, check performance results on a summary graph instead of “Average transactions response times” graph.

- Place 1 second wait before first call to the server. If server breaks, you won’t flood it with bunch of requests.

- Try to avoid using standard C string functions (strcpy, strcat, etc…) if you can. Or if you really need, just remember about null pointer issue. More details here http://blog.testsautomation.com/2009/11/exception-access_violation/.

- Comment your code – at the end LR script is (most cases) a C program. So all programming best practices apply as well.

- Make your code to look nice (tabs, indents, etc…). It will be easier for you to work with it and spot any issue.

- Don’t hard code anything. Instead put it into parameter.

- If some calls repeat constantly, put them into separate action and call this action instead. It will help you during refactoring your tests.

- NEVER store Controller scenario in QualityCenter. It’s just bad idea. Use local hard drive for it. Controller can store really big files there.

- When you work on a script in VuGen, try to run it there at lest for 2 or 3 iterations. Sometimes errors occur only after the first iteration.

- If you want you can store LR script on a shared drive. It won’t affect your tests since Controller will always copy it to temp directory before running the scenario (even if the script is stored locally on the same drive).

- If you want to send LR script to someone use File -> Zip Operations -> Zip and Email. On popup select “Run-Time files” instead of “All files”. This second options can zip results as well which may generate really unnecessary huge file.

LoadRunner

LoadRunner levels of integration with web pages

March 9th, 2010
Comments Off

LoadRunner basically allows two approches for load testing your web application. Before going into details lets see how communication between user and web server looks like.

As you can see between user and web server there are mainly 2 layers:

  • Layer 1 is where user interacts with the browser by e.g. clicking a button, selecting values from the list or submitting a form
  • Layer 2 is where browser communicates with web server which includes:
    • creating and sending HTTP requests to the web server based on user’s actions
    • receiving HTTP responses from the web server
    • rendering HTTP responses, forming an UI and displaying it to the user

Below I’m providing two scripts that do exactly the same thing but in slightly different way. The goal for each script is to search for a “linux” word using google search.

First, lets look at “Web (HTTP/HTML)” type of the script:

  1. Action()
  2. {
  3.         web_url("www.google.com",
  4.                 "URL=http://www.google.com/",
  5.                 "Resource=0",
  6.                 "RecContentType=text/html",
  7.                 "Referer=",
  8.                 "Snapshot=t1.inf",
  9.                 "Mode=HTML",
  10.                 LAST);
  11.  
  12.         lr_think_time(6);
  13.  
  14.         web_url("search",
  15.                 "URL=http://www.google.co.uk/search?hl=en&source=hp&q=linux&btnG=Google+Search&meta=&aq=f&oq=",
  16.                 "Resource=0",
  17.                 "RecContentType=text/html",
  18.                 "Referer=http://www.google.co.uk/",
  19.                 "Snapshot=t2.inf",
  20.                 "Mode=HTML",
  21.                 LAST);
  22.  
  23.         return 0;
  24. }

What happens here?

  • In line 3 we are entering google.com web site
  • In line 14 we are sending HTTP request that browser would generate after searching for value “linux”

Now, lets look at “Web (Click and Script)” type of the script:

  1. Action()
  2. {
  3.  
  4.         web_browser("www.google.com",
  5.                 DESCRIPTION,
  6.                 ACTION,
  7.                 "Navigate=http://www.google.com/",
  8.                 LAST);
  9.  
  10.         web_edit_field("q",
  11.                 "Snapshot=t1.inf",
  12.                 DESCRIPTION,
  13.                 "Type=text",
  14.                 "Name=q",
  15.                 ACTION,
  16.                 "SetValue=linux",
  17.                 LAST);
  18.  
  19.         web_button("INPUT",
  20.                 "Snapshot=t2.inf",
  21.                 DESCRIPTION,
  22.                 "Type=submit",
  23.                 "Tag=INPUT",
  24.                 "ID=",
  25.                 "Value=Google Search",
  26.                 ACTION,
  27.                 "UserAction=Click",
  28.                 LAST);
  29.  
  30.         return 0;
  31. }
  • In line 4 we are entering google.com web site
  • In line 10 we are typing value “linux” into the input field
  • In line 19 we are clicking “Google Search” button that will move us to the page with search results

So what is the difference between these two scripts?

First script operates on much lover level comparing to second script. It deals with HTTP requests without taking care about what actions user actually performs. It doesn’t care how fast user’s browser renders and display the UI. It only checks how fast web server is able to response with correct message.

Second script operates only on UI level without taking care what happens underneath. Response time here includes not only time needed to send/receive HTTP traffic but also time needed to form/display UI to the user.

Basically second script approach is less error prone because you don’t have to deal with low level things like HTTP parameters. And You are replicating user’s actions in a natural way.

So if you need to choose, then I would recommend Web Click and Script type of the script.

LoadRunner ,

Average transaction response time vs Granularity

November 7th, 2009
Comments Off

Correct me if I’m from but I always thought that for particular transaction there should be only ONE average response time. But it looks like it’s not the case in LR Analysis.

I recently discovered that by changing granularity on average transaction response time graph, Analysis also recalculates average times under the graph. In my understanding granularity is responsible only for making the graph easier to read and thats all it should do. But by changing granularity we are also changing how many points LR actually counts which results in different average response times for different granularity.

So shame that this is not mentioned in the manual. Of course according to HP support it is mentioned there and this is a feature, not a bug :)

So my tip for today: be careful when looking at response times on average transaction response graph. Use “Summary” page if you want to omit any mistakes.

LoadRunner , ,

Exception ACCESS_VIOLATION

November 7th, 2009
Comments Off

Web (HTTP/HTML) scripts in LoadRunner are implemented using C programming language. And like always with C, you should remember about some basics. One of them is that few string handling functions can return NULL value instead of correct pointer which will definitely lead to exception like the one below:

  1. Action.c(7): Error: C interpreter run time error: Action.c (7):  Error — memory violation : Exception ACCESS_VIOLATION received.

Basically if you see message like this, it is not any internal LoadRunner error. It means that you made a mistake in your script and you need to fix it. But let’s start from the beginning with some example:

  1. Action()
  2. {
  3.         char * x_p;
  4.        
  5.         lr_save_string("hello_world!", "MESSAGE");
  6.         x_p = (char *)strchr(lr_eval_string("{MESSAGE}"), ‘ ‘);
  7.         lr_save_string(x_p, "MESSAGE_FROM_SP");
  8.         lr_output_message(lr_eval_string("{MESSAGE_FROM_SP}"));
  9.        
  10.         return 0;
  11. }

This small piece of code takes parameter MESSAGE with value “hello_world”, then search for the first space character and display the string starting from that place up to the end. Function strchr() is responsible for searches for the space character. If it’s found then strchr() will return a valid pointer (something like 0xb36ac56e). But if the space is not there (which is our case since there is no space in the hello message), strchr() will return NULL which refers to 0×0 memory address location.

Such memory address is a very special address. Basically any read attempt from there is treated as incorrect operation and results in memory access violation (not only in LoadRunner).

Now, howto deal with it? If you see ACCESS_VIOLATION, in most cases it means that your LR script is working on incorrect/incomplete values. You are responsible for error handling in your scripts and you should always:

  • validate parameters values
  • check results of C functions to handle any errors, unexpected conditions
  • remember that you can’t always expect correct values and you need to handle it as well

Here is our example with fix showing howto deal with ACCESS_VIOLATION. We are calling strchr() function and checking if value returned is not NULL.

  1. Action()
  2. {
  3.         char * x_p;
  4.        
  5.         lr_save_string("hello_world!", "MESSAGE");
  6.         x_p = (char *)strchr(lr_eval_string("{MESSAGE}"), ‘ ‘);
  7.        
  8.         if(x_p) // if the pointer is not NULL display correct message
  9.         {
  10.                 lr_save_string(x_p, "MESSAGE_FROM_SP");
  11.                 lr_output_message(lr_eval_string("{MESSAGE_FROM_SP}"));
  12.         }
  13.         else //if pointer is NULL display error message
  14.         {
  15.                 lr_error_message("Space not found in MESSAGE parameter");
  16.         }
  17.         return 0;
  18. }

More details about NULL pointer here http://en.wikipedia.org/wiki/Pointer_(computer_programming)#Null_pointer

LoadRunner , ,

LoadRunner + QTP = End-to-End performance

October 2nd, 2009
Comments Off

The easiest way to measure web app performance is just to record a script and run it in HP Controller. Such test is fast and simple, but it’s also incomplete and incorrect. It’s because LoadRunner is not the tool for performance measurements. It’s a tool only for generating the load on the server (Load Runner – tool for “Running” the “Load”).

So how to deal with that?

LoadRunner finishes taking measurements when web page code and all non-HTML elements (jpg,gif,js) are download into the clients memory. But starting from that point the browser need to form UI and display it to the user. And that’s the part that LR is actually skipping. Full end-to-end test should measure time between clicking “submit” button (or whatever triggers a call) and displaying full page within the browser. So how to do it in LR? The quickest answer is “not possible”. For that you need to use QuickTest Professional.

In that case of course you need at least two scripts. One prepared in LoadRunner and second prepared in QTP. Each script will have different task to accomplish. LR script will generate the load on the server simulating different amount of remote users connecting with AUT. QTP script will measure times only for one user, and that’s what we actually need. I believe every single web user care only about how fast page works in his browser. Not how fast it works across all remote users together in average. Of course one is connected with the another.

LoadRunner and QTP have one thing in common – transactions. And basically to measure end-to-end time in QTP all you need to do is to use transactions and remember to synchronize each page correctly. By synchronize I mean to finish transaction when page is fully downloaded and displayed to the user. One solutions for checking when downloading is finished is to call WaitProperty on browser’s status bar element. Example QTP code:

  1. Services.StartTransaction "SUBMIT_PAGE"
  2. Browser("IE").Page("Add_User").Image("Submit").Click 50,16
  3. Browser("IE").WinObject("Status_Bar").WaitProperty "visible", FALSE
  4. Services.EndTransaction "SUBMIT_PAGE"

If you have LR and QTP scripts already in place, now just add them in HP Controller into your scenario. To see QTP scripts in Browse window just change File Type from “Vuser Scripts” into “QuickTest scripts”. One think to remember is that you are allowed to run only single QTP instance at a time so hacks like few QTP scripts running simultaneously are not possible (unless you are using Citrix which is another story).

Only with such scenario you are able to measure full end-to-end performance.

LoadRunner

Web Services testing in LoadRunner

July 24th, 2009
Comments Off

Some time ago I described how to test web services in LoadRunner with HTTP/HTML script. Right now I would like to describe the correct way – testing with Web Services script.

First of all we need a web services. And there is one available exactly for training. Here is the WSDL http://soatest.parasoft.com/store-01.wsdl. I hope that folks from Parasoft don’t mind we are not using SOATest :)

In any case, we have an WSDL file. Now lets create new script.

Click File / New and select “Web Services” from list of available scripts types.

1

Now, when we have new script we should see new toolbar under the standard one. It allows to add Web Services description to the script (from WSDL file), add XML request using form and add XML request from file. So lets click on “Manage Services” button and then “Import”. Enter WSDL url http://soatest.parasoft.com/store-01.wsdl and click “Import”.

2

After WSDL file is imported, just click “Apply” and “OK”. From this point LoadRunner has description of our web services so we can use it send some requests. We will actually create two requests. One using “Add Service Call” and second using “Import SOAP” buttons from toolbar.

Click on “Add Service Call”. In “Operation” dropdown list select value “getItemById”. On left side select “id” under Input Arguments tree node. Then on right side type “1″ into Value editbox.

3

Now our script should look like this:

  1. Action()
  2. {
  3.    web_service_call( "StepName=getItemById_101",
  4.        "SOAPMethod=Cart|ICart|getItemById",
  5.        "ResponseParam=response",
  6.        "Service=Cart",
  7.        "ExpectedResponse=SoapResult",
  8.        "Snapshot=t1248415874.inf",
  9.        BEGIN_ARGUMENTS,
  10.        "id=1",
  11.        END_ARGUMENTS,
  12.        BEGIN_RESULT,
  13.        END_RESULT,
  14.        LAST);
  15.    return 0;
  16. }

Now lets add Web Service request using Import SOAP. Lets assume we have XML request saved in file on the disk. Here is an example:

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  3.  <SOAP-ENV:Body>
  4.   <getItemByTitle xmlns="http://www.parasoft.com/wsdl/store-01/">
  5.    <titleKeyword>Linux</titleKeyword>
  6.   </getItemByTitle>
  7.  </SOAP-ENV:Body>
  8. </SOAP-ENV:Envelope>

Click “Import SOAP” button and select your file. Change type from “Web Service Call (Recommended)” to “SOAP Request”. Select URL from the list, and type into SOAPAction this value “getItemByTitle”. Click OK.
Now we’ve added second call that ask for book details for title “Linux”. Our script should look like this:

  1. Action()
  2. {
  3.    web_service_call( "StepName=getItemById_101",
  4.        "SOAPMethod=Cart|ICart|getItemById",
  5.        "ResponseParam=response",
  6.        "Service=Cart",
  7.        "ExpectedResponse=SoapResult",
  8.        "Snapshot=t1248415874.inf",
  9.        BEGIN_ARGUMENTS,
  10.         "id=1",
  11.         END_ARGUMENTS,
  12.         BEGIN_RESULT,
  13.         END_RESULT,
  14.         LAST);
  15.    soap_request("StepName=SOAP Request",
  16.        "URL=http://ws1.parasoft.com/glue/store-01",
  17.        "SOAPEnvelope="
  18.        "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
  19.        "<SOAP-ENV:Envelope xmlns:SOAP-ENV=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">"
  20.        "<SOAP-ENV:Body>"
  21.        "<getItemByTitle xmlns=\"http://www.parasoft.com/wsdl/store-01/\">"
  22.        "<titleKeyword>Linux</titleKeyword>"
  23.        "</getItemByTitle>"
  24.        "</SOAP-ENV:Body>"
  25.        "</SOAP-ENV:Envelope>",
  26.        "SOAPAction=getItemByTitle",
  27.        "ResponseParam=response",
  28.        "Snapshot=t1248416271.inf",
  29.        LAST);
  30.    return 0;
  31. }

As you can see, each request contain “ResponseParam=response”. LoadRunner will automatically save response XML into parameter with name “response”. We can easily display this parameter by addind

  1. lr_message(lr_eval_string("Response XML is \n{response}"));

after each call. So at the end out script should look like this:

  1. Action()
  2. {
  3.    web_service_call( "StepName=getItemById_101",
  4.        "SOAPMethod=Cart|ICart|getItemById",
  5.        "ResponseParam=response",
  6.        "Service=Cart",
  7.        "ExpectedResponse=SoapResult",
  8.        "Snapshot=t1248415874.inf",
  9.        BEGIN_ARGUMENTS,
  10.         "id=1",
  11.         END_ARGUMENTS,
  12.         BEGIN_RESULT,
  13.         END_RESULT,
  14.         LAST);
  15.  
  16.    lr_message(lr_eval_string("Response XML is \n{response}"));
  17.  
  18.    soap_request("StepName=SOAP Request",
  19.        "URL=http://ws1.parasoft.com/glue/store-01",
  20.        "SOAPEnvelope="
  21.        "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
  22.        "<SOAP-ENV:Envelope xmlns:SOAP-ENV=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">"
  23.        "<SOAP-ENV:Body>"
  24.        "<getItemByTitle xmlns=\"http://www.parasoft.com/wsdl/store-01/\">"
  25.        "<titleKeyword>Linux</titleKeyword>"
  26.        "</getItemByTitle>"
  27.        "</SOAP-ENV:Body>"
  28.        "</SOAP-ENV:Envelope>",
  29.        "SOAPAction=getItemByTitle",
  30.        "ResponseParam=response",
  31.        "Snapshot=t1248416271.inf",
  32.        LAST);
  33.  
  34.    lr_message(lr_eval_string("Response XML is \n{response}"));
  35.  
  36.    return 0;
  37. }

At the end lets run our script. Output should be something like this:

4

LoadRunner

Validating Web Service response with XPath

May 18th, 2009
Comments Off

The easiest way for checking web service response in HP LoadRunner is by using XPath query language. LR API contains few functions designed especially for dealing with XML.

  1. lr_xml_get_values()  //Retrieves values of XML elements found by a query
  2. lr_xml_set_values()  //Sets the values of XML elements found by a query
  3. lr_xml_extract()  //Extracts XML string fragments from an XML string
  4. lr_xml_delete()  //Deletes fragments from an XML string
  5. lr_xml_replace()  //Replaces fragments of an XML string
  6. lr_xml_insert()  //Inserts a new XML fragment into an XML string
  7. lr_xml_find()  //Verifies that XML values are returned by a query
  8. lr_xml_transform()  //Applies Extensible Stylesheet Language (XSL) Transformation to XML data

Now, lets say we have sample web service for on-line book store and we want to ask what is the author for book id 123. Our web service can send following XML as a response:

  1. <books>
  2.   <book>
  3.       <id>123</id>
  4.       <author>John Smith</author>
  5.       <title>Working with Legacy code</title>
  6.       <publisher>Microsoft</publisher>
  7.   </book>
  8. </books>

For checking if the “author” element within XML response contains “John Smith” value we will use lr_xml_get_values() function. Here is the code that calls web service and checks if the value is as expected:

  1. Action()
  2. {
  3.      web_add_header("SOAPAction", "\"CallMe\"");
  4.      lr_start_transaction("AUTHOR");
  5.      soap_request("StepName=Sample Soap Request",
  6.        "ExpectedResponse=ANY",
  7.        "URL=http://foo.com/api",
  8.        "SOAPEnvelope= "
  9.        "<?xml version=\"1.0\" encoding=\"utf-8\"?>"
  10.        "<soap:Envelope "
  11.        "xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\" >"
  12.        "<soap:Body soap:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\">"
  13.        "<GetBookDetails>"
  14.        "<idValue>123</idValue>"
  15.        "</GetBookDetails>"
  16.        "</soap:Body>"
  17.        "</soap:Envelope>",
  18.        "Snapshot=t765765765.inf",
  19.        "ResponseParam=Response_Xml",LAST);
  20.  
  21.  
  22.      lr_xml_get_values("XML={Response_Xml}",
  23.           "ValueParam=Author_Name",
  24.           "Query=/books/book/author",
  25.           LAST);
  26.  
  27.      lr_output_message(lr_eval_string("Author is = {Author_Name}"));
  28.  
  29.      if(strcmp(lr_eval_string("{Author_Name}"),"John Smith") == 0)
  30.      {
  31.         lr_end_transaction("AUTHOR", LR_PASS);
  32.      }
  33.      else
  34.      {
  35.         lr_end_transaction("AUTHOR", LR_FAIL);
  36.      }
  37.      return 0;
  38. }

First argument in lr_xml_get_values() call is parameter name that holds response XML. Second argument in name of new parameter that will hold author value extracted from response XML. Third argument is XPath query that extracts author element value.

LoadRunner , , ,