error checking in loadrunner
There is no real point in running a script if you’re not going to check and handle for errors. In my opinion, this is what defines a good loadtester/automator from a fake one. I have seen many fake testers, who are palmed off into our industry as experts, but can only record and playback. But that’s a whole different gripe I have.
I’ve spent the last few weeks discussing this topic in detail with my manager, and senior mentor Nick Singh from JKVine and together we have come up with a ‘framework’ (buzz word) that is not too time consuming and has proved to be very successful.
There is a point where error collection/handling becomes too expensive, overkill and a waste of total effort. Especially in performance testing, where the scripts are not ‘regression’ type scripts and are thrown away after the project goes live, so it only makes sense to not pour blood and sweat into them.
In previous posts, I have spoken about content checks and their usefulness in error collection. After detailed investigation, I have discovered that is not a good form of collecting and handling errors. In fact, it is pretty nasty. In a nutshell, this method is good for global errors, however, it does not allow you to deal with the error, and creates a waterfall of errors that are presented in the controller during runtime, which are inaccurate and have the potential to create and display more problems than what is really happening. For this reason, I have decided to turn off content checking in my scripts and adopt the following method.
There is no doubt, that checking for text on the next displayed page is one of the best ways in determining if the action has succeeded or not. As long as the text is unique to the next page. My method is simple and effective and involves 3 steps of error checking and collection:
step 1 – a form of content checking
Before every submit, I perform a web_reg_find for each one of my content checks:
web_reg_find("Text=Template for message NUMBER_FORMAT_EXCEPTION", "SaveCount=invalid_correl_location",LAST); web_reg_find("Text=MimeType text/html is not valid.", "SaveCount=Invalid_Mime_Type",LAST); web_reg_find("Text=Lock End Date should be greater than Lock Start Date", "SaveCount=lock_end_date",LAST); web_submit_data("main.jsp", "Action=http://mishmashmoo.com/blog/", "Method=POST", ............................ ............................
step 2 – what is my next page?
Second step is to check for text that I am expecting on the next page. Obviously before the submit
web_reg_find("Text=technical blog of sameh abdelhamid","SaveCount=pageFound",LAST); web_submit_data("main.jsp", "Action=http://mishmashmoo.com/blog/", "Method=POST", ............................ ............................
step 3 – handle and report the errors
Firstly I need to handle the web_reg_find ‘content checks’. For these errors, I do not want to fail the script, I just want to send a message to controller that “this text was found on the screen”. The reason I don’t want to fail the script here is because the error displayed may be a valid error and the application might be in a state where it is able to continue.
This code goes after the submit.
if (atoi(lr_eval_string("{invalid_correl_location}")) >=1) { lr_error_message("%s","invalid_correl_location"); } if (atoi(lr_eval_string("{Invalid_Mime_Type}")) >=1) { lr_error_message("%s","Invalid_Mime_Type"); } if (atoi(lr_eval_string("{lock_end_date}")) >=1) { lr_error_message("%s","lock_end_date"); } if (atoi(lr_eval_string("{page_not_found}")) >=1) { lr_error_message("%s","page_not_found"); }
Then I need to collect and handle the error if the page was not found. If the next page is not displayed, a serious problem has occurred, and I defiantly do not want to continue running the rest of the actions in the script if this happens. It will result in sending allot of invalid requests to the server, and allot of errors will come back, and the script may result in bringing several systems down by sending this junk across the network.
To simulate real user action, which is our goal, the user would normally quit the session, and start a new one when a serious error occurs. So this is what we will be doing. To do this I call the vuser_end action, which logs the user off, then I call the vuser_init action which logs the user back in, then I exit the current iteration and continue after the vuser_init, which is where the vuser is waiting. I also send a fail to the controller, and report the error “Could not Find Next Page Successfully – Ending This Iteration”.
if (strcmp(lr_eval_string(pageFound), "0") ==0) { lr_error_message("%s","Could not Find Next Page Ending This Iteration"); vuser_end(); vuser_init(); lr_exit(LR_EXIT_ITERATION_AND_CONTINUE,LR_FAIL); }
I place this code before and after every submit. If the ‘content check’ is successful it will report that this error was found, this is useful information once you analyse your results and report a defect. Together they provide a great error handling and reporting system which is not time consuming or expensive. The only downfall is that your scripts become very messy and long, however, to overcome this, it only makes sense to functionalise such actions, which is what I have done in my scripts. For more info, read my post on creating loadrunner functions.
Enjoy!
Good stuff Sammy.
Another couple of things we do to bolster this process is encapsulated in the function below (included in a global header)
We use lr_set_transaction_status(LR_FAIL) to fail the transaction, otherwise it ends up as a ‘STOP’ in your summary data. We also log a lr_user_data_point so we can view when these conditions were hit in their own graph / raw data.
Our vuser_init_recovery() method has the following
It’s sometimes important to have the additional cleanup functions, especially cookies if your vuser_end doesn’t end a user session properly. For example, many web apps I test don’t have a logout. For this reason, it is important to destroy user context so you start with a fresh user session in your vuser_init recovery process.
I’m not a big frameworks person, but I do agree, stuff like this is important to make scripts more robust, especially with soak test scenarios. Keep up the good work!
Regards,
Koops
http://90kts.com
Excellent Sammy..
Can fully understand where you are coming from with the cumbersome global Content Checks
On previous projects I have used a similar structure but have usually included a HTTP check when an error has been found and determined the action based on that as well..
For example… HTTP error 500.. exit and re-initialse..
HTTP 400 or the like.. retry the page first, if same result then exit and re-initialse..
This is really good stuff for me.
I am new to this field and i was looking for similar kind of function.
Thanks a Lot!!!
A good article and i fully agree about the problems that content check involves but I just wanted to question how i would go about capturing a snapshot on ERROR with the method you have described for your web_reg_find in Step 2?
Since I find capturing a screenshot is a very useful facility to take back to application developers but yet with this method the snapshot is not caught since the web_reg_find is not in an error status and introducing an lr_error_message following does not initate a snapshot.
Also I forget to mention something else here where you probably will get a problem that will spoil your error handling. If you are correlating, as I expect you should be, you will possibly have several web_reg_save_params before some of your steps.
How do you suggest to handle the errors thrown by these statements if it cannot find these?
A simple way would be to use
web_reg_save_param(”myParam”,”LB=someBoundary”,”RB=anotherBoundary”,”NotFound=warning”, LAST);
instead and then make a decision of the myParam_Count==0
Which works no problem. But how do you suggest to create this into a function since param names will be dynamic. Any suggestions??
@Martin
You pose an intersting question Martin, and some valid points.
To put it simply, the snapshot on error will not work this this function of error capturing. One would assume that the reporting of “lr_error_message” would activate the snapshot creation in the controller, but it does not. There is no code that will work either to take a snapshot, and a third party application wont work, due to the fact that there is no ‘gui’ when your’e running.
In saying that what you could do, is create a function that grabs the entire body of the screen and saves it in a parameter, turn on paramater substituion and where trail the logs to find where the error occurs. By grabbing the html body of the page, you could save that as a HTML and view the page…Not efficent, but it would work.
Your second query was right on the ball also. And after writing this argicle, I noticed this problem, and made some adjustments accordingly.
I thought I updated my post but It seems I did not, maybe I’m getting old and losing some of my memory.
Adding “notfound=warning”, allows the function to execute, becuase, without this, the error checking function never gets called (as the script executes when it fails). Good point you raised. I would always reccomend that web_reg_find are saved with this option, becuase if the web_reg_find fails, you will get a waterfall of errors after this, which are not valid as they are from the same source.
Cheers.
Thanks for your response Sammy.
It is annoying that the application will not initate a snapshot on error when lr_error_message but i wouldnt recommend saving each page as a parameter, if you do then i would throughly recommend that you prototype your injectors resources to ensure they are capable of coping with this additional overhead.
I do have an alternative solution though an that is to introduce an error to capture it after the call has been made..i.e into the vuser_pagecheck function that Tim has detailed above. .. A simple one would be to use a web_find as the although superceeded by web_reg_find, this function still very much works and is present in the latest version of LR. So for example:
vuser_pagecheck(char *PageCheckValue)
{
if (atoi(lr_eval_string(”{PageCheck}”)) < 1)
{
lr_set_transaction_status(LR_FAIL);
lr_error_message(”Unable to find %s”, PageCheckValue);
lr_user_data_point(PageCheckValue, 1);
LR_web_find(”Intoduced Failure”,”What=You should not find this!”,LAST);
}
}
However as you can see due to the Failure, the vuser_init_recovery from Tims example would not be able to run as loadrunner will terminate the script (iteration) at the point of failing the lr_web_find. However i dont see this to be a major issue since those commands in the recovery can be placed at the start of your main action. I suppose its very much based on the AUT.
Hi there, thanks for the informative post.
Do you know of a way to “execute code on fail”? I have a proper logout() function and I want that to run if I run into a problem in the middle of the script, without having to paste a code block similar to yours before/after every request.
@James
Hi James,
No, there is no way to do that. You have to tell your code what to do under what conditions. The only way to do that is to call a function, and/or put code before and after a submit.