User Guide

Thank you for using the RealLoad product.

This User Guide gives you an overview of how to use RealLoad and also contains numerous links to relevant details.

The order of the chapters in this guide corresponds to the order of the menus in the Main Navigation Bar.

Projects Menu

After you have signed in to the RealLoad Portal, you will see a Main Navigation Bar whose first menu on the left is Projects. This is a file browser that displays all of your files. The view is divided into Projects and Resource Sets which contain all the files for executing your tests and the test results of the executed load tests. You can also store additional files in a Resource Set which contain e.g. instructions on how a test should be performed.

Among other things, you can:

• Create, clone and delete projects.
• Create, clone and delete resource sets (subdirectories of projects).
• Searching for files by file name and by content.
• Upload and download files, create new files, display the content of files, edit and delete files.

There is also a recycle bin from which you can restore deleted projects, resource sets and files.

In the Developer Tools dropdown menu, there is a wizard that supports converting a Selenium IDE Test to a RealLoad Test, and an example of developing a JUnit Test from scratch.

Tests Menu

The Test Menu shows all tests that you have defined. You can also filter the view according to Projects and Resource Sets, and sort the tests in different ways.

Tests can defined:

• By the HTTP Test Wizard (Web Surfing Sessions and HTTP API Tests)
• From exported Selenium IDE Tests (Execution of real Web Browser Tests)
• From imported or self-developed JUnit Tests (Testing any type of Network Protocol such as DNS, SMTP or UDP)

…and can be executed:

• As Load Test Job
• As Regression Test (Part of a Test Suite)
• As Synthetic Monitoring Job

Note that a test is something like a bracket that only contains references to the files that are required for the test execution. There a no files stored inside the test itself.

Each test has a base location from which it was defined (Project + Resource Set) and to where also the load test results are saved.

A test itself can reference its resource files in two ways:

• Resource Files that reside in the base location of the test (the test core files).
• Referenced Files that are located in other Projects / Resource Sets. These are typically libraries, plug-ins and input files which are used/shared by several, different tests.

Defining a Load Test Job

After you have clicked on a test on “Define Test Job”, the intermediate “Define Test Job” menu is displayed.

In contrast to a “test”, all referenced files of the test are copied into the “load test job”. For this reason there is also the option “Synchronize Files at Start of Job” which you should always have switched on. At “Additional Job Arguments” you can enter test-specific command line arguments that are transferred directly to the test program or test script. However, you usually do not need to enter anything.

After you have clicked on “Define Load Test Job” the test job is created and the view changes from the “Tests” menu to the Load Test Jobs menu.

Monitoring Menu

This menu is documented at Synthetic Monitoring.

Load Test Jobs Menu

In this menu all load test jobs are displayed with their states. The green point at the top right next to the title indicates that all measurement agents and cluster controllers which are set to ‘active’ can also be reached. The color of the point changes to yellow or red if one or more ‘active’ measuring agents and cluster controllers cannot be reached / are not available.

A load test job can have one of the following state:

• invalid: The job is damaged and can only be deleted.
• defined: The job is defined locally in the portal server, but has not yet been transmitted to a measuring agent or cluster controller. Jobs that are in this state can still be modified.
• submitted: The job was transmitted to a measuring agent or cluster controller, but is not yet ready to be started.
• ready to run: The job on the measuring agent or cluster controller is ready to start.
• start failed: The start of the job has failed.
• running: The job is currently running and can be monitored at real time.
• completed: A job that was previously running is now completed (done).

As soon as a job is in the “defined” state, it has a “local job Id”. If the job is then submitted to a measuring agent or cluster controller, the job has additionally a “remote job Id”.

From this menu you can also create Test Job Templates and edit Test Suites - which can be executed as Regression Tests.

Starting a Load Test Job

After you have clicked on “Start Test Job” in a load test job you can select/modify the Measuring Agent or Cluster of Measuring Agents on which the job will be executed, and you can configure the job settings.

Input Fields:

• Measuring Agent | Cluster: The Measuring Agent or Cluster of Measuring Agents on which the load test is executed.
• Number of Users: The number of simulated users that are started.
• Max. Test Duration: The maximum test duration.
• Max. Loops per User: The maximum number of sessions executed per user (the number of iterations of the test per user). Each time a new session of a user is started, the user’s session context is reset.
• Loop Iteration Delay: The delay time after a session of a user has ended until the next session of the user is started.
• Ramp Up Time: The length of time at the beginning of the test until all simulated users are started. Example: with 20 simulated users and a time of 10 seconds, a new user is started each 0.5 seconds.
• Additional Arguments: Additional values which are transferred on the command line when the test script is started. These arguments are test specific. For tests that were created with the “HTTP Test Wizard” you can specify for example the following values: “-tcpTimeout 8000 -sslTimeout 5000 -httpTimeout 60000” (TCP connect timeout / SSL handshake timeout / HTTP processing timeout) which are considered by the executed URL calls and override the default values.
• Debug Execution: This option effects that detailed information are written to the log file of the test. For example variable values which have been extracted from input files or from HTTP responses as well as variable values which are assigned at runtime. Only activate this option if you have problems with the execution of the test.
• Debug Measuring: Effects that the Data Collector of the Measuring Agent writes the JSON objects of the All Purpose Interface to its log file. This option can be enabled to debug self-developed tests that have been written from scratch.
• (User Input Fields): If the test contains User Input Fields you can enter their values here.

Normally you do not have to enter any “Additional Arguments” and leave “Debug Execution” and “Debug Measuring” switched off.

After clicking Start Test Job, the job is started on the Measuring Agent or Custer of Measuring Agents and is then in the state “Running”. Then click on “Monitor Jobs” and the view changes from the “Load Test Jobs” menu to the Real Time View menu.

Real Time View Menu

The real time view shows all currently running load test jobs including their measured values and errors.

You can also suspend a running job for a while and resume it later. However this has no effect on the “Max. Test Duration”.

After the job is completed you can click on “Analyze Result”. The view changes then to the Test Results menu.

Test Results Menu

The “Test Results” menu is a workspace into which you can load multiple test results from any type of test. You can also switch back and forth between the test results. All measured values and all measured errors are displayed. In addition, Percentile Statistics and Diagrams of Error Types are also displayed in this menu.

This menu enables you also to combine several test results into a so-called Load Curve - to determine the maximum number of users that a system such a Web server can handle (see next chapter).

The Summary Statistic of a test result contains some interesting values:

• ∅ Measured Samples per Second: These are the number of successful measured values per second, counted over the whole test (in other products also so-called as “hits per second”).
• Max. Concurrent Users: These are the maximum number of concurrent user that have been really reached during the test (which may differ from the “Number of Users” defined when starting the test).
• Max. Pending Samples: This is the maximum amount of requests to the server, for which no immediately response has been received, measured over the whole test (the maximum traffic jam of the requests).
• ∅ CPU Load on Measuring Agent: This is the average CPU load in percent on the measuring agent (load generator) or on the members of a measuring agent cluster - which was captured during the execution of the test. If this value is greater than 95% the test is invalid because the measuring agent was overloaded.

Since several thousand to several million response times can be measured in a very short time during a test, the successfully measured response times are summarized in the response time diagrams at 4-second intervals. For this reason, a minimum value, a average value and a maximum value is displayed for each 4-second interval in such diagrams.

However, this summarization is not performed for Percentile Statistics and for Measured Errors. Every single measured value is taken into account here.

Determining System Capacity - Load Curve

The maximum capacity of a system, such as the maximum number of users that a web server can handle, can be determined by a so-called Load Curve.

To obtain such a Load Curve, you must repeat the same load test several times, by increasing the number of users with each test. For example a test series with 10, 50, 100, 200, 400, 800, 1200 and 1600 users.

The easiest way to repeat a test is to Clone a Load Test Job. You can then enter a higher number of users when starting the cloned load test job.

A measured Load Curve looks like this, for example:

As you can see, the throughput of the server increases linearly up to 400 users - with the response times remaining more or less the same (Avg. Passed Session Time). Then, with 800, 1200 and 1600 users, only individual errors are measured at first, then also many errors, with the response times now increasing sharply.

This means that the server can serve up to 400 users without any problems.
But could you operate the server with 800 users if you accept longer response times? With 800 users, 745,306 URL calls were successfully measured, with only 50 errors occurring. To find it out, let’s compare the detailed response times of “Page 1” of 400 users with 800 users.

Response Times of “Page 1” at 400 Users:

Response Times of “Page 1” at 800 Users:

The 95% percentile value at 400 users is 224 milliseconds and increases to 1952 milliseconds at 800 users. Now you could say that it just takes longer. However, if you look at the red curve of the outliers, these are only one time a little bit more than 1 second at 400 users, but often more than 8 seconds at 800 users. Conclusion: The server cannot be operated with 800 users because it is then overloaded.

Now let’s do one last test with 600 users. Result:

The throughput of the server at 600 users is a little bit higher than at 400 users and also little bit higher than at 800 users. No errors were measured.

Response Times of “Page 1” at 600 Users:

The 95% percentile value at 600 users is 650 milliseconds, and there are only two outliers with a little bit more than one second.

Final Conclusion: The server can serve up to 600 Users, but no more.

Troubleshooting a Load Test

In rare cases it can happen that a test does not measure anything (neither measurement results nor errors). In this case you should either wait until the test is finished or stop it directly in the “Real Time View” menu.

Then you can then acquire the test log files in the “Load Test Jobs” menu and search for errors.

Network Components Menu

The Network Components menu allows you to manage Measuring Agents, Cluster Controllers and HTTP/S Remote Proxy Recorders. In addition, you can also start additional components that you only need for a short time as AWS Cloud Instances by spending RealLoad Cloud Credits.

Measuring Agents & Cluster Controllers

On the one hand, depending on your license (Price Plan), you can add and manage Measuring Agents and Cluster Controllers that you operate yourself. On the other hand, you can also launch additional Measuring Agents and Cluster Controllers using RealLoad Cloud Credits.

How to launch and operate Measuring Agents and Cluster Controllers using your own AWS account is documented at AWS Measuring Agents.

HTTP/S Remote Proxy Recorders

HTTP/S Remote Proxy Recorders can be used to record Web Surfing Sessions, which can then be post-processed with the HTTP Test Wizard and converted into a RealLoad Test.

The most important points here are:

• You must import the CA Root Certificate of the Proxy Recorder into the web browser with which you record the web surfing session (Firefox recommended).
• To control the Proxy Recorder in the RealLoad Portal, you need a second web browser from another manufacturer (Goggle Chrome recommended), or alternatively installation of the RealLoad Browser Extension.
• Optional: When recording, HTTPS/X509 client certificates are also supported to authenticate yourself to the corresponding web server(s) - if required. You can upload the client certificates directly in this menu. The client certificates are then automatically transferred to the proxy recorder and applied during recording.

The complete documentation is available at HTTP/S Remote Proxy Recorder.

Launched Cloud Instances (by Cloud Credits)

This menu shows (only) all AWS instances that were launched using RealLoad Cloud Credits. Here you can also terminate such instances early - whereby Cloud Credits for complete unused hours are refunded.

Wizards Menu

This menu only displays the wizards that can be accessed via direct navigation. In addition, the RealLoad product contains numerous other wizards that are accessible depending on the context.

HTTP Test Wizard

The functionalities of the HTTP Test Wizard (+ Plug-Ins and Published Plug-Ins) is documented at HTTP Test Wizard.

User Input Fields Wizard

“User Input Fields” are additional values that are entered when starting or defining a test execution and that are available as (global) variables during test execution.

With this wizard you can create or edit JSON files containing definitions of User Input Fields. By convention, the names of such files must always start with ‘InputFields_’ and end with ‘.json’.

A User Input Field contains the following attributes:

• GUI Label : The label of the User Input Field that is displayed in the user interface.
• Variable Name : The name of the global variable at test execution.
• Input Type : The datatype of the input value (String, Integer, Real or Boolean). Please note that the Input Type is only used to validate the input. During test execution, the value of the variable is always converted to a string.
• Default Value : The default value of the variable.

User Input Fields are supported by All Types of RealLoad Tests.

Tools Menu

This menu contains tools that are helpful for developing and debugging tests:

• Base64 Tool : Converts plain text to Base64 format or vice versa.
• DNS Tool : This allows you to resolve DNS hostnames through one or more specific DNS servers.
• HTTPS Server Certificate Tool : With this tool you can check the validity of HTTPS server certificates and also create your own (client) Trust Stores.
• JSON Editor : This allows you to create and edit JSON files, as well as display JSON data in a formatted view. Tip: You can also copy raw JSON data into the JSON Editor and then view it as formatted JSON data with a mouse click - without having to save the data to a file.
• PKCS12 Client Certificate Tool : With this tool you can view the contents of an SSL/X509 client certificate which is in PKCS12 format.
• Query String Tool : With this tool you can convert URL Query Strings to the unescaped format or vice versa. See Wikipedia - Query String
• Text Phrase Tool: Determines the most meaningful human-readable text phrases or keywords from any given URL with HTML/JSON/XML response content. This tool’s algorithm is also used by the HTTP Test Wizard to verify the content of HTTP responses.
• Unix Time Tool : Converts a Unix time to a formatted local time - or vice versa.
• WebSocket Tool : This allows you to test a WebSocket connection to a web server. The tool also supports SSL/X509 client certificates in PKCS12 format.
• XML Editor : This allows you to create and edit XML files.

User Account Menu

Submenus:

• Sign Out : This will sign out you out without asking. Any tests that may be running are not affected by this and will continue.
• To Welcome Page : Displays the welcome page analogously after sign up.
• Personal Data & Profile Settings : Here you can edit your personal data and also determine whether parts of it are publicly displayed in the portal.
• Team Members : Displays the list of all team members.
• Team Members Accounts : Management of team member accounts by the main user account. Team members have no access to this submenu.
• Mobile Companion : Displays a list of all mobile devices that have logged in to the portal using the Mobile Companion App. Furthermore, all push notifications that were sent to the mobile devices are displayed.
• API Authentication Tokens : Here you can create and manage authentication tokens that are used for authentication against the portal server API.
• Price Plan : Displays your current price plan and your access permissions to RealLoad resources.
• Manage Licenses : Displays your purchased licenses and allows you to add additional licenses and cloud credits.
• Cloud Credit Statement : Displays your Cloud Credit balance and a list of all Cloud Credit transactions.
• Set New Password : Here you can set a new password for your portal account.
• Delete Account : This will permanently delete your account in the portal - including deleting all data associated with your account.

The following submenus are only accessible depending on your configured “Personal Data & Profile Settings”:

• Public In-App Users : Displays a list of all portal users which are visible to other portal users. You can also contact them within the portal. This list is only shown if you are also a public portal user yourself.
• In-App Message Box : Here you can sent and display received messages to/from all portal users which are visible to other portal users.
• Chat with Signed-In Users : Opens the chat popup that allows chatting with other in the portal signed in users who have enabled this functionality in their profile settings.
• Received Contact Messages : If you have enabled “Publish My Profile as Technical Expert:” in your profile settings then all received contact messages will be displayed here.

Downloading RealLoad Components and RealLoad Libraries Java Doc

Clicking this icon will open a new browser window or tab to download.realload.com

Purchase of Licenses and Cloud Credits

Clicking this icon will open a new browser window or tab to shop.realload.com

Here you can Purchase Licenses and additional Cloud Credits.

If you have any questions, please email us at support@realload.com