$ echo $JAVA_HOME
/opt/jdk
$ echo $PATH
/opt/jdk/bin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/gamesExecution - Executing Locally
Table of Contents
Execution - Executing Locally
The ESB Performance Test Suite is developed so that by default the Load generation client, the ESB under test and the backend service to which requests are being forwarded are all on the same system, talking over the local network interface. Thus the test suite does not require network isolation. The test suite is shipped as part of the UltraESB download (~40MB) and can be even used to compare any other ESBs in-addition to the UltraESB.
Where network isolation is available, or when it can be assumed to hold true between two Amazon EC2 nodes etc, this test can easily be distributed across two or three machines, by manipulating the URLs used at the Local generation client, and ESB configuration files.
The Load generation client used is an ApacheBench (AB) style clone implemented in Java, based on the Apache HttpComponents/NIO library. The mock service used for load testing is based on an embedded Jetty server. The test suite includes scripts that invokes the load test client with a selected payload, and the specified number of concurrent users, and captures the result of the action, the time consumed, and any errors encountered etc. (i.e. the typical ApacheBench style output). This output can be then be easily converted into a CSV file for analysis with a spreadsheet application using another utility shipped with the framework.
Thus, at a high level, executing the test suite involves five simple steps as follows:
-
Setup the ESB/s under benchmark for the scenarios
-
Start the mock service
-
Start one ESB
-
Start the load test client
-
Process and analyze results
The above steps maybe repeated for the selected ESBs, and the results compared. Detailed instructions follows below.
1. Standalone Execution
|
It is strictly reccomended that the performance test be carried out on a Linux system for best results - irrespective of the ESBs under test. A Linux system can be easily optimized for performance [as shown below] and applications such as ESBs generally performs much better on Linux systems than over other supported Operating Systems.
|
-
Download the UltraESB distribution (~40MB) from the UltraESB download page and extract into a directory of your choice
-
Ensure that a Java Development Kit (JDK - NOT JRE) version 1.6.x is available in the system, and that the PATH and JAVA_HOME environment variables are set correctly against the JDK. The Sun JDK is recommended.
-
Download the "Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files 6" and set it up for the JDK. This is required for the WS-Security test cases. You will need to replace the two JAR files in your $JDK_HOME/jre/lib/security with those from the above archive.
e.g.
-
Tune the operating system for optimal performance as follows. These settings are generic settings to utilize the full port range, and to reduce the TCP fin timeout for better socket reuse, and to set a good limit on the number of file handles permitted to the process.
-
Edit /etc/sysctl.conf and append
net.ipv4.ip_local_port_range 1024 65535 net.ipv4.tcp_fin_timeout 30 fs.file-max 2097152 net.ipv4.tcp_tw_recycle 1 net.ipv4.tcp_tw_reuse 1 net.ipv4.tcp_syncookies 0 net.core.somaxconn 1024 -
Edit /etc/security/limits.conf and append
* soft nofile 32768 * hard nofile 65535 -
Edit /etc/pam.d/common-session and append
session required pam_limits.so -
Reboot the system for the settings to take effect
-
After rebooting, ensure the file limit is correctly set by checking the output of "ulimit -n" as follows
$ ulimit -n 32768
-
-
Start the Echo Service
Change to the bin directory, and execute "./toolbox.sh -echo" to start the echo service in the non-GUI mode
username@host:~/java/ultraesb-1.7.0/bin$ ./toolbox.sh -echo Starting AdroitLogic UltraESB ToolBox ... Using JAVA_HOME : /opt/jdk Using ULTRA_HOME: /home/username/java/ultraesb-1.7.0 Starting sample Echo server on port 9000... -
Start the configured ESB under test. It is suggested to allocate the same amount of RAM and Threads to each of the ESBs under test. A general reccomendation is 2GB of RAM and approximately 300 threads. When invoking a load test sample (with a sample number equal to or above 900) via the "-sample" option, the ESB will automatically start up with 2GB of memory instead of the default 1GB.
username@host:~/java/ultraesb-1.7.0/bin$ ./ultraesb.sh -sample 900 -
Start the load generation client / load test
Change to the samples/bin directory and first execute the "smoketest.sh" script passing the URL prefix for the selected ESB being tested. For example, "./smoketest.sh http://localhost:8280/service" to test the UltraESB
username@host:~/java/ultraesb-1.7.0/samples/bin$ ./smoketest.sh http://localhost:8280/service Executing in directory /home/username/java/ultraesb-1.5.0/lib/samples Warm-up... Running iteration 1 ....The smoke test will run a short test of each scenario and ensure that the test is setup correctly. Once the smoke test passes, execute the "./loadtest.sh" similarly passing the prefix URL as an argument, and direct the output into a file as follows
username@host:~/java/ultraesb-1.7.0/samples/bin$ ./loadtest.sh http://localhost:8280/service > test-output.txt -
Once the test completes, convert results saved into the above file into a CSV file for analysis with MS Excel or OpenOffice. Execute the following utility, and then open the resulting file (e.g. data.csv) from your analysis software.
username@host:~/java/ultraesb-1.7.0/samples/bin$ java -jar ab-to-csv.jar test-output.txt > test-results.csv