As stated earlier, XML or SOAP over HTTP/S makes up most of the messages passed around through an ESB today. REST use cases are increasingly becoming a considerable chunk as well. Most ESBs supports non-XML payloads such as Hessian, Text, HTML, binary etc as well as Fast Infoset support and Gzip compression of the output over HTTP/S as well. Typically ESBs support HTTP/S, JMS, Email (POP3/IMAP/SMTP), File, S/FTP, FTP, FTP/S, MLLP/S, TCP/S, FIX, Database etc and many others.

Some of the ESB’s that have been based over existing EAI or Messaging technologies sometimes may convert each message into a JMS or other messaging system based message, irrespective of the transport. This may give good performance for JMS or Messaging Oriented Middleware [MOM] system based use cases, while messages processed over other transports may suffer undesired overheads.

The performance of JMS or MOM infrastructure depends heavily on the form of persistence provided by the underlying technology. Thus, a vendor using in-memory persistence of messages may easily show better performance over another that used Database or File based persistence for recovery. Typically most ESBs available free of charge or as open source ESB’s - works with many JMS or MOM systems - that maybe usually be hosted externally, outside of the ESB - similarly to how a typical web application may use an external Database. Thus the performance of JMS or MOM transports will be based on the underlying JMS/MOM frameworks used - and will not convey the performance of the ESB itself - for handling the messaging load.

The performance test uses messages of size:

With the concurrent number of users:

Sample request message

This scenario will demonstrate the ability of the selected ESB to act as a virtualization layer for backend web services - similarly to how the Apache Web Servers or Hardware Load balancers were used in front of web backends to virtualize and balance load and provide failover capabilities. Although this scenario does not include load balancing, fail-over, security verification or schema validation etc, most of the ESBs does support these features. This scenario assumes the service to be a SOAP Web Service, and thus expects the ESB to present the WSDL of the service to its clients.

Requirements:

Typically this pattern is used in practice to route request messages to different endpoints, or handle them differently at the ESB based on the message attributes - such as the payload, or transport headers etc. This scenario is now presented in three variants:

CBR on Transport Header

The first scenario evaluates the most optimal way to route messages, based on HTTP transport level headers. This technique is widely used when routing Hessian or other non-SOAP payloads, as is extremely efficient. The condition performs a simple String string comparison over a HTTP header.

CBR on SOAP Header

This scenario performs a routing decision on a SOAP header, as it could be better optimized than processing a large SOAP payload body. The condition performs an XPath evaluation over a SOAP header. This test scenario extracts a SOAP header specified as a simple String, and then performs a comparison against a given constant string.

CBR on SOAP Body

This scenario will demonstrate the ability of the selected ESB to route an XML payload based on the evaluation of an XPath expression over the payload body. The condition performs an XPath evaluation over a SOAP body. This test scenario expects the first order elements’ symbol to be equal to "IBM" to forward the request to the backend service. On a failure, its expected to return a SOAP fault to the client with an error message.

Requirements:

This scenario will demonstrate the ability of the selected ESB to transform XML payload messages (such as SOAP) between different schemas. In practice transformations are used to convert messages from XML to CSV or Text etc, or to change a request adhering to one version of a schema to a different version (for backwards compatibility of services/clients), or to create messages that are enriched with some attributes from the original request, and other attributes picked up during mediation of the message - for example from a Database. Typically ESB’s offer multiple options for transformations - including XSL, XQuery, Scripting language, Milyn etc. In future this scenario maybe extended with other scenarios.

This test expects the transformation to convert the original message to the format shown below - by reversing the element names. Thus, the Echo service backend will echo back this reverse message, which is transformed again to the original format and sent to the client.

Requirements:

This scenario will demonstrate the ability of the selected ESB to act as a Security Gateway. Typically an ESB is used to validate and verify security credentials, and establish the authenticity of messages, before passing them through mediation and to internal services within a corporate environment. Although this test case considers WS-Security, many SOA deployments rely on SSL, HTTP Basic and Digest authentication and other mechanisms as well.

This test expects the client to send a timestamped, digitally signed and encrypted request, and verifies the message and forwards the actual payload without the WS-Security header to the backend service. Once the Echo service responds back with the same message, the ESB timestamps, digitally signs and encrypts the response and sends it back to the client.

The test suite has bundled a set of requests that wrap the original requests with a timestamp valid for 10 years. These requests are found with file names of the form "<size>_buyStocks_secure.xml". Thus, the Apache Bench style load generator can very efficiently push a load of secured messages for a valid performance test of the ESB. However, it was noted that the Mule ESB did not accept the 10 year valid messages due to the older creation date, and hence the current framework includes a utility to regenerate the secured requests before an execution run against Mule or other such ESBs that will not accept the pre-secured messages.

The messages are secured using the standard WS-Security interoperability test suite keystore, and are signed as the user "alice" and encrypted for user "bob", and the passwords for the keystore as well as "alice" and "bob" is "password" - as per the standard WS-Security interoperability test cases.

Requirements: