Testing Socket.IO and WebSockets on the Grid

Socket.IO is a WebSocket like API that lets developers create things like realtime web apps and games. You can learn more about the protocol here which was essential in understanding how we might simulate load for this on the Grid.

Overview of the Socket.IO protocol

In short, a client will first determine what transport to use, common transports being WebSocket or XHR-polling amongst others. The client will then handshake with the server via a HTTP request at which point the server will respond with a session ID, heartbeat timeout and socket timeout. This session ID is then used in subsqeuent communication with the server via the chosen transport.

XHR-Polling in JMeter

It is possible to test this process via JMeter using XHR-polling as the transport. A test plan would look something like this:

socket.io-jmeter

The handshake would be a POST to a URL like this:

/socket.io/1?t=${__time(,)}

With a regular expression extractor for the session ID / token like this:

Screen Shot 2012-12-13 at 10.16.31 AM

The subsequent connect request would be a GET to a URL like this:

/socket.io/1/xhr-polling/${token}?t=${__time(,)}

And depending on your use case, you could send events as a raw post body like this:

5:::{"name":"my other event","args":[{"my":"data"}]}

All good so far. However whilst we’re exercising the functionality of Socket.IO (handshake, connect, event, disconnect) via XHR polling, we’re still stuck on the hack version of push technology.

WebSockets affords us full duplex communication between the client and server, perfect for those real time applications in which the browser supports it. Simulating this in JMeter is going to be a tad difficult, and certainly not possible via the HTTP Sampler.

WebSockets in Gatling

That’s where the power of tools like Gatling come into play. Gatling already offers us a much higher concurrency:machine ratio per Grid node, and it’s straightforward to extend (with a little help from the developers).

We’re advocates of choosing the right tool for the job, and in this case we felt it was more appropriate to use the Gatling-WebSocket plugin to achieve our client’s test outcomes.

We’ve already compiled and added the plugin to our Grid nodes, so all that’s left for you to do is create a custom test plan in Gatling. The test plan will look something like this:

import com.excilys.ebi.gatling.core.Predef._
import com.excilys.ebi.gatling.http.Predef._
import com.giltgroupe.util.gatling.websocket.Predef._
import akka.util.duration._
import bootstrap._

class TestPlan extends Simulation {
  def apply = {

    val threads   = Integer.getInteger("threads",  50)
    val rampup    = Integer.getInteger("rampup",   60).toLong
    val duration  = Integer.getInteger("duration", 60).toLong

    val host      = System.getProperty("host")
    val port      = Integer.getInteger("port", 80)
    val path      = System.getProperty("path")
    val testguid  = System.getProperty("testguid")

    val httpConf = httpConfig
      .baseURL("https://"+host+path)
      .responseInfoExtractor(
        response => Option(response.getHeader("Content-Length"))
        .map(List(_)).getOrElse(List("0")))

    val scn = scenario(testguid)
      .during(duration seconds) {
        exec(
            http("handshake")
            .get("/socket.io/1")
            .check(regex("(.+?):").saveAs("token")))  
        .exec(websocket("socket")
          .open("wss://"+host+"/socket.io/1/websocket/${token}", 
            "socket_open"))
        .exec(websocket("socket")
          .sendMessage("my data", "socket_send"))
        .pause(10)
        .exec(websocket("socket")
          .close("socket_close"))
      }

    List(scn.configure.users(threads).ramp(rampup)
      .protocolConfig(httpConf))
  }
}

Note we’ve added some variables to the script to make it easier to parametize and run on the Grid.

We’ve also got a mix of HTTP (for handshaking) and WebSocket (open, sendMessage, close) protocols in the script. We highly recommend using the cheat sheet for the Gatling DSL.

Now you can test some high concurrency WebSocket style communications from the Grid.

Screen Shot 2012-12-13 at 10.35.39 AM

Gatling is reserved for our paid/beta accounts only. Please get in touch with us if you’d like to know more.

4 thoughts on “Testing Socket.IO and WebSockets on the Grid

    • Thanks and good work on the plugin. I’ll keep track of that issue, in the interim, our client requirements have been fairly simple in terms of message handling. I’ll provide feedback as we get it from our clients.

      Thanks again.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s