QA Testing Video Chat App – GSoC Week 4

I would recommend you to read my last post first before reading this one. This should help you understand the content of this post better 🙂

Last week, I did both dev testing and load testing of video chat app. In this post, I am gonna show you the results I got.

Bugs & Fixes

Bugs I found during testing:-

  • Re-rendering video object: When a user moves from video chat app to an irc channel/server and then back to video chat again, video stream(both local and remote) doesn’t get re-render. Actually,  this was a browser specific issue and I found it’s fix by chance. Have a look at the fix.
  • Click event on ‘Accept/Reject’ button gets triggered more than once: This one was simple to fix. I just had to use jQuery off method to remove click event attached to element before attaching new click event. Fixed.
  • Option to enable/disable video chat: I know this isn’t a bug. My mentor told me to add this feature.

Load Testing

Framework

In my last post, I mentioned I used EasyRTC framework to build p2p(peer to peer) video chat. Although, it’s a p2p app, it still requires server for signaling(read more). For signaling, EasyRTC uses socket.io built on node.js. So, basically I load tested EasyRTC server’s socket.io implementation.

I used socket.io-client module to interact/connect with EasyRTC server by writing against their server side API, as documented here. Below is the snippet of simple client I created.

var Client = function () {
  /* Some code omitted */
  this.connectToServer = function (callback) {
    var client = io.connect(
        'http://'+SERVER_HOST+':'+SERVER_PORT ,
        {'force new connection': true}
    );
    if (!client) {
      throw "Couldn't connect to socket server";
    }

    var msg = {
      msgType: 'authenticate',
      msgData: {
        apiVersion: "1.0.11",
        applicationName: "waartaa"
      }
    };

    var easyrtcAuthCB = function (msg) {
      if (msg.msgType == 'error') {
        callback('easyrtc authentication falied');
      } else {
        // code omitted
      }
    };

    client.json.emit('easyrtcAuth', msg, easyrtcAuthCB);
  };
}

Script & Parameters

I made a command line tool in nodejs to ran tests.

node run.js --no-of-connection=700 --no-of-concurrent-connections=[1,5,10]

In above mentioned command you should notice two things, namely, no. of connections and concurrent connection. My system(Intel(R) Core(TM) i5 CPU M 460 @2.53GHz, 4 GB RAM) couldn’t stand more than 700 connections. Server process started consuming 1 GB RAM for more than 700 connections. I will tell you the reason for this later. For now, 700 was suffice for initial testing. My mentor told me to keep concurrent connections very low because we won’t be expecting too high concurrency in production initially.

Data collected

After the script was ran several times, I collected the following data each time for different concurrency against no. of connections:-

  • Roundtrip time(ms): Time taken for client to get connected to server. This includes latency(which is negligible because client and server are on same machine) + client-to-server message time + server-to-client message time.
  • Avg. CPU load(%) & Memory (MB): I used this node module to collect these data.

Below you can see the data plotted to graph and their analysis:-

Roundtrip time analysis roundtrip_time

  • As no. of connections increases, round-trip time increases. This is because the default behavior of EasyRTC is to send every client in a room a separate update whenever a new client leaves a room, enters a room, or changes its status. And this leads to increase in round-trip time.
  • Increase in concurrency too increases round-trip time. I guess this is because nodejs being a single threaded server processes one request at a time. Please correct me if I am wrong.
  • Roundtrip time < 500 ms is a good no. For concurrency = 1 and any no. of connections in above graph, round-trip time < 500 ms.

CPU analysis cpu

  • Nothing too unnatural about cpu load increasing with no. of connections and concurrency.
  • Good thing is the rise is not too steep for lower concurrency.

Memory analysis memory

  • As I mentioned earlier, 700 connections consumed 1 GB memory and you can clearly see that in above graph. Well, this is because as I said earlier, the default behavior of EasyRTC is to send every client in a room a separate update whenever a new client leaves a room, enters a room, or changes its status. That means for 700 connections (700*701)/2 = 245350 messages were sent by server. You can do the math now 🙂
  • For any concurrency, memory rises at the same rate with increase in no. of connections. Thus, memory consumption is independent of concurrency.

Conclusion

  • More bugs will pop up when my mentors and other people start using the app until then a developer can only believe his/her app works perfectly 😉
  • The default behavior of EasyRTC server can cause scaling issues. We might have to change it in future by adding custom event listeners on server side.

PS: If you have made this far and found anything wrong in this post, please do comment. 🙂

<script>JS I love you</script>

p2p Video Chat – GSoC Week 3

After finished implementing chat log search last week, I moved on to my next major task – implementing p2p video chat in waartaa.

Introduction

A p2p(peer to peer) plugin-free video/audio Real Time Communication(short for RTC) had always been a nightmare for developers to implement. This was changed few years back with the introduction of WebRTC – an open source, plugin-free, built into browsers technology. Today, there are many web services out there which uses webRTC eg. Bistri.

Implementation

A WebRTC application needs to do several things:

  • Get streaming audio, video or other data.
  • Get network information such as IP addresses and ports, and exchange this with other WebRTC clients (known as peers) to enable connection, even through NATs and firewalls.
  • Coordinate signaling communication to report errors and initiate or close sessions.
  • Exchange information about media and client capability, such as resolution and codecs.
  • Communicate streaming audio, video or data.

There are many libraries/frameworks available which does that for you. I am using EasyRTC because it is open source, maintains an active repo on GitHub and uses a node server for handling signaling.

Our initial goal was to use irc nicks for communication between peers but we had to settle using waartaa’s username for now because of trust/authentication issues with irc nicks. Imagine user has to first register his irc nick before starting video chat, that would have consumed alot time of user.

Screenshots

I have tried to tightlty integrate video chat UI with current UI so that user can do both video chat and normal chat(in an IRC channel) in same browser window simultaneously.

Client A
Client A1
Client A2
Client B1
Client B2

Conclusion

I am not sure if this is production ready yet because I haven’t done much research on how much load can easyrtc server handle. Also, by default easyrtc uses a public stun server of google to handle NAT and firewall between peers and that doesn’t sound good to me. It is quite possible that we might have to setup our own stun server.

PR: https://github.com/waartaa/waartaa/pull/119

<script>JS I love you</script>