Videochat

I like to play with technology, push it and use it on not very common scenarios, this project is just that, some technology(SSE) being used for something(Video Conference) that is better acomplished with another technology, in this case WebRTC.

There are many things that can be done better, but the purpose of this project, is not to have a perfect piece of code, but to have a working proof of concept.

Why Server Sent Events?

Compared with WebSockets, Server Sent Events apps are very easy to build with Asp.Net MVC, and with the help of Needletail.MVC, which is a library that helps you to build SSE apps without breaking your MVC pattern, it's even easier.

The main limitations of using SSE for this kind of application are:

• SSE is not peer to peer, neither bi-directional, nor full duplex, so it's slower.
• The ammount of data received and sent is bigger with SSE, to upload data to the server, you need to send it as a file, or as a string, if you send strings you need to be sure that those are Base64(which increase the size) otherwise the server may reject the request, also, when sending data form the server to the client, you can only do it either serving files or sending strings.
• Trial and error is the way you sicronize the image and audio.

The application supports only two users, but with minor changes it may support more.

It uses the following open source projects:

• Bootstap
• JQuery
• Recorderjs
• libmp3lame-js

I modified the Recorderjs library to record mono audio instead of stereo and also to automatically convert the wav file to mp3 file using libmp3lame-js.

Important SSE is not supported on Internet Explorer(any version), also, sometimes SSE does not work fine on IIS Express, SSE is supported on the new version of Chrome for Android, so you can test the app on an Android tablet or cellphone.

Points of interest

The code in the server that receives the sequence of images and the audio and sends it to the other user is very simple:

        [HttpPost]
        public JsonResult PushVideoFragment()
        {
            //Get the audio stream and store it on a local collection
            MemoryStream ms = new MemoryStream();
            Request.Files[0].InputStream.CopyTo(ms);
            ms.Position = 0;
            string id = Guid.NewGuid().ToString().Replace("-", "");
            audios.Add(id, ms);
            //store the images locally
            var frames = new List();
            foreach (string k in Request.Form.Keys)
            {
                frames.Add(Request.Form[k]);
            }

            //send the audio to everyone but me
            var receivers = RemoteController.Users.Where(u => u != Request.Cookies["videoChatUser"].Value);
            foreach (var u in receivers)
            {
                dynamic call = new ClientCall { CallerId = Request.Cookies["videoChatUser"].Value, ClientId = u };
                //since we cannot send the audio, we just send the ID of the audio that we just received
                call.updateVideoFragment(id, frames, Request.Cookies["videoChatUser"].Value);
                RemoteExecution.ExecuteOnClient(call, false);
            }
            return Json(new { success = true });
        }

The most important pieces of code are in the Javascript code

//Send all the data to the server after the audio is converted
function sendAudioAndVideo() {
    var formData = new FormData();
    //set the frames
    for (var x in frames) {
        //only process first 10 frames, this is to sync video and audio
        if (x > 9)
            break;
        //append a new frame
        formData.append('frame' + x, frames[x]);
    }
    //clear the frames
    frames = [];
    //set the audio
    formData.append('edition[audio]', currentAudio);
    //Send the audio to the server as a file attachment
    $.ajax({
        type: 'POST',
        url: pushVideoFragmentUrl,
        data: formData,
        contentType: false,
        cache: false,
        processData: false,
    });
}

//This function is "invoked" from the MVC HomeController
function updateVideoFragment(id, remoteFrames, user) {
    //update frames
    for (var x in remoteFrames) {
        if (remoteFrames[x].length == 0)
            continue;
        pendingFrames.push(remoteFrames[x]);
    }

    //set the user who send the video
    $('#remoteUser').html(user);
    //we are not allowed to send the audio file directly, so we just set the src of the stream
    //play new audio
    $("#audio").attr("src", updateAudioUrl + '/' + id);
    $("#audio")[0].play();
}

Since only one browser(or tab) is allowed to access the camera and microphone, you need to use two computers to test the application

Since some audio encoding is done in the browser, you may need a couple of decent computers or cellphones so you can view it fluently.

Enjoy!