Websocket Producer Server with Python Example

Background

My company is moving to a new office. And while being at it, they want to install screens that'll display fancy "current status of our data streams" visualizations. Who doesn't want them?

It was a nice problem for our intern and I gave him an architectural idea of having a Python back-end that queries the database to learn recent developments and sends relevant updates to JavaScript front-end via websockets. HTML and CSS are powerful enough to create an appealing data stream dashboard.

After making him struggle and suffer for a day, unintentionally of course, I realized that my instructions on how to implement a Websockets server in Python were vague and also Python Websockets is not a popular topic for Internet tutorials. Some examples that show up in web searches were implementing Websockets from scratch in plain Python. [1], [2]

Which made me sit down and prepare a bare bone example upon which our intern can build the rest.

The Task

The goal is to have a server that sends non-periodic updates to a client which connects to the server via Websockets. For the simplicity we don't need a database component. The server will generate random values at random intervals. And the client is a web page, when loaded, connects to the server and after establishing the connection, updates a DOM element whenever a new message comes from server.

Websockets

Websockets is a TCP-based communication protocol that transmits messages between both ends of the channel, a.k.a. full-duplex. It is like a telephone conversation where both parties can talk at the same time, and unlike HTTP where the client/browser sends requests to a server and the server sends back a response where the communication is one-way.

Browsers come with a simple Websocket API which was standardized by W3C. Just typing var ws = new WebSocket('ws://WS_SERVER_URL') creates a JavaScript WS client and connects it to the given server. [3]

Our reason to use Websockets instead of AJAX is that we don't want to periodically query the server about recent updates. We just want to be notified by the server when an update happens. As an alternative approach you can look at long polling.

Implementation

Client (index.html)

<!DOCTYPE html>
<html>
<body>
<span>variable: </span><span id="variable"></span>
<script>
var ws = new WebSocket('ws://localhost:5000');
var variableSpan = document.getElementById('variable');

ws.onmessage = function (event) {
    var message = JSON.parse(event.data);
    variableSpan.innerHTML = message.variable;
}
</script>
</body>
</html>

Here we have only one HTML element, a span with id variable, that is going to display the incoming messages.

On the JS side, we create a websocket that connects to our server on localhost port 5000. Whenever a message comes, its data content is parsed as JSON and the span's innerHTML is set to the variable value.

If you open this html file in your browser before starting to server you'll get a WebSocket connection to 'ws://localhost:5000/' failed error on the console.

Server (server.py)

import json
import random
import time

from eventlet import wsgi, websocket, listen


@websocket.WebSocketWSGI
def serve(ws):
    variable = 0
    while True:
        duration = random.random()
        time.sleep(duration)
        random_increment = int(random.random() * 10) + 1
        variable += random_increment
        message = json.dumps({'variable': variable})
        ws.send(message)

if __name__ == '__main__':
    wsgi.server(listen(('', 5000)), serve)

Here, an eventlet wsgi server listens to port 5000. And when a connection happens calls the serve function, which is decorated as a WebSocket handler.

serve initializes our variable to 0 and enters into an infinite loop. At each iteration it waits between 0 to 1 second. Generates a random increment between 1 to 10. Increases the variable. Serializes it into a JSON object and sends it as a message to client.

Make sure that eventlet library is installed on your Python environment. You can get it via pip install eventlet
Run the server via python server.py
Open the index.html with your browser.
The variable counter should increase randomly at random times

Git Repository

I uploaded the code to a GitHub repository: vug/python-websocket-producer-example

On Server Implementations

I had a failed attempt to write a server that has an infinite loop in it. I tried SimpleWebSocketServer, because it looked like the simplest Websocket example I found on the web.

import time
from SimpleWebSocketServer import WebSocket, SimpleWebSocketServer

class SimpleEcho(WebSocket):
    def handleConnected(self):
        variable = 0
        while True:
            time.sleep(1)
            variable += 1
            self.sendMessage(u'{}'.format(variable))

if __name__ == '__main__':
    server = SimpleWebSocketServer('localhost', 5000, SimpleEcho)
    server.serveforever()

However, this got stuck in the loop and never send a message to client-side. After some confusion I found the answer on Stackoverflow: sockets - simple websocket server on Python using time.sleep - Stack Overflow

The server that you are using is a synchronous, "select" type server. These servers use a single process and a single thread, they achieve concurrency through the use of the select() function to efficiently wait for I/O on multiple socket connections.

The advantage of select servers is that they can easily scale to very large number of clients. The disadvantage is that when the server invokes an application handler (the handleConnected(), handleMessage() and handleClose() methods for this server), the server blocks on them, meaning that while the handlers are running the server is suspended, because both the handlers and the server run on the same thread.

Another option for you to consider is to use a different server architecture. A coroutine based server will support your handler function as you coded it, for example. The two servers that I recommend in this category are eventlet and gevent. The eventlet server comes with native WebSocket support.

And this answer was the reason why I chose eventlet for this example. ^_^

Where to go from here

We can use more specialized libraries for websocket communication. For example the Socket.IO, which also has a Python implementation, python-socketio documentation.
This example can only handle one client. A definite direction to go is dealing with multiple clients. For example, socket.io has an emit method where the server can send messages to all connected clients.
More sophisticated design for handling the state in server-side. Maybe a State class with a serialize method. Or, to use a Redis database.

References

published at: 2016-11-21 12:47 edited at: 2016-11-21 13:53 UTC-5
tags: python websockets