After running around in the realtime tech world for a couple of months since I started working for deepstreamHub, I think a lot of people have many misconceptions about a lot of concepts within this world. deepstream.io is an open realtime server that let’s you build realtime functionality right into your existing application while not having to worry about any of the backend magic. Almost all of deepstream works over websockets, which totally makes sense in the realtime world. However, a couple of months back, deepstream decided to release an HTTP API as well. One of the main reasons for this was that websockets weren’t a real choice when it came to IoT sensors. This got me curious and I wanted to dig a little deeper into the reasoning.
Let’s begin with a little background about the three main protocols that IoT sensors might work with:
As mentioned earlier, websockets are currently one of the main players in the realtime world. Websockets allows persistent connections with simultaneous bi-directional communication. They come with all the benefits of HTTP since websockets initially start off as an HTTP handshake, before getting elevated to continue the rest of the communication in websockets.
The very essence of websockets is a push-based strategy. This means that unlike polling or long-polling which involves asking the server at regular intervals if there’s any new data, with push-based- the server itself sends a message to the interested client whenever any new data becomes available. This is referred to as publish-subscribe paradigm or simply pub/sub. This is possible in websockets (vs. HTTP) because websockets keep the connection open until deliberately closed by one of the two parties (i.e the client or the server).
MQTT (Message Queue Telemetry Transport) is a lightweight application level protocol that allows you to set up a persistent connection between a server and a client. It’s mostly preferred to be used with low-power devices which is just the case with IoT sensors. It supports the publish-subscribe architecture. Further, it also provides certain other features such as QoS (Quality of Service) by allowing you to choose the quality of communication from a few available options which ensures the level of guaranteed delivery
By this point of evolution of the internet you most probably know that HTTP (Hyper Text Transfer Protocol) is a stateless protocol which generally works over TCP/IP and allows communication using the generic request-response paradigm. It is not considered to be a suitable protocol in the realtime sense since it provides non-persistent connections, i.e a new request-response cycle is established each time the client wishes to communicate with the server.
There is a possibility to do polling or long-polling on a HTTP server in order to continually ask for updates at a given frequency. HTTP also comes with a `persistent connection` function using which you can keep the connection open. Additionally, the new HTTP/2 protocol intrinsically supports persistent connections. But neither HTTP nor HTTP/2 support pub/sub or specifically, they are not push-based. So, there’s no way that HTTP could work in the realtime ecosystem. Right!?
So, what now?
All these protocols come with their own benefits, use-cases as well as disadvantages like any other protocol ever designed. But here we are particularly concerned about using them with the low-powered IoT sensors.
Now different kinds of sensors require different frequency with which they send their updates. In most cases, it’s an event that triggers the sensor to send some data. In other cases, they might be sending an update every hour or so. It’s quite rare for a sensor to continuously send some information to a server. Now, don’t get me wrong, a sensor of course could be continuously monitoring some data but that doesn’t necessarily mean it is sending all the data it monitors.
So, in such a case having a persistent connection would only intimidate these sensors and serve to be nothing but an overhead. However, surprisingly MQTT is still a solution in many scenarios. Although MQTT is persistent, it is extremely light weight and was designed to be used by such low power devices. Now if you think about it, using MQTT makes a lot of sense right?
As I mentioned earlier, it is very important to evaluate the need for a persistent connection based on the use-case. If the sensor which is monitoring some changes in its environment just needs to let a client or a server know about an event, we could simply use HTTP for this purpose. This is one of those situations where the simple old-school method of doing things indeed turns out to be the best possible way! In fact in some of the low-power devices, a persistent connection would require them to be active all the time whereas they could be sleeping otherwise thus saving on a lot of battery and other factors.
For this reason, although the HTTP protocol doesn’t seem to be the one to be use in a realtime use case, it is one of the best possible options for IoT sensors, since what happens after the sensing device sends an update can still be implemented in realtime and the magic essentially continues. In a few use cases where a persistent connection would actually make sense, MQTT would be the one to go with. However, websockets in my opinion are never an option because it’s simply too much overhead to setup which is not worth the benefits, at least considering the low-power scenario of the IoT devices.
This is essentially the main reasoning behind any realtime framework/ API/ server to extend a HTTP support.