MQTT: The Lightweight Protocol Powering Real-Time Everything

Why MQTT deserves your attention

Every time you watch a delivery rider's icon crawl across a map, something has to move that GPS point from a phone on a motorbike to your screen — reliably, within a second or two, over a patchy mobile network, and without draining the rider's battery. Multiply that by tens of thousands of riders reporting every few seconds, and the choice of messaging protocol stops being a detail. It becomes the architecture.

MQTT (Message Queuing Telemetry Transport) is the protocol built for exactly this problem. It was created in 1999 by Andy Stanford-Clark (IBM) and Arlen Nipper to monitor oil pipelines over expensive, unreliable satellite links. The design goals were simple: tiny messages, minimal bandwidth, low power, and graceful handling of connections that drop without warning.

Two decades later, those same constraints describe a smartphone on a 4G network in city traffic, a battery-powered sensor in a cold-storage warehouse, or a GPS tracker bolted onto a shipping container. MQTT is now an OASIS open standard (versions 3.1.1 and 5.0), and 3.1.1 is also published as ISO/IEC 20922. It is supported natively by AWS IoT Core, Azure IoT Hub, and practically every IoT platform on the market.

In this article I'll cover how MQTT works, why it often beats HTTP and other protocols for real-time workloads, where it falls short, and how it is used in ride-hailing, food delivery, IoT data capture, and asset tracking.

How MQTT works

MQTT uses a publish/subscribe model with a central broker. Clients never talk to each other directly. A client publishes a message to a topic, and the broker forwards it to every client subscribed to that topic. Publishers and subscribers don't need to know each other's address, or even be online at the same time.

The building blocks

• Broker: The server that receives, filters, and routes messages. Popular options include Eclipse Mosquitto, EMQX, HiveMQ, and VerneMQ, plus managed services like AWS IoT Core.
• Client: Any device or app that publishes or subscribes — a rider's phone, a temperature sensor, a backend microservice, a dashboard.
• Topic: A UTF-8 string organised in levels separated by /, for example city/dhaka/rider/4821/location. Topics are created on the fly; there is nothing to pre-register.
• Wildcards: + matches one level and # matches all remaining levels. city/dhaka/rider/+/location gives you every rider's location in Dhaka.
• Persistent TCP connection: A client opens one long-lived connection (port 1883, or 8883 with TLS) and keeps it alive with tiny ping packets. MQTT over WebSockets is also available for browsers.

Quality of Service (QoS)

MQTT lets each message choose its own delivery guarantee:

Features that make it production-ready

• Retained messages: The broker stores the last message on a topic and hands it instantly to any new subscriber. A customer opening the app sees the rider's last known position immediately, without waiting for the next ping.
• Last Will and Testament (LWT): A client registers a "will" message at connect time. If it disconnects ungracefully — battery dies, tunnel, crash — the broker publishes it for them, e.g. rider/4821/status = offline.
• Persistent sessions: With a durable session, the broker remembers a client's subscriptions and queues QoS 1/2 messages while it is offline, delivering them on reconnect.
• Keep-alive: A configurable heartbeat lets both sides detect dead connections quickly.

What MQTT 5.0 added

MQTT 5.0 (2019) is the version to choose for new projects. Key additions:

• Shared subscriptions ($share/group/topic) to load-balance messages across several backend consumers.
• Message and session expiry so stale data (a location from 10 minutes ago) is discarded automatically.
• Reason codes on every acknowledgement, making failures debuggable.
• User properties — custom key-value headers, like HTTP headers, for trace IDs or schema versions.
• Topic aliases that replace long topic strings with a short integer to save bytes on every message.
• Request/response support via response topics and correlation data.
• Flow control to stop a slow client from being flooded.

Key advantages of MQTT

1. Extremely lightweight. The fixed header is just 2 bytes. A GPS update can be sent in well under 100 bytes, while the same data over HTTP often carries several hundred bytes of headers before the payload even starts.
2. Battery and bandwidth friendly. One persistent connection replaces thousands of TCP and TLS handshakes. For a rider app sending a location every 3–5 seconds for a 10-hour shift, that is a real difference in battery life and mobile data cost.
3. Real-time push, not polling. The broker pushes messages the moment they arrive. Clients don't waste requests asking "anything new?" — latency drops to milliseconds and server load drops with it.
4. Built for unreliable networks. QoS levels, persistent sessions, and automatic reconnects mean messages survive tunnels, lift rides, and handovers between cell towers.
5. Decoupled architecture. Publishers and subscribers are independent. You can add a new analytics service, fraud detector, or dashboard just by subscribing to a topic — no change to the devices.
6. One-to-many and many-to-one at scale. A single broadcast (e.g. "surge pricing in zone 7") reaches thousands of devices, and thousands of devices can feed one backend. Modern clustered brokers handle millions of concurrent connections.
7. Presence detection for free. Last Will and keep-alive tell you instantly when a driver, device, or vehicle goes offline.
8. Payload agnostic. Send JSON, Protobuf, CBOR, binary images — MQTT doesn't care. Pair it with Protobuf for the smallest messages.
9. Open standard, huge ecosystem. Client libraries exist for Android, iOS, JavaScript, Python, Java, Go, C, and microcontrollers like ESP32. No vendor lock-in.
10. Secure when configured properly. TLS encryption, username/password or client certificates (mTLS), token-based auth, and per-topic access control lists.

How MQTT compares with other protocols

MQTT is not "better" than everything — it is better at a specific job: moving small, frequent messages between many devices and a backend over imperfect networks. Here's how it stacks up.

The key comparisons in plain words

• MQTT vs HTTP: HTTP makes the client ask; MQTT lets the server tell. For a location feed, polling HTTP every 3 seconds means a full request and response each time, even when nothing changed. MQTT sends only when there is data, over a connection that is already open.
• MQTT vs WebSockets: WebSockets give you a pipe; MQTT gives you a messaging system on top of a pipe. With raw WebSockets you must build topics, fan-out, delivery guarantees, offline queues, and presence yourself. You can also run MQTT over WebSockets and get both.
• MQTT vs CoAP: CoAP runs over UDP and suits the tiniest battery devices. MQTT is the better choice when you need a broker, fan-out, and guaranteed delivery. MQTT-SN, a sensor-network variant, closes some of this gap.
• MQTT vs AMQP: AMQP is richer for enterprise workflows but heavier on the client. MQTT wins on constrained devices and mobile apps.
• MQTT vs Kafka: They are complementary, not competitors. MQTT is the edge protocol that connects millions of devices; Kafka is the backend backbone that stores and processes the stream. A common pattern is devices → MQTT broker → Kafka → analytics and databases.

Pros and cons at a glance

Bottom line: MQTT's weaknesses are mostly operational rather than fundamental. A well-run, clustered broker with TLS, ACLs, and a clear topic design removes most of them.

Real-world use cases

1. Ride-hailing: live driver location and trip matching

A ride-hailing app has two real-time problems: knowing where every driver is right now, and getting a trip offer to the right driver in seconds.

• The driver app publishes GPS coordinates every 2–5 seconds at QoS 0 to a topic like rides/dhaka/driver/{id}/location. Losing one ping doesn't matter because the next one replaces it.
• A matching service subscribes (via a shared subscription) to all driver locations, updates a geospatial index, and finds the nearest available drivers.
• Trip offers go to rides/driver/{id}/offers at QoS 1, so the offer is guaranteed to arrive even if the driver's connection blinked.
• During the trip, the passenger's app subscribes to rides/trip/{tripId}/driver-location. A retained message means the car appears on the map the instant the screen opens.
• Last Will marks a driver offline when the phone loses signal, so they are removed from matching automatically.

2. Food and parcel delivery: order status and rider tracking

Delivery platforms juggle three parties — customer, restaurant or merchant, and rider — all needing the same events at the same time.

• Order lifecycle events (placed → accepted → preparing → picked up → delivered) are published to orders/{orderId}/status at QoS 1. The customer app, merchant tablet, support dashboard, and notification service all subscribe to the same topic.
• Rider location streams to the customer's live map exactly as in ride-hailing.
• Dispatch pushes new assignments to riders instantly, replacing constant polling and saving riders' mobile data — which matters in markets where riders pay for their own data plans.
• Zone-wide broadcasts ("heavy rain in Gulshan, delivery times extended") reach every rider in an area with a single publish.
• Riders in basements or lifts reconnect with a persistent session and receive any queued assignments immediately.

3. IoT devices for data capture

Sensors are MQTT's original home. Examples include temperature and humidity loggers in cold chains, smart energy meters, air-quality monitors, and agricultural soil sensors.

• Low-power microcontrollers such as the ESP32 run MQTT clients in a few kilobytes of memory.
• Devices wake up, publish a reading, and sleep again — the small packet size directly extends battery life.
• Commands flow the other way: devices/{id}/commands for configuration changes or firmware-update triggers.
• A "device shadow" pattern (used by AWS IoT Core and Azure IoT Hub) keeps a retained desired-vs-reported state for each device.
• In a restaurant kitchen, fridge and freezer sensors can raise an alert the moment temperature crosses a food-safety threshold, instead of waiting for a manual check.

4. Asset and fleet tracking

For tracking vehicles, containers, pallets, e-bikes, or delivery boxes that report their position often, MQTT keeps costs predictable.

• GPS trackers with cellular (4G/LTE-M/NB-IoT) modems publish location, speed, battery, and ignition state at intervals tuned to movement: every few seconds when moving, every few minutes when parked.
• Geofencing services subscribe to location topics and raise events when an asset enters or leaves a zone (a warehouse, a customer's area, a restricted road).
• Telematics such as fuel level, engine diagnostics, door open/close, and cargo temperature ride on the same connection.
• Fleet managers get a live map, route replay (from data stored downstream), and theft or tampering alerts.
• The data savings add up quickly: across thousands of SIM-connected devices, smaller packets directly cut the monthly cellular bill.

5. Other proven domains

• Industrial IoT (IIoT): Factories use MQTT with the Sparkplug B specification to connect PLCs and sensors to SCADA and cloud systems.
• Smart homes: Home Assistant and many smart plugs, lights, and hubs speak MQTT.
• Connected cars: Vehicle-to-cloud telemetry and remote commands such as lock, unlock, or pre-cool.
• Messaging and notifications: Facebook Messenger famously adopted MQTT in 2011 for fast, battery-efficient mobile messaging.
• Healthcare: Wearables and remote patient monitors streaming vitals to care teams.
• Energy and utilities: Smart grids, solar inverters, and EV charging stations.

A reference architecture for a delivery platform

Here's how the pieces typically fit together in a delivery or ride-hailing system.

flowchart LR
    R[Rider app] -- location QoS 0 --> B((MQTT broker cluster))
    C[Customer app] <-- order status and live map --> B
    M[Merchant tablet] <-- new orders --> B
    T[IoT trackers and sensors] -- telemetry --> B
    B -- shared subscriptions --> D[Dispatch and matching]
    B -- bridge --> K[(Kafka event stream)]
    K --> A[Analytics and ETA models]
    K --> S[(Time-series DB)]
    D -- assignments QoS 1 --> B

The broker handles device connections and fan-out. Backend services consume through shared subscriptions for horizontal scaling, and a bridge streams everything into Kafka for storage, analytics, and machine-learning models such as ETA prediction.

Example topic design

Two rules keep this clean: put the most general level first (so wildcards work naturally), and never put personal data such as phone numbers in topic names.

Best practices for running MQTT in production

1. Start with MQTT 5.0. Shared subscriptions, expiry, and reason codes save a lot of custom work.
2. Always use TLS (port 8883, or 443 over WebSockets) and authenticate every client with certificates or short-lived tokens such as JWTs.
3. Enforce topic-level ACLs. A rider should only publish to their own topics and never be able to subscribe to #.
4. Pick QoS per message, not per app. QoS 0 for high-frequency telemetry, QoS 1 for business events, QoS 2 only where duplicates are truly unacceptable.
5. Make consumers idempotent. QoS 1 can deliver duplicates; include a message ID and deduplicate.
6. Use compact payloads. Protobuf or CBOR instead of verbose JSON, and send deltas where possible.
7. Tune publish frequency adaptively. Send location more often when moving fast and less often when idle.
8. Cluster the broker across availability zones and load-test for your peak concurrent connections, not your average.
9. Bridge to a durable store. Stream into Kafka, a time-series database, or a data lake for history, replay, and analytics.
10. Monitor the right metrics: connected clients, message rates, dropped messages, queue depth per session, and reconnect storms after network outages.

Conclusion

MQTT has quietly become the nervous system of the connected world. A protocol designed for oil pipelines over satellite links turns out to be exactly what a delivery rider's phone, a cold-chain sensor, or a GPS tracker on a truck needs: small messages, low power, real-time push, and resilience when the network misbehaves.

It isn't a replacement for HTTP APIs or a streaming platform like Kafka. It's the layer that connects the edge to your backend. If your product involves anything that moves, measures, or needs to know what's happening right now, MQTT belongs in your architecture.

Have you used MQTT in production, or are you weighing it against another protocol? I'd love to hear about your experience.