MQTT, Mosquitto and Other Brokers

MQTT is the small messaging protocol behind a lot of the Internet of Things, from oil pipelines to the temperature sensor in a Home Assistant setup. Here’s where it came from, how it works, and which broker to pick, starting with Mosquitto.

• IoT
• Networking
• Self-hosting

Where it came from

In 1999, Andy Stanford-Clark of IBM and Arlen Nipper of Arcom (now Eurotech) needed to monitor oil pipelines over expensive, unreliable satellite links. They designed a protocol with a tiny overhead that keeps working when the connection drops. The name comes from IBM’s MQ Series message-queuing products: MQ Telemetry Transport. Today it’s just treated as a name, since MQTT isn’t a queue.

• 1999: invented at IBM and Arcom for pipeline telemetry
• 2010: IBM publishes MQTT 3.1 royalty-free, so anyone can implement it
• 2014: MQTT 3.1.1 becomes an OASIS standard
• 2016: 3.1.1 is also ratified as the international standard ISO/IEC 20922
• 2019: MQTT 5.0 becomes an OASIS standard

Versions 3.1.1 and 5.0 are the ones you’ll meet in practice. Most devices and brokers speak both.

How it works

MQTT is publish/subscribe. Devices never talk to each other directly. They all connect to a central server, the broker. A sensor publishes a message to a topic, and the broker passes it to every client that has subscribed to that topic. Publishers don’t know who is listening, and subscribers don’t know who sent it.

• Topics are paths like home/livingroom/temperature. They don’t need to be created first; publishing to one is enough.
• Wildcards work only when subscribing: + matches one level (home/+/temperature gives every room), and # matches everything below (home/#).
• Quality of service is chosen per message. QoS 0 is “at most once” (fire and forget), QoS 1 is “at least once” (may arrive twice) and QoS 2 is “exactly once” (slowest).
• Retained messages are stored by the broker, and a new subscriber gets the last value immediately instead of waiting for the next update. Ideal for states like “the light is on”.
• Last will: when a client connects, it can leave a message that the broker publishes if the client disappears without saying goodbye. This is how dashboards show a device as offline.
• Persistent sessions let the broker remember a client’s subscriptions and keep its QoS 1 and 2 messages while it’s offline.

The smallest MQTT packet is two bytes, and it runs over plain TCP on port 1883, over TLS on port 8883, or over WebSockets, so browsers can use it too.

What MQTT 5 added

Mostly things that make larger systems easier to run: reason codes that say why something failed, user properties (custom headers), shared subscriptions to spread messages across several workers, message and session expiry, topic aliases that replace long topic names with a number, and a standard request/response pattern.

Mosquitto

Eclipse Mosquitto is the broker most people start with. Roger Light started it around 2009, and it’s now an Eclipse Foundation project. It’s written in C, so it’s tiny: it runs happily on a Raspberry Pi or a router. It speaks MQTT 3.1, 3.1.1 and 5.0, and comes with the mosquitto_pub and mosquitto_sub command-line tools, which are useful with any broker. It’s also the broker behind the Mosquitto add-on in Home Assistant. The current release is 2.1, and the license is EPL-2.0 or BSD-3-Clause, your choice.

Try it in five minutes

On Debian or Ubuntu, install the broker and the tools:

sudo apt install mosquitto mosquitto-clients

In one terminal, subscribe to everything under home/. -v prints the topic next to each message:

mosquitto_sub -h localhost -t 'home/#' -v

In a second terminal, publish a retained reading:

mosquitto_pub -h localhost -t home/livingroom/temperature -m 21.5 -r

The first terminal prints home/livingroom/temperature 21.5. Because of -r, anyone who subscribes later gets 21.5 straight away.

Let other devices connect

Since version 2.0, Mosquitto is locked down by default: with no listener configured, it accepts connections only from the same machine. Devices on your network need a listener and a login. Create a password file:

sudo mosquitto_passwd -c /etc/mosquitto/passwd sensor1

Then add a config file such as /etc/mosquitto/conf.d/lan.conf:

listener 1883
allow_anonymous false
password_file /etc/mosquitto/passwd

Restart it with sudo systemctl restart mosquitto. Version 2.1 moved password and access-control files into plugins. The password_file option above still works, but it’s marked for removal in 3.0. Once anything leaves your network, use TLS on port 8883 instead of plain 1883, because MQTT sends passwords in clear text otherwise.

Other brokers

Mosquitto runs on a single server. It can forward topics to another broker (bridging), but it doesn’t cluster. When you need clustering, huge numbers of connections or a broker built into your app, look at these:

Watch the licenses. EMQX moved from Apache-2.0 to the Business Source License in version 5.9 (2025): you can run a single node in production for free, but a cluster, or offering it as a service, needs a commercial license. Each release turns into Apache-2.0 four years after it’s published. Older versions keep their original license.

If you’d rather not run a broker at all, every big cloud has a managed one, such as AWS IoT Core, the MQTT broker in Azure Event Grid, HiveMQ Cloud and EMQX Cloud. They’re convenient, but your devices then depend on an internet connection and someone else’s pricing.

Things that will bite you

• Client IDs must be unique. If two devices connect with the same ID, the broker disconnects the first one, which reconnects and kicks out the second, over and over. It looks like a flaky network.
• QoS is per hop. The QoS a subscriber gets is the lower of the publisher’s and its own. Publishing at QoS 2 doesn’t help if the subscriber asked for QoS 0.
• Retained messages stay forever. To delete one, publish an empty retained message to the same topic: mosquitto_pub -t home/old/topic -r -n.
• Plain MQTT isn’t encrypted. Usernames and passwords go over the wire as readable text on port 1883. That’s fine inside your home network, not across the internet.
• Plan your topic names. Something like site/room/device/measurement keeps wildcards useful. Don’t start topics with $; brokers use $SYS/ for their own statistics.

Wrap-up

MQTT has lasted over 25 years because it does one thing well: moving small messages between many devices over bad connections, through a broker. For anything at home or at the edge, Mosquitto is the obvious start. Once you need clustering or millions of devices, compare EMQX, HiveMQ and VerneMQ, and read their licenses before you commit.

← Back to Scratchpad