How to Connect an ESP32 WiFi Microcontroller to a Raspberry Pi Using IoT MQTT

A close-up of a circuit board Description automatically generated with low confidence

One of the most important features of an Internet of Things (IoT) device is its ability to “speak” to the network.

The steps a device must take to communicate successfully with another device is called a communication protocol.

A protocol is a set of official procedures and rules that govern a particular affair. In terms of IoT devices, a communication protocol lays out the procedure that devices must take in order to send/receive information.

Communication protocols are split into two categories, Network Connection Protocols and Data Communication Protocols

Let’s imagine that we are laying out a new city. First we begin with an empty lot of land somewhere.

Second, we must place physical roads, with different sizes and weight capabilities, to enable people to move into the new city.

NOTE: This is a long, very detailed article so here's a free PDF version of it for easy reading and future reference.

As seen in the figure below, I’ve laid out the roads. The roads in our analogy are the Network Protocols. Network protocols are the physical layers of the network. In an OSI Model they are usually labeled as the Physical layer, DataLink layer and Network layer.

Now we need to populate the roads (Network protocol) with cars and people (application data protocol). As you can see in the updated figure below, different vehicles can fit different routes and vice versa.

In our case, the network protocol and the data protocol work hand in hand to facilitate a communication channel (roads) that allows data to travel through the internet to their intended destination.

As a quick refresher, an Open Systems Interconnection (OSI) Model outlines 7 layers (steps) that computer systems must use to communicate over a network. The following figure shows these 7 steps and an example of communication protocols that are used for the layers.

There are many protocols that are being used today that facilitate the network and the application layer of the OSI Model. In this article, we will focus on the data protocol part and in particular the Message Queuing Telemetry Transport (MQTT) protocol.

In most applications, when designing or programming your IoT device, the communication module that you are using will take care of the Network protocol, and you will only need to focus on the application and data protocol.

To demonstrate the use of MQTT in a real application, we will show how to connect an ESP32 Development Module to a Raspberry Pi over a local network, and communicate via MQTT to send lightweight data.

MQTT is a protocol made for IoT devices. It allows for lightweight communication between devices and servers. The protocol is well documented in the official MQTT website.

In simple terms, it allows a device to subscribe and publish to topics (endpoints) that facilitate sending small data over the network protocol.

An MQTT broker is used as the manager that routes all the messages to the rightful owners (subscribers) and accepts publishing messages from different devices. Every MQTT communication mechanism must start with planning the topics that are to be used for both the server side and the device side.

FREE GUIDE: Introduction to Microcontrollers

In our demo, we will use a Raspberry Pi as our MQTT broker (server) and an ESP32 as the publisher to publish light sensor data to the MQTT broker topic as seen in the following diagram.

It is generally a good idea to begin with a flow diagram as seen above with the planned topics and clients. We will use a light sensor to demonstrate the idea of using MQTT to push data to the MQTT broker on the Raspberry Pi.

In a real application, security is a third layer that must be considered. In our case we will not worry too much about the security since it is running in a local network with no external connection.

But in cases where you expose your broker to the internet, you want to keep your network secure.

We will break down the demo into the following sections:

Raspberry Pi:
– MQTT broker

ESP32:
– Light Sensor
– MQTT client
– WiFi connection

Raspberry Pi

The Raspberry Pi will act as the MQTT Broker server. It will route all of our messages to their appropriate subscriber and accept messages from publishers.

Step 1

To begin, enable SSH on your Raspberry Pi in order for you to access the terminal from your host PC. If you are using Windows, be sure to install an SSH client such as PuTTy.

Find your Raspberry Pi’s IP address and SSH into it. There’s a great article on how to do this here.

Step 2

Install mosquitto and mosquitto-clients to start an MQTT broker server and to test the server with the mosquitto-clients. Use the following command to install mosquitto:

sudo apt install mosquitto mosquitto-clients

Step 3

At this point the mosquitto MQTT broker service should be running. To confirm this run the following command:

sudo systemctl status mosquitto

If you see the green “active (running)” status, that means it is running in the background and your setup is good to go thus far.

Step 4

To autostart the mosquitto service every time the Raspberry Pi restarts, run the following command:

sudo systemctl enable mosquitto.service

Step 5

Configure the mosquitto conf file so you are able to connect to it via the local network by enabling port 1883.

Start with the following command to edit the mosquitto.conf file:

sudo nano /etc/mosquitto/mosquitto.conf

Then, move to the very end of the file and enter the following lines:

listener 1883

allow_anonymous true

Next, press CRTL-X to exit and then reboot your Raspberry Pi.

Step 6

Keep a note of your Raspberry Pi IP address so you can use it as the hostname for all the MQTT communication.

Step 7

Let us go ahead and start a subscriber that will show all the messages that have been sent to the topics “esp32/light,” in order to see the messages the ESP32 will be sending to that topic. To do this, run the following command:

mosquitto _sub -h <RPi IP Address> -t “esp32/light”

Be sure to replace the <RPi IP Address> with the IP address of your Raspberry Pi.