Getting Started with the ESP32 and the ESP-IDF

With onboard WiFi and Bluetooth, low-power consumption, and multiple configuration options, the Espressif ESP32 microcontroller has established itself as a staple in the world of IoT. This guide will walk you through setting up an ESP32 development board using the Espressif ESP-IDF.

Prerequisites

Before you begin, you will need:

• An ESP32 Development board (e.g., ESP32-DEVKITC-32D)
• Familiarity with the command line
• Familiarity with the Espressif ESP-IDF

Setting up your Development Environment

Before you can begin, you will need to have the Espressif IoT Development Framework (esp-idf) and the proper USB drivers installed. Without this, you won’t be able to flash firmware to your device.

Important: If it’s your first time using the ESP32, download and install the USB drivers.

Here are the steps to install the ESP-IDF and drivers:

1. Install prerequisites for Windows, Linux or macOS.
2. Get the ESP-IDF.
3. Set up the tools.
4. Set up the environment variables.

We’ll go into each step in more detail below:

1. Install ESP-IDF prerequisites

To install the prerequisites, refer to the Espressif ESP-IDF documentation and install the correct prerequisites for your machine.

2. Get ESP-IDF

Using Git, clone the ESP-IDF repository.

Important: There are (at the time of writing) multiple options for which ESP-IDF you use, and for purposes of this guide we will be using the production stable version of the ESP-IDF. You can read more about ESP-IDF versions in the ESP-IDF documentation.

To clone the repository, issue the following commands:

mkdir esp
cd esp
git clone -b v4.1 --recursive https://github.com/espressif/esp-idf.git

For production environments, we recommend the latest stable release offered by Espressif, which can be found here.

3. Set up the ESP-IDF tools

In Step 2, you cloned the ESP-IDF into a parent folder named esp. Now, let’s set up the ESP-IDF tools.

Once the ESP-IDF is cloned, be sure to run the install script provided by Espressif by running ./install.sh or install.bat (.bat specifically for Windows) in the esp-idf folder. You can do this by issuing the following commands:

1. Inside of the esp directory created in Step 2, navigate to the esp-idf directory (this was included in the downloaded repository).

cd esp-idf 

2. install.sh installs the tools used by ESP-IDF, such as the compiler, debugger, Python packages, etc.

./install.sh

3. export.sh - A script the ESP-IDF provides to make the tools usable from the command line.

. ./export.sh

To see OS-specific instructions, refer to the esp-idf Getting Started walkthrough for commands associated with your operating system.

If you run into any installation issues, please refer to the ESP-IDF documentation.

4. Set up the environment variables

For MacOS or Linux, the ESP-IDF path needs to be added to our shell profile.

If you are using the standard bash, in your home folder (~/) open your shell profile (.profile, .zshrc, etc.) in a text editor, and add:

export IDF_PATH=~/esp/esp-idf
export PATH="$IDF_PATH/tools:$PATH"

Adding these two lines allow you to use the idf.py tool provided by the ESP-IDF to build, flash, and edit configuration settings.

Before you use the ESP-IDF tools, you must run the export.sh for each terminal session. Optionally, you can add an alias to run the export script from any directory:

alias get_idf='. $HOME/esp/esp-idf/export.sh'

Create a new ESP-IDF Project

Now let’s create a new ESP-IDF project that will allow us to connect our ESP32 device to Losant.

You may find the full code for this project on GitHub.

1. Inside of the esp directory created in Step 2, clone the losant-esp-idf-esp32 project:

git clone git@github.com:Losant/losant-esp-idf-esp32.git

2. Navigate to the new project:

cd losant-esp-idf-esp32

This project is an example of reporting state to a Losant device every 5 seconds. In the example, the following static state is reported (you can change as desired):

{\"data\": {\"message\": \"hello from ESP32\",\"number\": 14}}

Configure WiFi

To configure WiFi on the device, perform the following steps:

1. First, open the ESP-IDF menu:

idf.py menuconfig

If successful, you should get a menu that looks similar to this (color scheme may vary):

If idf.py menuconfig does not launch the menu, you may need to go back and ensure that your environment is set up correctly.

2. Using the arrow keys, navigate to Example Connection Configuration and press enter. In this menu, you set your WiFi credentials.

3. Once done, you may save by pressing shift+s to save and/or shift+q to quit.

This creates a new Project Configuration File called sdkconfig in your project and stores the WiFi configuration. You can read more about this file within the Project Configuration ESP-IDF documentation.

For more details on configuring WiFi:

• ESP-IDF WiFi documentation
• ESP-IDF WiFi examples

Authenticating to Losant

You may find the main file for this project at:

losant-esp-idf-esp32/main/app_main.c

To connect to Losant, a device must provide a device ID, Access Key, and Access Secret. Within the app_main.c file, we can configure them with the following global variables:

• LOSANT_DEVICE_ID
• LOSANT_ACCESS_KEY
• LOSANT_ACCESS_SECRET

These values are used to authenticate the device to the MQTT Broker. To learn more refer to the Losant MQTT documentation.

Publishing State to Losant

Within app_main.c the function sendMessage is publishing state using esp_mqtt_client_publish. Here is the code:

void sendMessage(void *pvParameters)
{
    esp_mqtt_client_handle_t client = *((esp_mqtt_client_handle_t *)pvParameters);

    // create topic variable
    char topic[128];

    // Set the topic variable to be a losant state topic "losant/DEVICE_ID/state"
    sprintf(topic, "losant/%s/state", LOSANT_DEVICE_ID);

    // Using FreeRTOS task management, forever loop, and send state to the topic
    for (;;)
    {
        // You may change or update the state data that's being reported to Losant here:
        esp_mqtt_client_publish(client, topic, "{\"data\": {\"message\": \"hello from ESP32\",\"number\": 14}}", 0, 1, 0);

        vTaskDelay(pdMS_TO_TICKS(5000)); // wait 5 seconds
    }
}

You can read more about the esp_mqtt_client_publish function in the esp-idf documentation.

Receiving Commands from Losant

To subscribe to commands from Losant, we can configure the device to subscribe to the command topic, and receive the subscribed data.

Within app_main.c the function mqtt_event_handler_cb is subscribing with esp_mqtt_client_subscribe and the MQTT_EVENT_DATA event. Here is the code:

static esp_err_t mqtt_event_handler_cb(esp_mqtt_event_handle_t event)
{
    esp_mqtt_client_handle_t client = event->client;

    int msg_id;

    // define Losant device Command topic
    char command_topic[128];
    sprintf(command_topic, "losant/%s/command", LOSANT_DEVICE_ID);

    switch (event->event_id)
    {
    case MQTT_EVENT_CONNECTED:
        // on connect, subscribe to the command topic
        msg_id = esp_mqtt_client_subscribe(client, command_topic, 0);
        ESP_LOGI(LOGGING_TAG, "sent subscribe successful, msg_id=%d", msg_id);
        break;
    case MQTT_EVENT_DATA:
        ESP_LOGI(LOGGING_TAG, "MQTT_EVENT_DATA");
        printf("TOPIC=%.*s\r\n", event->topic_len, event->topic);
        printf("DATA=%.*s\r\n", event->data_len, event->data);
        break;
    default:
        ESP_LOGI(LOGGING_TAG, "Other event id:%d", event->event_id);
    }
    return ESP_OK;
}

The above function listens for MQTT events while the firmware is running. The example utilizes two events:

• MQTT_EVENT_CONNECTED - This event occurs when client has successfully established a connection to the broker and is ready to send and receive data. Now that the client is successfully connected, esp_mqtt_client_subscribe(client, command_topic, 0); is used to subscribe to the command topic for this device.
• MQTT_EVENT_DATA- This event occurs when the client has received data from a subscribed topic. In this case, the events provides the topic to which the message was received along with the data included with that message.

A full list of MQTT events provided by the esp-idf can be found here: ESP-IDF Events Documentation.

Build Firmware

It’s best practice to build your firmware to ensure that it compiles correctly.

To build your project, run the command:

idf.py build

Flash Firmware

To build and flash the firmware, you may do so with the following command:

idf.py -p <your port> flash monitor

You must replace <your port> with the value of the serial port to the ESP32 device. To flash your firmware, you need to find what port your device is connected to by running the command (in the terminal, on MacOS) ls /dev/cu.* and look for SLAB_USBtoUART. That will be the port that you use.

For most MacOS or Linux devices, the command will be:

idf.py -p /dev/cu.SLAB_USBtoUART flash monitor

Upon a successful flash, your device will connect to Losant, and begin sending state to the Losant Platform. To verify that your device is reporting, you may look at the Device Logs, which displays a real-time stream of device events.

Now you are ready to build Dashboards, Workflows, and Experiences. Happy Developing!

Resources

The following are resources that will help in your development:

Verify Environment Setup

To verify that your development environment is set up correctly:

1. Navigate to the esp folder: cd ~/esp
2. Copy the hello_world example project: cp -r $IDF_PATH/examples/get-started/hello_world .
3. Navigate into the hello_world project: cd hello_world
4. Run the project configuration utility provided by Espressif: idf.py menuconfig

If your Development Environment has been correctly set up, the following menu will appear:

If you did not get the above menu, refer back to the Getting Started Guide from Espressif.

A common issue that will result in the above menu not being shown is that once the hello_world project is copied, the install.sh and export.sh tools need to be run again. To do this:

1. Navigate to the esp-idf folder:
   • cd ~/esp/esp-idf
2. Run the install script:
   • ./install.sh
3. Run the export script:
   • . ./export.sh