OTA UPDATES ON ESP32 WITH EC25 #
Introduction #
The ESP32 microcontroller, in conjunction with the EC25 module, enables over-the-air (OTA) firmware updates. By storing the latest firmware on a GitHub repository, the ESP32 can periodically check for updates and download them via GPRS, ensuring that the device always runs the most recent version of the firmware. This process eliminates the need for physical access to the device and allows for efficient deployment of new features and security patches.
Project Overview #
The firmware update process works as follows:
- Current Version Check: The ESP32 checks the version.txt file in the GitHub repository to determine the latest firmware version.
- New Firmware Detection: If a new version is detected (i.e., different from the current firmware version running on the ESP32), the device downloads the latest (firmware_vX.X.X.bin) firmware file.
- OTA Update: The ESP32 initiates the OTA update and installs the new firmware. Once the update is complete, the ESP32 restarts and runs the new firmware.
Branch Setup
- The release branch of your GitHub repository should contain:
- The latest firmware file (firmware_vX.X.X.bin)
- A version.txt file that stores the version number of the latest firmware.
/release/
├── firmware_v1.0.1.bin
├── version.txt
- version.txt: This file should contain only the version number of the latest firmware. e.g., 1.0.1
Key components
- Quectel EC25 Module: Responsible for connecting to the internet via GPRS and making HTTPS requests.
- Update.h Library: Handles the Over-the-Air(OTA) process.
- Root CA Certificate: As GitHub uses HTTPS, the CA root certificate for raw.githubusercontent.com must be uploaded to the EC25 module to ensure secure communication. This is required since the OTA process downloads the firmware from a secure GitHub URL. The cert.h file in this repository contains the CA root certificate.
Software Setup #
A. Install the ESP32 board in Arduino IDE:
- a. Go to File > Preferences.
- b. Go to Tools > Boards > Boards Manager, search for “ESP32”, and install.
- c. OTA on ESP32 (https://github.com/IndustrialArduino/OTA-on-ESP/tree/main/OTA_on_ESP32_over_EC25)
B. Install Required Libraries (Arduino):
Go to Sketch > Include Library > Manage Libraries and install:
- Update.h: For communication and OTA functionality
- cert.h: Contains the CA certificate needed for HTTPS communication.
C. Configure the code:
- Open the code provided in your Arduino IDE.
- Replace the GPRS credentials (apn, gprsUser) with your SIM card provider details.
- Set the correct firmware version file paths. (ex: https://raw.githubusercontent.com/IndustrialArduino/OTA-on-ESP/release/version.txt)
Code Explanation #
This Arduino code demonstrates how to implement an Over-The-Air (OTA) firmware update on an ESP32 using a EC25 modem to connect to the internet. The program fetches the latest firmware version from a GitHub repository, compares it with the current version running on the device, and downloads the new firmware if necessary. Here’s a breakdown of the key components of the code.
- Part 1
#include u003cArduino.hu003en#include u003cWiFi.hu003en#include u003cUpdate.hu003en#include u0022cert.hu0022nString gsm_send_serial(String command, int delay);n#define SerialMon Serialn#define SerialAT Serial1n#define GSM_PIN u0022u0022n#define UART_BAUD 115200n#define MODEM_TX 32n#define MODEM_RX 33n#define GSM_RESET 21n// Your GPRS credentialsnconst char apn[] = u0022dialogbbu0022;nconst char gprsUser[] = u0022u0022;
This Part defines essential libraries, constants, and variables for an OTA firmware update project on an ESP32 using the EC25 module. It includes libraries for OTA process, GPRS credentials, and firmware update settings.
- Part 2
String current_version = u00221.0.0u0022;nString new_version;nString version_url = u0022https://raw.githubusercontent.com/IndustrialArduino/OTA-on-ESP/release/version.txtu0022;nString firmware_url;n//variabls to blink without delay:nconst int led1 = 2;nconst int led2 = 12;nunsigned long previousMillis = 0; // will store last time LED was updatednconst long interval = 1000; // interval at which to blink (milliseconds)nint ledState = LOW; // ledState used to set the LED
This code initializes variables for firmware versions and LED control. It defines the URL of the text file containing the latest version number and sets up the HTTPS transport for communication with the EC25 module.
- Part 3
void setup() {n // Set console baud raten Serial.begin(115200);n delay(10);n SerialAT.begin(UART_BAUD, SERIAL_8N1, MODEM_RX, MODEM_TX);n delay(2000);n pinMode(GSM_RESET, OUTPUT);n digitalWrite(GSM_RESET, HIGH); // RS-485n delay(2000);n pinMode(led1, OUTPUT);n pinMode(led2, OUTPUT);n Init();n connectToGPRS();n connectTohttps();n}
This code initializes serial communication, resets the EC25 module, initialize the and establishes a GPRS connection. Then it connects to the GitHub through the https protocol. So, basically this is the setup of the program.
- Part 4
void loop() {n if (checkForUpdate(firmware_url)) {n performOTA(firmware_url);n }n delay(1000);n //add the code need to run and this is an example programn //loop to blink without delayn unsigned long currentMillis = millis();n if (currentMillis - previousMillis u003e= interval) {n // save the last time you blinked the LEDn previousMillis = currentMillis;n // if the LED is off turn it on and vice-versa:n ledState = not(ledState);n // set the LED with the ledState of the variable:n digitalWrite(led1, ledState);n digitalWrite(led2, ledState);n }n}n
This code loop() function continuously checks for firmware updates by calling the check ForUpdate() function. If an update is available, the performOTA() function is executed to download and install the new firmware. A delay of 1000 milliseconds is then added to prevent excessive polling. The code also includes an example program that blinks two LEDs without using delays, demonstrating the basic structure for adding other functionalities to the main loop.
- Part 5
void Init(void) { // Connecting with the network and GPRSn delay(5000);n gsm_send_serial(u0022AT+CFUN=1u0022, 10000);n gsm_send_serial(u0022AT+CPIN?u0022, 10000);n gsm_send_serial(u0022AT+CSQu0022, 1000);n gsm_send_serial(u0022AT+CREG?u0022, 1000);n gsm_send_serial(u0022AT+COPS?u0022, 1000);n gsm_send_serial(u0022AT+CGATT?u0022, 1000);n gsm_send_serial(u0022AT+CPSI?u0022, 500);n String cmd = u0022AT+CGDCONT=1,"IP","u0022 + String(apn) + u0022"u0022;n gsm_send_serial(cmd, 1000);n gsm_send_serial(u0022AT+CGACT=1,1u0022, 1000);n gsm_send_serial(u0022AT+CGATT?u0022, 1000);n gsm_send_serial(u0022AT+CGPADDR=1u0022, 500);n}nvoid connectToGPRS(void) {n gsm_send_serial(u0022AT+CGATT=1u0022, 1000);n String cmd = u0022AT+CGDCONT=1,"IP","u0022 + String(apn) + u0022"u0022;n gsm_send_serial(cmd, 1000);n gsm_send_serial(u0022AT+CGACT=1,1u0022, 1000);n gsm_send_serial(u0022AT+CGPADDR=1u0022, 500);n}nvoid connectTohttps(void) {n int cert_length = root_ca.length();n String ca_cert = u0022AT+QFUPL="RAM:github_ca.pem",u0022 + String(cert_length) + u0022,100u0022;n gsm_send_serial(ca_cert, 1000);n delay(1000);n gsm_send_serial(root_ca, 1000);n delay(1000);n gsm_send_serial(u0022AT+QHTTPCFG="contextid",1u0022, 1000);n gsm_send_serial(u0022AT+QHTTPCFG="responseheader",1u0022, 1000);n gsm_send_serial(u0022AT+QHTTPCFG="sslctxid",1u0022, 1000);n gsm_send_serial(u0022AT+QSSLCFG="sslversion",1,4u0022, 1000);n gsm_send_serial(u0022AT+QSSLCFG="ciphersuite",1,0xC02Fu0022, 1000);n gsm_send_serial(u0022AT+QSSLCFG="seclevel",1,1u0022, 1000);n gsm_send_serial(u0022AT+QSSLCFG="sni",1,1u0022, 1000);n gsm_send_serial(u0022AT+QSSLCFG="cacert",1,"RAM:github_ca.pem"u0022, 1000);n}n
Network Initialization and GPRS Activation (Init and connectToGPRS)
- These functions initialize the EC25 modem, check SIM and network registration (AT+CPIN?, AT+CREG?), and activate GPRS by configuring and enabling the PDP context using:
- AT+CGDCONT=1,”IP”,”<APN>” – Sets the APN for internet access.
- AT+CGACT=1,1 – Activates the PDP context.
- AT+CGATT=1 – Attaches the device to the GPRS service.
- AT+CGPADDR=1 – Retrieves the assigned IP address
SSL Certificate Upload and HTTPS Configuration (connectTohttps)
- Uploads the root CA certificate (stored as a string root_ca) to the EC25’s RAM using:
- AT+QFUPL=”RAM:github_ca.pem”,<length>,100 followed by the actual certificate content.
- Configures HTTPS and SSL settings:
- Binds the HTTPS client to PDP context 1 and SSL context 1.
- Enables response headers, SNI (Server Name Indication), and sets TLS version (version 1.2).
- Specifies cipher suite and security level.
- Associates the uploaded CA cert to the SSL context using:
- AT+QSSLCFG=”cacert”,1,”RAM:github_ca.pem”.
Separation of Logic for Modular OTA Communication
- Code is cleanly separated into:
- Init() → handles initial modem and GPRS setup after boot.
- connectToGPRS() → Connect to GPRS.
- connectTohttps() → sets up HTTPS with SSL using a root certificate.
- Part 6
// Check the version of the latest firmware uploaded to GitHubnbool checkForUpdate(String u0026firmware_url) {n Serial.println(u0022Making GET request securely...u0022);n // Ensure PDP context is active (optional but recommended)n gsm_send_serial(u0022AT+QIACT?u0022, 1000);n delay(100);n gsm_send_serial(u0022AT+QIACT=1u0022, 2000); // Reactivate if neededn delay(300);n // Clear SerialAT buffern while (SerialAT.available()) SerialAT.read();n // Step 1: Prepare the URLn gsm_send_serial(u0022AT+QHTTPURL=u0022 + String(version_url.length()) + u0022,80u0022, 1000);n delay(200);n gsm_send_serial(version_url, 2000);n bool got200 = false;n int contentLen = -1;n // Flush againn while (SerialAT.available()) SerialAT.read();n // Step 2: Send HTTP GETn Serial.println(u0022Send -u003e: AT+QHTTPGET=80u0022);n SerialAT.println(u0022AT+QHTTPGET=80u0022);n unsigned long startTime = millis();n const unsigned long httpTimeout = 8000; // Wait max 8sn String qhttpgetLine = u0022u0022;n while (millis() - startTime u003c httpTimeout) {n if (SerialAT.available()) {n String line = SerialAT.readStringUntil('
');n line.trim();n if (line.length() == 0) continue;n Serial.println(u0022[Modem Line] u0022 + line);n if (line.startsWith(u0022+QHTTPGET:u0022)) {n qhttpgetLine = line;n break;n }n if (line.indexOf(u0022+QIURC: "pdpdeact"u0022) u003e= 0) {n Serial.println(u0022[OTA] PDP deactivated! Reconnecting...u0022);n gsm_send_serial(u0022AT+QIACT=1u0022, 3000);n return false;n }n }n }n if (qhttpgetLine.length() == 0) {n Serial.println(u0022[OTA] HTTP GET response not received.u0022);n return false;n }nn // Step 3: Parse +QHTTPGET: 0,u003cstatusu003e,u003clenu003en int idx1 = qhttpgetLine.indexOf(',');n int idx2 = qhttpgetLine.indexOf(',', idx1 + 1);n if (idx1 == -1 || idx2 == -1) {n Serial.println(u0022[OTA] Malformed +QHTTPGET responseu0022);n return false;n }n int statusCode = qhttpgetLine.substring(idx1 + 1, idx2).toInt();n contentLen = qhttpgetLine.substring(idx2 + 1).toInt();n if (statusCode == 200 u0026u0026 contentLen u003e 0) {n got200 = true;n Serial.println(u0022[OTA] HTTP 200 OK. Content length: u0022 + String(contentLen));n } else {n Serial.println(u0022[OTA] HTTP GET failed. Status: u0022 + String(statusCode));n return false;n }n delay(300); // Give modem time to buffern // Step 4: Issue QHTTPREADn Serial.println(u0022Send -u003e: AT+QHTTPREAD=u0022 + String(contentLen));n SerialAT.println(u0022AT+QHTTPREAD=u0022 + String(contentLen));n // Step 5: Wait for CONNECTn bool gotConnect = false;n unsigned long waitStart = millis();n while (millis() - waitStart u003c 3000) {n if (SerialAT.available()) {n String line = SerialAT.readStringUntil('
');n line.trim();n Serial.println(u0022[Modem Line] u0022 + line);n if (line.startsWith(u0022CONNECTu0022)) {n gotConnect = true;n break;n }n }n }n if (!gotConnect) {n Serial.println(u0022[OTA] Failed to get CONNECT, abortingu0022);n return false;n }n // Step 6: Skip headers and extract the versionn String version = u0022u0022;n bool foundEmptyLine = false;n unsigned long readTimeout = millis() + 5000;n while (millis() u003c readTimeout) {n if (SerialAT.available()) {n String line = SerialAT.readStringUntil('
');n line.trim();n if (line.length() == 0) {n foundEmptyLine = true; // Found end of headersn continue;n }n if (foundEmptyLine) {n version = line; // First non-empty line after headersn break;n }n Serial.println(u0022[Header] u0022 + line); // Optional loggingn readTimeout = millis() + 1000; // Extend timeout while readingn }n }n version.trim();n Serial.println(u0022Extracted version: u0022 + version);n if (version.length() == 0) {n Serial.println(u0022[OTA] Failed to extract version string.u0022);n return false;n }n // Set global new_versionn new_version = version;n // Step 7: Compare and set firmware URLn if (new_version != current_version) {n Serial.println(u0022New version available. Updating...u0022);n firmware_url = u0022https://raw.githubusercontent.com/IndustrialArduino/OTA-on-ESP/release/firmware_vu0022 + new_version + u0022.binu0022;n Serial.println(u0022Firmware URL: u0022 + firmware_url);n return true;n } else {n Serial.println(u0022Already on latest version.u0022);n return false;n }n}n
This code checks for updates by making a GET request to the GitHub repository and extracting the latest version number from the response. It compares the current version with the available version and updates the firmware URL if a new version is found. The code also prints relevant information to the serial monitor, including the status code, response body, and current and available versions.
- Part 7
void performOTA(String firmware_url) {n gsm_send_serial(u0022AT+QHTTPURL=u0022 + String(firmware_url.length()) + u0022,80u0022, 1000);n delay(100);n gsm_send_serial(firmware_url, 2000);n gsm_send_serial(u0022AT+QHTTPGET=80u0022, 1000);n Serial.println(u0022[OTA] Waiting for +QHTTPGET response...u0022);n long contentLength = -1;n unsigned long timeout = millis();n while (millis() - timeout u003c 5000) {n if (SerialAT.available()) {n String line = SerialAT.readStringUntil('
');n line.trim();n if (line.length() == 0) continue;n Serial.println(u0022[Modem Line] u0022 + line);n if (line.startsWith(u0022+QHTTPGET:u0022)) {n int firstComma = line.indexOf(',');n int secondComma = line.indexOf(',', firstComma + 1);n if (firstComma != -1 u0026u0026 secondComma != -1) {n String lenStr = line.substring(secondComma + 1);n contentLength = lenStr.toInt();n Serial.print(u0022[OTA] Content-Length: u0022);n Serial.println(contentLength);n }n }n if (line == u0022OKu0022) break;n }n delay(10);n }n Serial.println(u0022[OTA] HTTPS GET sentu0022);n // Save response to RAM filen gsm_send_serial(u0022AT+QHTTPREADFILE="RAM:firmware.bin",80u0022, 1000);n // Wait for final confirmation and avoid overlapn unsigned long readfileTimeout = millis();n while (millis() - readfileTimeout u003c 5000) {n if (SerialAT.available()) {n String line = SerialAT.readStringUntil('
');n line.trim();n if (line.length() == 0) continue;n Serial.println(u0022[READFILE] u0022 + line);n if (line.startsWith(u0022+QHTTPREADFILE:u0022)) break;n }n delay(10);n }nn // Clear SerialAT buffern while (SerialAT.available()) SerialAT.read();n // Send QFLST directlyn SerialAT.println(u0022AT+QFLST="RAM:firmware.bin"u0022);n long ramFileSize = 0;n timeout = millis();n while (millis() - timeout u003c 5000) {n if (SerialAT.available()) {n String line = SerialAT.readStringUntil('
');n line.trim();n if (line.length() == 0) continue;n Serial.println(u0022[OTA Raw] u0022 + line);n // Find +QFLST linen if (line.startsWith(u0022+QFLST:u0022)) {n int commaIdx = line.lastIndexOf(',');n if (commaIdx != -1) {n String sizeStr = line.substring(commaIdx + 1);n sizeStr.trim();n ramFileSize = sizeStr.toInt();n break;n }n }n }n delay(10);n }n Serial.println(u0022[OTA] File size: u0022 + String(ramFileSize));n if (ramFileSize u003c= 0) {n Serial.println(u0022[OTA] ERROR: Invalid file size.u0022);n return;n }n int headerSize = ramFileSize - contentLength;n if (headerSize u003c= 0 || headerSize u003e ramFileSize) {n Serial.println(u0022[OTA] Invalid header size!u0022);n return;n }n Serial.println(u0022[OTA] Header size: u0022 + String(headerSize));n // Clear SerialAT buffer before commandn while (SerialAT.available()) SerialAT.read();n // Send QFOPEN directlyn SerialAT.println(u0022AT+QFOPEN="RAM:firmware.bin",0u0022);n int fileHandle = -1;n unsigned long handleTimeout = millis();n while (millis() - handleTimeout u003c 5000) {n if (SerialAT.available()) {n String line = SerialAT.readStringUntil('
');n line.trim();n if (line.length() == 0) continue;n Serial.println(u0022[OTA Raw] u0022 + line);nn if (line.startsWith(u0022+QFOPEN:u0022)) {n String handleStr = line.substring(line.indexOf(u0022:u0022) + 1);n handleStr.trim();n fileHandle = handleStr.toInt();n break;n }n }n delay(10);n }n Serial.println(u0022[OTA] File handle: u0022 + String(fileHandle));n if (fileHandle u003c= 0) {n Serial.println(u0022[OTA] ERROR: Invalid file handle.u0022);n return;n }n // Seek to payloadn gsm_send_serial(u0022AT+QFSEEK=u0022 + String(fileHandle) + u0022,u0022 + String(headerSize) + u0022,0u0022, 1000);n delay(300);n // Step 7: Begin OTAn if (!Update.begin(contentLength)) {n Serial.println(u0022[OTA] Update.begin failedu0022);n return;n }n Serial.println(u0022[OTA] Start writing...u0022);nnn size_t chunkSize = 1024;n size_t totalWritten = 0;n uint8_t buffer[1024];n while (totalWritten u003c contentLength) {n size_t bytesToRead = min(chunkSize, (size_t)(contentLength - totalWritten));n SerialAT.println(u0022AT+QFREAD=u0022 + String(fileHandle) + u0022,u0022 + String(bytesToRead));n // Wait for CONNECT (start of binary data)n bool gotConnect = false;n unsigned long startWait = millis();n while (millis() - startWait u003c 2000) {n if (SerialAT.available()) {n String line = SerialAT.readStringUntil('
');n line.trim();n if (line.startsWith(u0022CONNECTu0022)) {n gotConnect = true;n break;n }n }n delay(1);n }n if (!gotConnect) {n Serial.println(u0022[OTA] Failed to get CONNECTu0022);n Update.abort();n return;n }nn // Read exactly bytesToRead bytes of binary datan size_t readCount = 0;n unsigned long lastReadTime = millis();n while (readCount u003c bytesToRead u0026u0026 millis() - lastReadTime u003c 3000) {n if (SerialAT.available()) {n buffer[readCount++] = (uint8_t)SerialAT.read();n lastReadTime = millis();n } else {n delay(1);n }n }n if (readCount != bytesToRead) {n Serial.println(u0022[OTA] Incomplete read from modemu0022);n Update.abort();n return;n }nn // After reading data, wait for the final OKn bool gotOK = false;n startWait = millis();n while (millis() - startWait u003c 2000) {n if (SerialAT.available()) {n String line = SerialAT.readStringUntil('
');n line.trim();n if (line == u0022OKu0022) {n gotOK = true;n break;n }n }n delay(1);n }n if (!gotOK) {n Serial.println(u0022[OTA] Did not receive final OK after datau0022);n Update.abort();n return;n }nn // Write to flashn size_t written = Update.write(buffer, readCount);n if (written != readCount) {n Serial.println(u0022[OTA] Flash write mismatchu0022);n Update.abort();n return;n }nn totalWritten += written;n Serial.printf(u0022
[OTA] Progress: %u / %u bytesu0022, (unsigned)totalWritten, (unsigned)contentLength);n }nn Serial.println(u0022
[OTA] Firmware write complete.u0022);n // Close the filen SerialAT.println(u0022AT+QFCLOSE=u0022 + String(fileHandle));n delay(500);n // Finalize OTA updaten if (Update.end()) {n Serial.println(u0022[OTA] Update successful!u0022);n if (Update.isFinished()) {n Serial.println(u0022[OTA] Rebooting...u0022);n delay(300);n ESP.restart();n } else {n Serial.println(u0022[OTA] Update not finished!u0022);n }n } else {n Serial.println(u0022[OTA] Update failed with error: u0022 + String(Update.getError()));n }n}n
1. Download Firmware to RAM File
- Performs HTTPS GET of the .bin file and saves it to RAM:firmware.bin using AT+QHTTPREADFILE.
- Parses the response to get content length, file size, and calculates header size for skipping HTTP headers.
2. Flash Firmware to ESP32
- Opens the RAM file, skips headers using AT+QFSEEK, and reads firmware in 1024-byte chunks with AT+QFREAD.
- Streams each chunk to Update.write(). Once complete, closes file and finalizes OTA with Update.end() and restarts ESP if successful.
How OTA Update work #
1.Version Check: The ESP32 sends a request to the server (GitHub) to check the latest firmware version by fetching a simple text file version.txt.
- Current version on ESP32: 1.0.0.
- Latest version from the server: e.g., 1.1.0.
2.Download Firmware: If a newer version is detected, the firmware binary is downloaded securely via HTTPS. The firmware file is hosted at a URL like:https://github.com/IndustrialArduino/OTA-on-ESP/tree/release
3.Perform OTA Update: The downloaded binary is written to the ESP32’s memory. After a successful update, the ESP32 will restart automatically and run the new firmware.
Testing setup #
1.Upload the Initial Firmware
- Compile and upload the firmware to the ESP32 with the initial version (1.0.0).
- Verify that the device is connected to the GSM network.
2.Host a New Firmware
- After making changes to the code export the code as binary and then rename the binary file
- as the new version of the firmware (e.g., 1.1.0) and upload the binary file to your server.
- Update the version.txt file to reflect the new version number.
- Monitor the Update
- Open the Serial Monitor at 115200 baud.
The ESP32 will: - Connect to the GSM network.
- Fetch the latest version from the server.
- Compare the versions.
- If an update is available, it will download the firmware and install it.
- Reboot the device with the updated firmware.
3.Expected Output:
- ESP32 logs network connection and update status.
- If an update is available, the new firmware is downloaded and applied.
Making GET request securely...n[OTA] HTTP 200 OK. Content length: 112345n[OTA] Waiting for +QHTTPGET response...n[OTA] File size: 113123n[OTA] Header size: 778n[OTA] Start writing...n[OTA] Progress: 112345 / 112345 bytesn[OTA] Firmware write complete.n[OTA] Update successful!n[OTA] Rebooting...n
Error Handling #
Connection Failure: Ensure that the APN, username, and password are correct for your SIM card provider. Check for proper power supply to the Quetcel EC25 module
• HTTPS Failures: If there are SSL/HTTPS errors, verify that the root CA certificate is properly loaded in the cert.h file.
• OTA Failures: If the update fails, the device will continue running the current version. Ensure that the binary file is correct and available at the specified URL.
Troubleshooting #
ESP32 Doesn’t Connect to Cellular Network:
- Double-check the wiring between the Quetcel EC25 and the ESP32.
- Make sure the SIM card has an active data plan and is properly inserted.
OTA Update Fails:
- Check the size of the binary file and ensure there’s enough memory on the ESP32 for the update.
- Ensure the server is reachable, and the firmware URL is correct.
HTTPS Request Fails:
- Confirm that the root CA is correctly added for HTTPS communication.
- Check if the GitHub URL is accessible and the server isn’t blocking your requests.