Build a 5$ IoT Thing with ESP8266, Mongoose OS and AWS IoT Core

Categories IoT, Mongoose OS

ESP8266 based development boards are available for as little as $3. The great thing about the ESP8266 is that it packs enough punch to handle the crypto required to work with an IoT backend like AWS IoT Core. In this video, I explain a full, end-to-end IoT project where we will hook up the ESP8266 to AWS IoT Core. We will be running Mongoose OS on the ESP8266. Since the ESP8266 has WiFi hardware built in and can be operated off a USB power bank, some very interesting possibilities emerge.

How it works

We connect a DHT11 temperature and humidity sensor to the ESP8266, get those values every 5 minutes and ship them off using MQTT to AWS IoT Core. There, we use AWS Lambda to take those readings and store them in AWS S3. In S3, we will also have a simple HTML file that will chart the temperature and humidity data for the past 24 hours.

Temperature / humidity chart

Some notes on the ESP8266

When someone refers to the ESP8266, they are referring to the module that contains the core chip with WiFi functionality. However, that is hardly usable. What you’ll find on the market however, are development boards built around the ESP8266, like the NodeMCU boards. Many online vendors will sell these boards online calling them NodeMCU, while NodeMCU is technically the software solution. These development boards have the core ESP8266 module mounted on a PCB with headers that make it easy for you to develop on the platform. These boards also have a serial chip, a micro-USB connector and a couple of push-buttons thrown in. If you’re having any confusion buying the board, just buy a NodeMCU compatible board.

Prerequisites

There are a few things you need to know and there are other things you need to setup before you can build this thing. Here is a list:

  • This tutorial is in C. You need to know the basics to understand what is going on.
  • Although I’ll be using the Mongoose OS, there is nothing much to really put lots of effort on learning it, since the developers have put in loads of effort making it easy to understand. They’ve also provided a great API and configuration system making it easy for anyone with the bit of C or Javascript knowledge to get started quickly.
  • The list of parts you’ll need are the ESP8266 development board like the NodeMCU, the DHT11, 3 female-to-female wires to connect the DHT11 to the ESP8266 and a micro-USB cable to connect the ESP8266 to your computer.
  • Head over to the Mongoose OS website and ensure that you are able to run their “mos” command line utility without issues.
  • Ensure that you have the right serial driver installed for the serial chip that is on your ESP8266 development board. Look behind the board and you might find clues about the serial chip you might need. Watch the video for more details.
  • You need an account with AWS
  • You need to setup a IAM “Role” that has full rights to AWS IoT Core.
  • This user credentials then need to be setup on your machine along with AWS command line utilities

You can find a lot of resources online to make sure that all prerequisites are satisfied. Contact me in the comments if you’re having trouble.

Schematic

DHT11 connections

Pin #1: Vcc

Pin #2: Data (Connected to D1 on the board)

Pin #3: Not connected

Pin #4: Ground

Notes on working with Mongoose OS

Clone an empty Mongoose OS project

$ git clone https://github.com/mongoose-os-apps/empty temp_humidity

See below configuration files mos.yml and watch the video for more details.
Once you have src/main.c edited, you can trigger a build by doing:

$ mos build --arch esp8266

Once the build is successful, you can flash your ESP8266

$ mos flash

To view the device’s serial console, you do:

$ mos console

Connecting the ESP8266 to Wifi:

$ mos wifi [your-wifi-ssid] [your-wifi-password-here]

You can provision an device on AWS IoT core with the following Mongoose OS command. However, you should have setup your AWS CLI tools properly for this to work.

$ mos aws-iot-setup

Mongoose OS project configuration file

author: mongoose-os
description: A Mongoose OS app skeleton
version: 1.0

libs_version: ${mos.version}
modules_version: ${mos.version}
mongoose_os_version: ${mos.version}

# Optional. List of tags for online search.
tags:
 - c

# List of files / directories with C sources. No slashes at the end of dir names.
sources:
 - src

# List of dirs. Files from these dirs will be copied to the device filesystem
filesystem:
 - fs

# Custom configuration entries, settable via "device configuration"
# Below is a custom firmware configuration example.
# Uncomment and modify according to your needs:

# config_schema:
#  - ["my_app", "o", {title: "My app custom settings"}]
#  - ["my_app.bool_value", "b", false, {title: "Some boolean value"}]
#  - ["my_app.string_value", "s", "", {title: "Some string value"}]
#  - ["my_app.int_value", "i", 123, {title: "Some integer value"}]

# These settings get compiled into the C structure, and can be accessed
# from the C code this way:
#
# printf("Hello from %s!\n", mgos_sys_config_get_device_id());
#
# Settings are cool: can be modified remotely without full firmware upgrade!
#
# To see all available compiled settings, buid the firmware and open
# build/gen/mgos_config.h file.
#
# Also, in this config_schema section, you can override existing
# settings that has been created by other libraries. For example, debug log
# level is 2 by default. For this firmware we can override it to 3:
#
# config_schema:
#  - ["debug.level", 3]

# List of libraries used by this app, in order of initialisation
libs:
 - origin: https://github.com/mongoose-os-libs/ca-bundle
 - origin: https://github.com/mongoose-os-libs/aws
 - origin: https://github.com/mongoose-os-libs/ca-bundle
 - origin: https://github.com/mongoose-os-libs/dash
 - origin: https://github.com/mongoose-os-libs/http-server
 - origin: https://github.com/mongoose-os-libs/ota-shadow
 - origin: https://github.com/mongoose-os-libs/ota-http-client
 - origin: https://github.com/mongoose-os-libs/ota-http-server
 - origin: https://github.com/mongoose-os-libs/rpc-service-config
 - origin: https://github.com/mongoose-os-libs/rpc-service-fs
 - origin: https://github.com/mongoose-os-libs/rpc-uart
 - origin: https://github.com/mongoose-os-libs/shadow
 - origin: https://github.com/mongoose-os-libs/sntp
 - origin: https://github.com/mongoose-os-libs/wifi
 - origin: https://github.com/mongoose-os-libs/dht

# Used by the mos tool to catch mos binaries incompatible with this file format
manifest_version: 2017-05-18

 

Project Source Code

#include "mgos.h"
#include "mgos_mqtt.h"
#include "mgos_dht.h"

static struct mgos_dht *s_dht = NULL;

static void led_timer_cb(void *arg) {
  /*bool val = mgos_gpio_toggle(2);*/
  bool val = 1;
  LOG(LL_INFO, ("%s uptime: %.2lf, RAM: %lu, %lu free", val ? "Tick" : "Tock",
                mgos_uptime(), (unsigned long) mgos_get_heap_size(),
                (unsigned long) mgos_get_free_heap_size()));
  (void) arg;
}

static void net_cb(int ev, void *evd, void *arg) {
  switch (ev) {
    case MGOS_NET_EV_DISCONNECTED:
      LOG(LL_INFO, ("%s", "Net disconnected"));
      break;
    case MGOS_NET_EV_CONNECTING:
      LOG(LL_INFO, ("%s", "Net connecting..."));
      break;
    case MGOS_NET_EV_CONNECTED:
      LOG(LL_INFO, ("%s", "Net connected"));
      break;
    case MGOS_NET_EV_IP_ACQUIRED:
      LOG(LL_INFO, ("%s", "Net got IP address"));
      break;
  }

  (void) evd;
  (void) arg;
}

static void report_temperature(void *arg) {
  char topic[100], message[160];
  struct json_out out = JSON_OUT_BUF(message, sizeof(message));
  
  time_t now=time(0);
  struct tm *timeinfo = localtime(&now);

  snprintf(topic, sizeof(topic), "event/temp_humidity");
  json_printf(&out, "{total_ram: %lu, free_ram: %lu, temperature: %f, humidity: %f, device: \"%s\", timestamp: \"%02d:%02d:%02d\"}",
              (unsigned long) mgos_get_heap_size(),
              (unsigned long) mgos_get_free_heap_size(),
              (float) mgos_dht_get_temp(s_dht), 
              (float) mgos_dht_get_humidity(s_dht),
              (char *) mgos_sys_config_get_device_id(),
              (int) timeinfo->tm_hour,
              (int) timeinfo->tm_min,
              (int) timeinfo->tm_sec);
  bool res = mgos_mqtt_pub(topic, message, strlen(message), 1, false);
  LOG(LL_INFO, ("Published to MQTT: %s", res ? "yes" : "no"));
  (void) arg;
}

static void button_cb(int pin, void *arg) {
  float t = mgos_dht_get_temp(s_dht);
  float h = mgos_dht_get_humidity(s_dht);
  LOG(LL_INFO, ("Button presses on pin: %d", pin));
  LOG(LL_INFO, ("Temperature: %f *C Humidity: %f %%\n", t, h));
  
  report_temperature(NULL);
  (void) arg;
}

enum mgos_app_init_result mgos_app_init(void) {
  /* Blink built-in LED every second */
  mgos_gpio_set_mode(2, MGOS_GPIO_MODE_OUTPUT);
  mgos_set_timer(1000, MGOS_TIMER_REPEAT, led_timer_cb, NULL);
  
  /* Report temperature to AWS IoT Core every 5 mins */
  mgos_set_timer(300000, MGOS_TIMER_REPEAT, report_temperature, NULL);

  /* Publish to MQTT on button press */
  mgos_gpio_set_button_handler(0,
                               MGOS_GPIO_PULL_UP, MGOS_GPIO_INT_EDGE_NEG, 200,
                               button_cb, NULL);
                               
  if ((s_dht = mgos_dht_create(5, DHT11)) == NULL) {
    LOG(LL_INFO, ("Unable to initialize DHT11"));
  }

  /* Network connectivity events */
  mgos_event_add_group_handler(MGOS_EVENT_GRP_NET, net_cb, NULL);

  return MGOS_APP_INIT_SUCCESS;
}

HTML to display temperature and humidity chart for the past 24 hours

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Temperature and Humidity coming from an ESP8266</title>

    <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.4.0/Chart.min.js"></script>
    <script src="https://code.jquery.com/jquery-3.3.1.min.js"></script>
    <style>
        body {
            width: 1200px;
            font-family: sans-serif;
            margin: auto;
            color: #555;
        }
    </style>
</head>

<body>
    <h1>Temperature and Humidity coming from an ESP8266</h1>
    <h2 id="latest-measurements"></h2>

    <canvas id="myChart"></canvas>
    <script>
        var ctx = document.getElementById('myChart').getContext('2d');
        var chart = new Chart(ctx, {
            // The type of chart we want to create
            type: 'line',

            // The data for our dataset
            data: {
                labels: [],
                datasets: [{
                    label: "Temperature",
                    borderColor: 'rgb(255, 99, 132)',
                    data: [],
                }, {
                    label: "Humidity",
                    borderColor: 'rgb(54, 162, 235)',
                    data: [],
                }]
            },

            // Configuration options go here
            options: {
                scales: {
                    yAxes: [{
                        ticks: {
                            beginAtZero: true
                        }
                    }]
                }
            }
        });

        $.getJSON("https://s3-ap-southeast-1.amazonaws.com/org.itywik.iot-devices/esp8266_4038B7", function(data) {
            chart.data.datasets[0].data = data.temperature_readings;
            chart.data.datasets[1].data = data.humidity_readings;
            chart.data.labels = data.timestamps;
            chart.update();

            l_temp = data.temperature_readings;
            l_temp = l_temp[l_temp.length - 1];
            l_humid = data.humidity_readings;
            l_humid = l_humid[l_humid.length - 1];

            $('#latest-measurements').html('Latest readings: Temperature:' + l_temp + '&deg; C / Humidity:' + l_humid);
        });
    </script>
</body>

</html>

Here is the AWS Lambda function code you’ll need

You’ll need to ensure that BUCKET_BASE is set to a proper bucket name. Create the bucket and update its name.

import json
import boto3

BUCKET_BASE     = 'name_of_your_s3_bucket'
TEMP_FILE       = '/tmp/temperatures.json'
MAX_READINGS    = int((60/5)*24) # one reading every 5 mins for 24 hours
MAX_READINGS    = MAX_READINGS - 1 # make it easy for calculations

def get_existing_values(device):
    s3 = boto3.resource('s3')
    existing_values = {"temperature_readings": [], "humidity_readings": [], "timestamps": []}
    try:
        s3.Bucket(BUCKET_BASE).download_file(device, TEMP_FILE)
        f = open(TEMP_FILE)
        existing_values = json.loads(f.read())
        f.close()
    except Exception as e:
        if e.response['Error']['Code'] == '404':
            print('No previous record for this device.')
        else:
            raise(e)

    return existing_values

def append_values_to_s3_object(device, existing_values, values_to_append):
    # Let's not store more than MAX_READINGS readings. Trim oldest if necessary.
    t_len = len(existing_values["temperature_readings"])
    if t_len > MAX_READINGS:
        print(t_len, MAX_READINGS, t_len-MAX_READINGS)
        existing_values["temperature_readings"] = existing_values["temperature_readings"][t_len-MAX_READINGS:]

    h_len = len(existing_values["humidity_readings"])
    if h_len > MAX_READINGS:
        existing_values["humidity_readings"] = existing_values["humidity_readings"][h_len-MAX_READINGS:]

    t_len = len(existing_values["timestamps"])
    if t_len > MAX_READINGS:
        existing_values["timestamps"] = existing_values["timestamps"][t_len-MAX_READINGS:]



    existing_values["temperature_readings"].append(values_to_append["temperature"])
    existing_values["humidity_readings"].append(values_to_append["humidity"])
    existing_values["timestamps"].append(values_to_append["timestamp"])
    print(existing_values)
    s3 = boto3.resource('s3')
    s3.Bucket(BUCKET_BASE).put_object(Body = json.dumps(existing_values), Key = device, ACL='public-read')

def lambda_handler(event, context):
    print('Event details:', event)
    device = event['device']
    existing_values = get_existing_values(device)
    print('Existing values:', existing_values)
    append_values_to_s3_object(device, existing_values, event)