PM2.5 Monitor

Log and Transfer Data

A few options are:

  • Turn ESP8266 to a webserver, check tttapa's github, an excellent guide to see how. Seriously, you should check this out. The only limitation is 1-2MB available for file storage on 4MB-flash ESP8266
  • ESP8266 with SD card module, and write file directly to the SD card. You have option to write lot of data.
  • Transfer data to a server via MQTT. This option requries more setup, more time, but you could do more.
  • Use public/free/open MQTT server such as HiveMQ.com. This is a good option for experimenting and learning. The trade-offs are anyone can listen to your data; others can acidentally write to your topic.

4.1 How to do it?

To transfer data, we need more libraries and setup.

  • First, data from ESP32/8266 sent to MQTT server using PubSubClient. This library allows to push (publish) or listen (subscribe) to a (group) topic on MQTT server.
  • Second, an app either on desktop or web-based to listen to the data in the MQTT server's buffer. We will use MQTT.fx (available on Windows, Linux, and MacOS) to listen to data. The data can be saved to local files by MQTT.fx

First, let explore HiveMQ.com of how to use this open, free MQTT server to send and listen to the data from the sensor. Before that, let prepare a sketch for ESP8266.

To transfer data from ESP8266/ESP32 to MQTT server, the data have to converted to array of chars or deserialize. If the data is built from concatinating from a String object, convert String to array is simple. Check out this link to see how String works. Using this approach, it is cumbersome to add name of each data to the string. Often, each field of data is seperated by a comma, and if you have several data fields, you need to make sure all fields are reported, in order.

PMS7003 dust sensor measured PM2.5 concentration plus PM1 and PM10 under a factory condition and an atmospheric condition. In addition, the number of particles in beyond (larger) than 0.3, 0.5, 1.0, 2.5, 5.0 and 10 µm/m3 per 0.1L of air. That is a lot of data. The accuracy of the counting is a big question, and it is hard to verify the numbers. For the moment, let assume that we want to capture all data from PMS7003. These specifications can be found on PMS7003 datasheet.

1. Include libraries

// Additional libraries
#include <PubSubClient.h>  // MQTT communication
#include <ArduinoJson.h>    // add data to JSON object 

2. Define MQTT server, topics

// Additional libraries
#define SENSORNAME "pms7003"
#define wifi_ssid "wifi_ssid"
#define wifi_password "wifi_password"
#define mqtt_server "broker.hivemq.com" // MQTT server address
#define publish_topic "sensors/pms7003"
#define subcribe_set "sensors/pms7003/set"
// Inside SETUP loop
client.setServer(mqtt_server, 1883);
client.setCallback(callback);

3. And declare some variables to hold the values of data stream

// Declare variables
uint16_t pm1_0_ac, pm2_5_ac,pm10_ac;
uint16_t pm1_0, pm2_5,pm10;
uint32_t um03;
uint16_t um05, um1, um2_5, um5, um10;

4. And in reading PMS function, we unpack all data from a struct created by pms.h library.

// Inside reading PMS7003 function
pm1_0 = data[0];
pm2_5 = data[1];
pm10= data[2];
pm1_0_ac = data[3];
pm2_5_ac = data[4];
pm10_ac = data[5];
um03 = data[6];
um05 = data[7];
um1 = data[8];
um2_5 = data[9];
um5 = data[10];
um10 = data[11];

You can tell that with all data from PMS7003, it is confusing to log data in a string format. Without a proper header and seperator, you are likely to count the comma a lots to know which number maps to which parameter. And now ArduinoJson comes to your rescuse. It creates a JSON object and converts a nested object (which JSON is) to a flat object (a char arrays).

5. Pack the data to a JSON object

//Inside compose data function
DynamicJsonDocument doc(512);  //created an JSON object with 512bytes
doc["sensor"] = SENSORNAME;
doc["uptime"] = uptime;
doc["pm1_0"] = pm1_0;
doc["pm2_5"]= pm2_5;
doc["pm10"] = pm10;
doc["pm1_0_ac"] = pm1_0_ac;
doc["pm2_5_ac"]= pm2_5_ac;
doc["pm10_ac"] = pm10_ac;
doc["um03"] = um03;
doc["um05"] = um05;
doc["um1"] = um1;
doc["um2_5"] = um2_5;
doc["um5"] = um5;
doc["um10"] = um10;
doc["sleep"] = __sleep;
doc["type"] = "json";

6. Convert JSON object to an array

For PubSubClient to transmit this data, it has to converted to a char array. For ArduinoJSON 6.x, here is how to do it.

// Convert JSON object to a char array
size_t len = measureJson(doc)+ 1;
char payloads[len];
serializeJson(doc, payloads, sizeof(payloads))

And publish (send) the char array to MQTT server.

// inside compose_data() function
if (!client.connected()) {
    reconnect();
    delay(1000);
}
if (client.publish(publish_topic, payloads, false)){
    Serial.println("Success: " + String(payloads));
} else {
    Serial.println("Failed to push: " + String(payloads));
}

And that's it. We need to work on MQTT server to make sure that we can capture data.

4.2 How to know that it looks correct?

1. Setup MQTT.fx. Alternatively, a web-based MQTT listener is available by HiveMQ

mqtt-fx

2. If everything is setup correctly,

working

3. The data could be dumpted to .bin file and open by a text editor

working

...and you can tell, that from the dashboard of HiveMQ server, there is a lot of traffics on the server (Oct 08, 2020)

hivemq

One distinct benefit using a clound HiveMQ is that you can push and listen the data everywhere, not limited to your home gateway. Alternatively, you could buy $5 a month server to install MQTT server or a droplet that have MQTT installed already. We will explore next how to transfer data a bit more securely.