Real-time Websocket Connection between Sensor Reading ESP8266 and Node.JS Server

About Websockets

The WebSocket protocol is an extension to the HTTP protocol, and allows to create real-time connections between web servers and a clients. Here, a client can be a web browser, smartphone app or, like in this project, an ESP8266.

While the HTTP protocol allows servers to send data only upon a client request, Websockets allow bi-directional communication between server and client. It enables web applications to efficiently exchange data in real-time without the overhead of conventional HTTP connections. Node.js is perfectly suited for writing WebSocket applications, and this tutorial explains how it all works.

The Overall Architecure

The websocket server is running on a laptop or a Raspberry Pi which is connected to a local network via a Wifi router. Also, the ESP8266 is connected to the same network via the Wifi router. It continously receives sensor data through its GPIO and sends it as a data stream to the Websocket server. By connecting to the Websocket server's IP address via browser, we are able to see a live stream of incoming sensor data sent from the ESP8266.

Similar to a chat server, our Websocket server will be a central server that will receive all massages, process them and send them to the other clients. In this example we have one single Websocket server, which the ESP8266 and our smartphone web browser is connected to.

The communication is bi-directional. So, the client not only receives messages. But rather we can also send a message from the client to the ESP8266, telling it to stop or start sending data, for example.

Image: Architecture for Websocket communication between ESP8266, NodeJs Server and a web browser running on a PC or smartphone

 
Note:
In a more advanced project there can be two Websockets servers (channels)- one for the front end communication between multiple clients (web browser, smartphone app) and server. And another Websocket server for the back end communication between multiple ESP8266's and the server.
By doing so, we could achieve a clean separation between, the micro-controller side and the web browser (client) side.

This could be useful when having many clients and sensors connected to the same server. With two separate channels, the Node.JS server application decides which commands are forwarded from the client channel to the ESP8266 channel and vice versa.

Materials Needed

For the sake of simplicity, this project is demonstrated in its most basic form in such that features and materials are kept at a minimum.  Following materials are needed for this project:

1. Raspberry Pi with Node.Js installed and Wifi capability
2. NodeMCU ESP8266
3. Wifi router
4. Smartphone or PC with Web Browser and Wifi capability

Server-Side Code (Node.Js)

On the server-side (Raspberry Pi) install the latest version of Node of Node.Js (as of now: 10.8.0) with following command in the console.

$ curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash - 

sudo apt install -y nodejs

Continue with installing required components such as the Websocket package by following NPM commands.

npm install express
npm install body-parser
npm install --save ws

After that, copy and paste the code below into a text editor and save it as "websocket_example.js".

/**************************websocket_example.js*************************************************/

var bodyParser = require("body-parser");
const express = require('express'); //express framework to have a higher level of methods
const app = express(); //assign app variable the express class/method
var http = require('http');
var path = require("path");
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
const server = http.createServer(app);//create a server


//***************this snippet gets the local ip of the node.js server. copy this ip to the client side code and add ':3000' *****
//****************exmpl. 192.168.56.1---> var sock =new WebSocket("ws://192.168.56.1:3000");*************************************
require('dns').lookup(require('os').hostname(), function (err, add, fam) {
  console.log('addr: '+add);
})

/**********************websocket setup**************************************************************************************/
//var expressWs = require('express-ws')(app,server);
const WebSocket = require('ws');
const s = new WebSocket.Server({ server });

//when browser sends get request, send html file to browser
// viewed at http://localhost:30000
app.get('/', function(req, res) {
res.sendFile(path.join(__dirname + '/index.html'));
});


//*************************************************************************************************************************
//***************************ws chat server********************************************************************************

//app.ws('/echo', function(ws, req) {
s.on('connection',function(ws,req){

/******* when server receives messsage from client trigger function with argument message *****/
ws.on('message',function(message){
console.log("Received: "+message);
s.clients.forEach(function(client){ //broadcast incoming message to all clients (s.clients)
if(client!=ws && client.readyState ){ //except to the same client (ws) that sent this message
client.send("broadcast: " +message);
}
});
// ws.send("From Server only to sender: "+ message); //send to client where message is from
});
ws.on('close', function(){
console.log("lost one client");
});
//ws.send("new client connected");
console.log("new client connected");
});
server.listen(3000);

 
&anbsp

ESP8266 Code

Include the Websocket library for the ESP8266 from here in your Arduino IDE. To do so, follow the link, click on "clone or download" -> download .ZIP-file. Then include the library in your Arduino IDE under: Sketch->Include Library->Add .ZIP library.

Now upload the code below to your ESP8266 device. Edit the "ssid" and "password" variable values according to your own router settings. Furthermore edit the "host" variable according to the ip address of your raspberry Pi in your wifi network.

Once the code is running, the Esp8266 will connect to your wifi router and try to establish a websocket connection to your Node.js server running on your Raspberry Pi. If connection can be established, the ESP8266 will send random ADC readings to the Websocket server in 300ms intervals. You can change the interval by changing the value for this variable in the code.

/*******************Esp8266_Websocket.ino****************************************/

#include <ESP8266WiFi.h>
#include <WebSocketClient.h>

boolean handshakeFailed=0;
String data= "";

char path[] = "/";   //identifier of this device

const char* ssid     = "enter your wifi ssid here";
const char* password = "enter your wifi password here";
char* host = "192.168.0.23";  //replace this ip address with the ip address of your Node.Js server
const int espport= 3000;
  
WebSocketClient webSocketClient;
unsigned long previousMillis = 0;
unsigned long currentMillis;
unsigned long interval=300; //interval for sending data to the websocket server in ms

// Use WiFiClient class to create TCP connections
WiFiClient client;


void setup() {
  Serial.begin(115200);
    pinMode(readPin, INPUT);     // Initialize the LED_BUILTIN pin as an output

  delay(10);

  // We start by connecting to a WiFi network

  Serial.println();
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  
  WiFi.begin(ssid, password);
  
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");  
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

  delay(1000);
  
wsconnect();
//  wifi_set_sleep_type(LIGHT_SLEEP_T);
}

void loop() {

  if (client.connected()) {
currentMillis=millis(); 
    webSocketClient.getData(data);    
    if (data.length() > 0) {
Serial.println(data);




    //*************send log data to server in certain interval************************************
 //currentMillis=millis();   
 if (abs(currentMillis - previousMillis) >= interval) {
previousMillis = currentMillis;
data= (String) analogRead(A0); //read adc values, this will give random value, since no sensor is connected. 
//For this project we are pretending that these random values are sensor values

webSocketClient.sendData(data);//send sensor data to websocket server
}

  }
  else{
}
delay(5);

  }
}
//*********************************************************************************************************************
//***************function definitions**********************************************************************************
void wsconnect(){
  // Connect to the websocket server
  if (client.connect(host, espport)) {
    Serial.println("Connected");
  } else {
    Serial.println("Connection failed.");
      delay(1000);  
   
   if(handshakeFailed){
    handshakeFailed=0;
    ESP.restart();
    }
    handshakeFailed=1;
  }

  // Handshake with the server
  webSocketClient.path = path;
  webSocketClient.host = host;
  if (webSocketClient.handshake(client)) {
    Serial.println("Handshake successful");
  } else {
    
    Serial.println("Handshake failed.");
   delay(4000);  
   
   if(handshakeFailed){
    handshakeFailed=0;
    ESP.restart();
    }
    handshakeFailed=1;
  }
}

Client-Side Code for Web Browser (HTML,CSS,Javascript)

Open your text editor and copy&paste the code below into your editor. Save the file as "index.html" and move it to the same folder that contains your Node.Js application on your Raspberry Pi. As soon as you run your node.js server code, it will output it's network ip in the console. Copy this ip and replace the websocket server address below.

Example: If IP address is 192.168.56.1, change the relevant line below to: var sock =new WebSocket("ws://192.168.56.1:3000");

index.html

<html lang="en">
<head>
  <meta charset="utf-8"/>

  <title>Websocket</title>

</head>

<body>
<input style="width: 800px;height: 200px" type="text" placeholder="enter text here to send to server" id="eingabe" onkeypress="myFunction(event)"/>

<button onclick="clearlog()">Clear Logs</button>

<div style="overflow:scroll; width: 800px; height: 550px; border: solid; border-radius: 9px" id="display"></div>

  <script>
  //var sock =new WebSocket("ws://localhost:5001");
  var sock =new WebSocket("ws://192.168.1.23:3000");  //replace this address with the one the node.js server prints out. keep port 3000
  var display=document.getElementById("display")
  
  sock.onopen=function(event){
  //alert("Socket connected succesfully!!!")
  setTimeout(function(){sock.send('connection succeeded');},1000);
  display.innerHTML+="connection succeeded <br />";

  };
  sock.onmessage=function(event){
  console.log(event);//show received from server data in console of browser
  display.innerHTML+=event.data+"<br />"; //add incoming message from server to the log screen previous string + new string(message)

 }
  
function myFunction(event) {
    //sock.send("Hello");
if(event.keyCode==13){	   //when enter is pressed...
  var text=document.getElementById('eingabe').value;    //assign value of the entered string to te text variable
  if(text!=""){     //if text is not an empty string
  //display.innerHTML+="From Client: "+text+"<br />"; //show the message from client in div
sock.send(text);    //send string to server
display.innerHTML+="<strong>Me: </strong>" + text+"<br />";       //display input text in div (client side)
document.getElementById('eingabe').value="";     //and clear the input field
}
else{
console.log("empty string not sent")
}
}}

function clearlog(){
display.innerHTML="";
}
  </script>
</body>
</html>

Test the Setup

Now that the code for the ESP8266 is uploaded and the server-side code is ready, we can run the Node.Js server by navigating the console to the folder that contains the node.js application "websocket_example.js". We start it by entering following command in the console.

 node websocket_example.js

 
You will see several status logs in the console. As soon as the ESP8266 is connected to a power source, it will try to establish a websocket connection with the node.js server and send its sensor data feed.

Now we can use our smartphone or PC browser and enter the ip address of the raspberry pi followed by the port number 3000 (e.g. 196.168.0.1:3000). The browser will load the .html file that we edited earlier. If everything is set up correctly, you will be able to see the sensor data feed from the ESP8266.