Tuesday, May 26, 2015

Lab 7.1 - Realtime Visualisation of Data - GSR and lie detectors

In the last lab we used a Python program that receives data from an XBee to visualize heart beats in close-to-real time. It was close to real time due to several factors:

  • big sampling rate of the pulse sensor
  • processing, queuing, transmission and propagation delay due to using 2 XBees
  • inefficient Python code that drew the graph containing all the values ever received
This time, we decided to focus on solving some of these issues in order to achieve a more accurate visualizing application.


The idea
By using 2 XBees, an Arduino and a Galvanic Skin Response Sensor, we are able to monitor the conductance of an individual's skin and send it wirelessly to a computer that plots the data in real time with the use of Python. Indirectly, this shows the stress the individual currently faces and could serve as a helping tool in finding if someone is lying.
By adding a buzzer we can get audio feedback when a threshold is reached.

How to do it? 
The GSR sensor measures electrodermal activity and it works by sensing the conductance of one's skin and sending analog values to its controller when this happens. The GSR sensor is connected to an Arduino, and the Arduino sends an encapsulated message containing the sensor reading to a connected XBee. The Arduino is also connected to a buzzer that beeps whenever a threshold crossed.

We decreased the sampling rate to 100 samples/second (in Arduino, delay(100) after sampling) as there is no need for a very high sampling rate. This solves one of the latency issues.

On the other wireless side, an XBee is connected to the computer and a Python application listens for new serial data. As soon as it receives a new message, it decapsulates it and adds the values to an array. It also calculates the time it received it at and adds that to another array. It then removes the first elements of the arrays so that the arrays always have a fixed number of elements. This helps decrease the latency of the application. It then plots them, displaying the graph in real time.

At the same time, because the data is not always received correctly and in the right order, possible errors must be checked for in the Python application.

Image of real time data





Video


https://www.youtube.com/watch?v=6YvWhxkTbZU&feature=youtu.be


Schematics

                                    

Materials used

  • 1 Arduino Uno
  • 2 XBees Explorer
  • 2 XBee
  • 1 Breadboards
  • 1 Galvanic Skin Response sensor (we used http://www.seeedstudio.com/depot/Grove-GSR-p-1614.html )

Software needed

  • XCTU - here
  • Python - here.
  • XBee library for Python here.
  • Pyserial library for Python - here.
  • XBee library for Arduino - here.
  • Matplotlib library for Python - here

Steps

  1. Set one XBee to Coordinator AT mode
    1. In XCTU - Set PAN ID to BBBB
    2. Connect this Xbee to Computer
  2. Set one XBee to Router AT mode
    1. In XTCU - Set PAN ID to BBBB
    2. Connect this Xbee to Arduino (Rx to Tx and Tx to Rx)
  3. Connect GSR to Arduino as shown in schematics
  4. Connect Buzzer to Arduino
  5. Upload Arduino code on Arduino
  6. Run Python code on computer

Code:


Arduino:
const int BUZZER = 3;
const int GSR = A2;
int threshold = 0;
int sensorValue;
long counter = 0;
void setup() {
  long sum = 0;
  Serial.begin(9600);
  pinMode(BUZZER, OUTPUT);
  digitalWrite(BUZZER, LOW);
  delay(1000);

  for (int i = 0; i < 500; i++)
  {
    sensorValue = analogRead(GSR);
    sum += sensorValue;
    delay(5);
  }
  threshold = sum / 500;
  Serial.print("threshold =");
  Serial.println(threshold);
}

void loop() {
  int temp;
  sensorValue = analogRead(GSR);
  //Serial.print("sensorValue=");
  Serial.println(sensorValue);
  temp = threshold - sensorValue;
  if (abs(temp) > 50)
  {
    sensorValue = analogRead(GSR);
    temp = threshold - sensorValue;
    if (abs(temp) > 50) {
      tone(BUZZER, 300);
      //Serial.println("YES!");
    }
  }
  else noTone(BUZZER);
  delay(50);
  //Serial.print(counter);

}

Python

import time
import serial
import numpy as np
from matplotlib import pyplot as plt


ser = serial.Serial('COM20', 9600) #on linux it's a different port, do LSUSB to find the various usbports used
#example on linux: ser = serial.Serial('dev/tty.usbserial', 9600)
y = [0] * 200 # values you can mess with depending on your sampling rate
x = [0] * 200 # it only shows the maximum number of values you can see on your plot
# make the plot
line, = plt.plot(y)
plt.xlabel("Time in miliseconds")
plt.ylabel("Value of sensor data")
plt.ylim([0,1000]) #limit the y axis, change this if your values are higher than 1000

# start data collection
while True:  
    data = ser.readline() # read data from serial port
    print "Time:" , time.clock()
    data=data.split()[0]
    print "Data received is", data
    ####Check if received Y is a correct value
    if (type(data) is not str and (data < 1023 and data > 0)):
        y.append(data)
        x.append(time.clock() * 1000)
        del y[0] #delete first element
        del x[0] #delete first element
        line.set_xdata(x)
        line.set_ydata(y)
        plt.xlim([int(x[len(x) - 1]) - 10000, int(x[len(x) - 1])])  # make x and y axis stable
        plt.draw()
        plt.pause(0.00001)

    else:
        if data.isdigit():
            y.append(int(data))
            x.append(time.clock() * 1000)
            del y[0]
            del x[0]
            line.set_xdata(x)
            line.set_ydata(y)
            plt.xlim([int(x[len(x) - 1]) - 10000, int(x[len(x) - 1])])  # make x and y axis stable
            plt.draw()
            plt.pause(0.00001)
            

Tuesday, May 19, 2015

Lab 6.2 - Wirelessly plotting the heart rate in real time

https://www.youtube.com/watch?v=wu3GdAQlGPE&feature=youtu.be

The idea

By using 2 XBees, an Arduino and an Infrared Pulse Sensor we are able to monitor the heart rate of an individual and send it wirelessly to a computer that plots the data in real time with the use of Python. By extending this to a sensor network, one could monitor the patients in a hospital wirelessly.



How to do it? 
The Infrared Pulse Sensor(Heart rate ear clip) works by sensing variations in the bloodflow of an individual and sending digital HIGH values when this happens. The heart-rate sensor is connected to an Arduino in order to add a timestamp, and the Arduino sends an encapsulated message containing the timestamp and the sensor reading to a connected XBee.

On the other wireless side, an XBee is connected to the computer and a Python application listens for new serial data. As soon as it receives a new message, it decapsulates it and adds the 2 values (timestamp and sensor reading) to 2 different arrays. It then plots them, displaying it in real time.


We had run into many problems during the development of the project, mainly because of the latency that the XBees produce, making the plotting of the data "close to" realtime. Nevertheless, because the Arduino adds a timestamp, the data is displayed correctly, just not in perfect real time.
At the same time, because the data is not always received correctly and in the right order, possible errors must be checked for in the Python application.

Image of real time data



Video



https://www.youtube.com/watch?v=wu3GdAQlGPE&feature=youtu.be


Schematics



Materials used

  • 1 Arduino Uno
  • 2 XBees Explorer
  • 2 XBee
  • 1 Breadboards
  • 1 Heart rate ear clip (Model: MED03212P)

Software needed

  • XCTU - here
  • Python - here.
  • XBee library for Python here.
  • Pyserial library for Python - here.
  • XBee library for Arduino - here.
  • Matplotlib library for Python - here

Steps

  1. Set one XBee to Coordinator API mode
    1. In XCTU - Set PAN ID to BBBB
    2. In XCTU - Set API Mode to 2
    3. Connect this Xbee to Computer
  2. Set one XBee to Router AT mode
    1. In XTCU - Set PAN ID to BBBB
    2. Connect this Xbee to Arduino (Rx to Tx and Tx to Rx)
  3. Connect Heart rate sensor to Arduino as shown in schematics
  4. Upload Arduino code on Arduino
  5. Run Python code on computer

Code:


Arduino:
//define pins
int HRpin = 2; //digital PIN 2
int reading;
long initialTime=0,time;

void setup() {
  Serial.begin(9600);
  // put your setup code here, to run once:
  pinMode(HRpin,INPUT);
}

void loop() {
  // put your main code here, to run repeatedly:
  if(initialTime == 0)
    initialTime = millis();
  reading = digitalRead(HRpin);
  time = millis() - initialTime;
  
  String sentData = String("Time:" + String(time) + "\n" + "Value:" + String(reading) + "\n");
  Serial.print(sentData);
  delay(150);

}

Python:
'''
Created on May 19, 2015
@author: goku
'''
import serial
import time
from xbee import ZigBee
import numpy as np
import matplotlib.pyplot as plt

#initialize empty lists x and y to be populated with timestamp and value of heart pulse(true/false)
x = [0]
y = [0]
new_x = new_y = 0
ser = serial.Serial('COM20', 9600)
while True:
    try:
        plt.clf() #clear the plot
        plt.xlabel("Time in miliseconds")
        plt.ylabel("Heart pulse")
        plt.title("Wireless Heart Monitor")
        plt.ylim([-0.2,1.2])
    
        #read data from XBee
        data = ser.readline()
        
        #decapsulate the data
        formated_data = data.split()
        if "Time:" in formated_data[0]:
            #the value of Time that we receive from arduino
            new_x = formated_data[0].split("Time:")[1]
            print new_x
        elif "Value:" in formated_data[0]:
            #the value of the pulse reading that we get from arduino
            new_y = formated_data[0].split("Value:")[1]
            print new_y
        
        #Check if received Y is a correct value
        if (type(new_y) is not str):
            print "y not str: true"
            y.append(new_y)
        else:
            if new_y.isdigit():
                y.append(int(new_y))
                
        #Check if received X is a correct value
        if (type(new_x) is not str):
            print "x not str: true"
            x.append(new_x )
        else:
            if new_x.isdigit():
                x.append(int(new_x)) #plot graph in seconds
        #plot the graph
        plt.plot(x,y, "g")
        plt.xlim([int(x[len(x) - 1])-10000,int(x[len(x) - 1])]) #make x and y axis stable
        #plt.text(x, y, s, fontdict, withdash)
        plt.pause(0.00001)
    except KeyboardInterrupt:
        break
ser.close()

Tuesday, May 12, 2015

Lab 6.1: Pinging a Buzzer


The idea: by using Pythonwe read a number 'n' from the keyboard and create a random string that is made of 'n' characters, which we encapsulate in an XBee frame and send it serially to the USB connected XBee. That one in turn sends it to another XBee that is connected to an Arduino. The Arduino decapsulates the frame and plays a song based on the 'n' characters it receives.



How to do it? The project is more about software than hardware, since we are playing with how the XBee protocol and its frames. Thus, in Python we create and XBee Transmit request frame which we decapsulate in Arduino in order to make sense of the data.

Video



Schematics



Materials used

  • 1 Arduino Uno
  • 2 XBees Explorer
  • 2 XBee
  • 2 Breadboards
  • 1 Buzzer
  • 1 LED

Software needed

  • XCTU - here
  • Python - here.
  • XBee library for Python - here.
  • Pyserial library for Python - here.
  • XBee library for Arduino - here.

Steps

  1. Set one XBee to Coordinator API mode
    1. In XCTU - Set PAN ID to BBBB
    2. In XCTU - Set API Mode to 2
    3. Connect this Xbee to Arduino (Rx to Tx and Tx to Rx)
  2. Set one XBee to Router API mode
    1. In XTCU - Set PAN ID to BBBB
    2. In XCTU - Set API Mode to 2
    3. Connect this Xbee to Computer
  3. Upload Arduino code on Arduino
  4. Run Python code on computer

Code:

1.Python code

from platform import system
from xbee.zigbee import ZigBee
import random
import string
import serial
import time
from datetime import datetime

ser = serial.Serial('COM19', 9600)
xbee = ZigBee(ser, escaped=True) #needed to use Zigbee instead of XBee in order to responses to be interpreted correctly
DEST_ADDR_LONG = "\x00\x13\xA2\x00\x40\x8B\x96\x2E" #destination Zigbee 64 bit address
#discover the short 16-bit address -> faster messages
xbee.send("tx",data="\x00\x03",dest_addr_long=DEST_ADDR_LONG,dest_addr="\xff\xfe")
response = xbee.wait_read_frame()
short_addr = response["dest_addr"]

input = raw_input('Enter the number of forevernotes you want your theme to have:')
#mario song encoded for our arduino to play it
#the letter s represents a break
mario = "4s4ss4sss~s4ss7sssssIsss ~sssIsssTsssspss]sss[sPsIsss4s7s9sss5s7ss4ss~s2s]ss ~sssIsssTsssspss]sss[sPsIsss4s7s9sss5s7ss4ss~s2s]ss"


if input.isdigit():
    n = string.atoi(input)
else: n = 0

# Continuously read and print packets
i = 0
while True:
    try:
        #if your input is "mario", play it TADAAA
        if input == "mario":
            for c in mario:
                tstart = datetime.now()
                xbee.send('tx',
                          frame_id='A',
                          dest_addr_long=DEST_ADDR_LONG,
                          dest_addr=short_addr,
                          data=c)
                print c
                time.sleep(0.1)
            response = xbee.wait_read_frame()
            tend = datetime.now()
            print tend-tstart
        #otherwise continuously create a random song made of n notes and play it 5 times
        else:
            i = i + 1
            if i % 5 == 0:
                input = ''.join(random.choice("qwertyuiop[`1234567890-=zb") for _ in range(n))
            print "sending data"
                        
            for c in input:
                xbee.send('tx',
                          frame_id='A',
                          dest_addr_long=DEST_ADDR_LONG,
                          dest_addr=short_addr,
                          data=c)
                print c
                time.sleep(0.1)
        #time.sleep(2) #NO REST FOR THE WICKED
    except KeyboardInterrupt:
        break

ser.close()
2.Arduino code
/*
WSN LAB 6.1 - Zigbee API & Arduino
*/

#include <XBee.h>
// Pins
const int ledPin = 13;
const int speakerPin = 12;

// Lower octave tones
int tones[] = {261, 277, 294, 311, 330, 349, 370, 392, 415, 440, 466, 494};
//            mid C  C#   D    D#   E    F    F#   G    G#   A    A#   B
// Higher octave tones
int tones2[] = {523, 554, 587, 622, 659, 698, 740, 784, 831, 880, 932, 988, 1047};
//               C    C#   D    D#   E    F    F#   G    G#   A    A#   B    C
char incomingChar;      // a variable to read incoming serial data into
/** XBee objects */
XBee xbee;
XBeeResponse response;
// create reusable response objects for responses we expect to handle
ZBRxResponse rx;

void setup()
{
  Serial.begin(9600);

  pinMode(ledPin, OUTPUT);
  pinMode(speakerPin, OUTPUT);

  xbee = XBee();
  response = XBeeResponse();
  rx = ZBRxResponse();
}

void loop()
{
  // This will read any data that is available:
  xbee.readPacket();
  if (xbee.getResponse().isAvailable())
  {
    if (xbee.getResponse().getApiId() == ZB_RX_RESPONSE)
    {
      xbee.getResponse().getZBRxResponse(rx);
      Serial.println(rx.getData(0));
      digitalWrite(ledPin, HIGH); // turn the led on indicating that you received an XBee frame
      incomingChar = rx.getData(0);//convert the received data to Char
      Serial.print("Playing character:");
      Serial.println(incomingChar);

      // play a different note depending on the
      // value received from XBee:
      switch (incomingChar) {
        case 'Q':
        case 'q':
          tone(speakerPin, tones[0]);
          delay(100);
          break;
        case 'W':
        case 'w':
          tone(speakerPin, tones[1]);
          delay(100);
          break;
        case 'e':
        case 'E':
          tone(speakerPin, tones[2]);
          delay(100);
          break;
        case 'r':
        case 'R':
          tone(speakerPin, tones[3]);
          delay(100);
          break;
        case 't':
        case 'T':
          tone(speakerPin, tones[4]);
          delay(100);
          break;
        case 'y':
        case 'Y':
          tone(speakerPin, tones[5]);
          delay(100);
          break;
        case 'u':
        case 'U':
          tone(speakerPin, tones[6]);
          delay(100);
          break;
        case 'i':
        case 'I':
          tone(speakerPin, tones[7]);
          delay(100);
          break;
        case 'o':
        case 'O':
          tone(speakerPin, tones[8]);
          delay(100);
          break;
        case 'p':
        case 'P':
          tone(speakerPin, tones[9]);
          delay(100);
          break;
        case '[':
        case '{':
          tone(speakerPin, tones[10]);
          delay(100);
          break;
        case ']':
        case '}':
          tone(speakerPin, tones[11]);
          delay(100);
          break;

        case 'a':
        case 'A':
          tone(speakerPin, tones2[0]);
          delay(100);
          break;
        case 'd':
        case 'D':
          tone(speakerPin, tones2[2]);
          delay(100);
          break;
        case 'f':
        case 'F':
          tone(speakerPin, tones2[3]);
          delay(100);
          break;
        case 'g':
        case 'G':
          tone(speakerPin, tones2[4]);
          delay(100);
          break;
        case 'h':
        case 'H':
          tone(speakerPin, tones2[5]);
          delay(100);
          break;
        case 'j':
        case 'J':
          tone(speakerPin, tones2[6]);
          delay(100);
          break;
        case 'k':
        case 'K':
          tone(speakerPin, tones2[7]);
          delay(100);
          break;
        case 'l':
        case 'L':
          tone(speakerPin, tones2[8]);
          delay(100);
          break;

        case '`':
        case '~':
          tone(speakerPin, tones2[0]);
          delay(100);
          break;
        case '1':
        case '!':
          tone(speakerPin, tones2[1]);
          delay(100);
          break;
        case '2':
        case '@':
          tone(speakerPin, tones2[2]);
          delay(100);
          break;
        case '3':
        case '#':
          tone(speakerPin, tones2[3]);
          delay(100);
          break;
        case '4':
        case '$':
          tone(speakerPin, tones2[4]);
          delay(100);
          break;
        case '5':
        case '%':
          tone(speakerPin, tones2[5]);
          delay(100);
          break;
        case '6':
        case '^':
          tone(speakerPin, tones2[6]);
          delay(100);
          break;
        case '7':
        case '&':
          tone(speakerPin, tones2[7]);
          delay(100);
          break;
        case '8':
        case '*':
          tone(speakerPin, tones2[8]);
          delay(100);
          break;
        case '9':
        case '(':
          tone(speakerPin, tones2[9]);
          delay(100);
          break;
        case '0':
        case ')':
          tone(speakerPin, tones2[10]);
          delay(100);
          break;
        case '-':
        case '_':
          tone(speakerPin, tones2[11]);
          delay(100);
          break;
        case '=':
        case '+':
          tone(speakerPin, tones2[12]);
          delay(100);
          break;
      }
    }
  }
  //stop the buzzer and turn the LED off as soon as it's done processing the serial buffer
  noTone(speakerPin);
  digitalWrite(ledPin, LOW);

}