In order to see the design to its completion, a software library and application were created. The software for this project was written in Python3 and HTML. Visual Studio Code was used for development and Git Bash was used to access the Raspberry Pi Zero for testing. An overview of the software created for the Care Sleeve is shown in Figure 7 below.
Back-end Front-end
Figure 7: Software Overview Flowchart
From left to right, the flowchart goes from back-end to front-end. First, we will discuss the basic design of our back-end, library software.
Infrastructure Layer
The library software consists of many external libraries (given in setup.py) and one large library file (nighttimeParenting.py). The external libraries are discussed briefly below and a more in-depth discussion about nighttimeParenting.py will follow shortly after.
setup.py
The team took advantage of many pre-existing libraries. The libraries used that had to be installed are assigned to the ‘install_requires’ variable in setup.py below.
Figure 8: setup.py
The SPI communication protocol is used for the LED Bar. I2C communication protocol for the OLED screen and the heart rate sensor. The ‘hrcalc’ and ‘max30102’ in setup.py are used for the heart rate sensor.
nighttimeParenting.py
The nighttimeParenting.py library can be broken into six main parts. Each of these parts have their own class in nighttimeParenting.py and are shown as blocks below.
Figure 9: Overview of nighttimeParenting.py Library
Below, the basic design and the decisions that went into each of the six classes in nighttimeParenting.py is described.
micCircuit:
In the micCircuit class there is a function called getPkPkVal(). This finds the peak-to-peak value of the audio output. Another function called trigger() compares this to a threshold value to determine if the baby is awake. A digital value in the low 30’s was found as a good cutoff for the threshold value. The peak-to-peak value is determined by finding the difference between the minimum and maximum in a given time interval. In addition, to peak-to-peak, the micCircuit has some minor functions. For example, there is a function to fetch the digital value from the ADC and there is also a function that converts the digital ADC value into an analog value between 0-3.3V.
It was ultimately decided to use peak-to-peak to monitor wakeups due to its straightforward implementation. However, this approach does have some drawbacks. A loud sound that is not a baby crying can trigger a false alarm. Therefore, in the future, it is planned to detect wakeups based on the frequency of a baby crying rather than just the peak-to-peak value.
StereoDecoder:
For the StereoDecoder class, a free, open-source set of Python modules called Pygame was used. This library was designed for developing video games, and its mixer module is meant for loading and playing sounds. Most of the functions of the StereoDecoder class are just wrappers over functions given by this module. Some logic was also taken from Adafruit’s example of our stereo decoder using pygame.
OLED:
For the OLED class, the smbus library is used for I2C communication. For graphics and text, adafruit-circuitpython-ssd1306 and Python Imaging Library are used. The two functions of the OLED class adopted much of its code from our fall semester lab, and from the clock example of the luma library.
HRSensor:
In the HRSensor class, there is a function called readSensor() that calls upon max30102.py to sample 100 data points from the heart rate sensor and calls upon hrcalc.py to calculate the BPM and SPo2 levels of the user. This function is then called in two functions called getHR_SPO2() and getHR_SPO2_blocking(). The difference between these two functions is getHR_SPO2_blocking() will not return until valid data is received from the user.
The biggest challenge in implementing this class was figuring out how hrcalc.py and max30102.py work. It had to be confirmed that the lower level libraries were being used properly to prevent it from blocking other functions from executing in the application layer.
ledBar:
In the ledBar class there is one function called set_bar_level(). This function takes in a parameter called ‘level’ and illuminates the LED bar based on the value sent to it.
Initially, there was a breathe_in() and a breathe_out() function in the infrastructure layer. However, they were removed and implemented in the application layer so the LED bar would not hold onto the spi communication channel for too long. Therefore, the grunt work for this part is done in the application layer.
PhysicalUI:
In the PhysicalUI class there are a few functions each of which address one aspect of the physical user-interface. The toggleVolume() function reads in a value from the ADC and adjusts the volume to a value from 0 to 1 accordingly. The getBrightness() and setBrightness() functions read in a value from the ADC and set the brightness, respectively. Lastly, triggerSOS() determines if the SOS button is pressed and returns true if this occurs.
The biggest challenge in implementing the physical UI was the brightness. Initially, there had been one function to control the brightness. However, because the SPI communication is used for the ADC and the I2C communication protocol is used for the OLED, there were problems in the application layer when trying to acquire both the I2C and SPI locks simultaneously. Therefore, it was ultimately decided to split the brightness into two functions: getBrightness() which uses SPI and setBrightness which uses I2C.
Application Layer
The application layer consists of one Python3 file: app.py. The application consists of four sections: tasks that run in the background, tasks that run in response to a wakeup, tasks that run in response to stress level, and tasks that run in response to the browser-based user-interface. A state diagram of the application is shown below.
Figure 10: State Diagram of app.py
The state diagram provides a basic overview of the application’s functionality and how it responds to events. Below, we will describe the basic design and the decisions that went into each of the four sections of app.py.
Tasks that Run in the Background:
There are two main tasks that run in the background, one task to continually monitor the baby and one task to continually calculate the stress levels of the user. These tasks are represented by the threads monitorBaby() and calculateStressLevel(). monitorBaby() continually checks if the microphone circuit is above the threshold value. If the microphone circuit is above the threshold value, monitorBaby() will set the ‘wakeup’ event which will begin the tasks that run in response to a wakeup.
Similarly, calculateStressLevel() continually checks if the user’ s BPM and SPO2 is above 110 and below 95%, respectively. If both of these conditions are satisfied, calculateStressLevel() will set the ‘stressHigh’, ‘enableBreathing’, ‘enableMessages’, and ‘EnableMusic’ events. These events start the tasks that run in response to heightened stress levels.
Ultimately, it was decided to allow calculateStressLevel() to run in the background instead of running in response to a wakeup event. This is because when no wakeup has occurred, no other thread is using the I2C communication protocol. Also, there will be no valid data if the user is not wearing the Care Sleeve, so calculateStessLevel() running in the background will not cause any other tasks to run in response to it.
Tasks that Run in Response to Wakeup:
There is only one task that runs in response to a baby wakeup: wakeupEvent(). First, the wakeupEvent() thread gets the current time as a string and appends this to a log to keep track of wakeups. Next, the thread sends an email to the caregiver notifying them that their baby is awake.
In the end, it was decided to use email to notify the caregivers. This is because it was most straightforward to implement in the time allotted for this project and task. However, in the future SMS messages are planned to be used to notify the caregivers of a wakeup event occurring.
Tasks that Run in Response to Stress Level:
There are five tasks that run in response to elevated stress levels. These are all encapsulated in threads and are called notifyStressLevels(), messageDisplay(), timeDisplay(), updateBreathing(), and playMusic().
Similar to wakeupEvent(), notifyStressLevels() sends an email to the user notifying them that their stress levels are elevated. messageDisplay() and timeDisplay() display encouraging messages and the time on the OLED screen, respectively. Ultimately, we decided to only show the timeDisplay() if the user did not want to view the encouraging messages. The message display can be switched to the time display in the browser user-interface.
Lastly, updateBreathing() and playMusic() begin the breathing exercises on the LED Bar and play music on the stereo decoder, respectively. It was decided that the stress relief would automatically begin when their stress levels are elevated, but it would be up to the user to decide when to end the stress relief. The stress relief can be terminated by using the two dials to turn off the OLED screen, LED Bar, and the music.
Tasks that Run in Response to Browser-Based User-Interface:
There are many tasks that run in response to the browser-based user-interface. Most of these tasks run in response to a button being pressed on the webpage. For instance, addMessage() and deleteMessage() run in response to the ‘add’ and ‘delete’ buttons being pressed, respectively. Additionally, pauseMusic() and resumeMusic() run in response to the ‘pause’ and ‘play’ buttons being pressed.
In addition to buttons, pauseMessages() and resumeMessages() run in response to the ‘messages’ checkbox. If the checkbox is selected, then the messages will pause. Otherwise, the messages will continue. Lastly, adjustVolume() runs in response to the volume bar being moved.
Most of these functions call upon the library layer. However, these functions were not directly implemented in the browser user-interface because we decided that our code should only be able to communicate one layer up or down. Therefore, in this case, app.py serves as a gateway between the user-interface layer and the infrastructure layer.
User-Interface Layer
Finally, the user-interface layer consists of four files: flaskapp.py, index.html, setup.html, and analytics.html. We designed this layer to be clear and easy to use. This layer was inspired by the user experience research that was conducted throughout this semester.
Flaskapp.py
This file hosts the web server for our websites. This file consists of four functions, which execute based on which page the user clicks on. home_template() and home() execute when the user navigates to ‘/templates/index’ and ‘/’, respectively.
setup_template() executes when the user navigates to ‘/templates/setup’. This function communicates with the application layer based on what button is pressed. Lastly, analytics_template() executes when the user navigates to ‘/templates/analytics’. This function was not fully implemented due to time constraints but is supposed to communicate with the application layer to collect the data for analytics.
Index.html
This file was implemented with the help of bootstrap studio and is responsible for the formatting of the home page of our website.
Figure 11: Features ‘Home’ Page
Setup.html
This file was implemented with the help of bootstrap studio and is responsible for the formatting of the setup page of our website.
Figure 12: Setup Page
Analytics.html
This file was implemented with the help of bootstrap studio and is responsible for the formatting of the analytics page of our website.
Introduction
Back-end Front-end Figure 7: Software Overview Flowchart From left to right, the flowchart goes from back-end to front-end. First, we will discuss the basic design of our back-end, library software.
Infrastructure Layer
setup.py
The team took advantage of many pre-existing libraries. The libraries used that had to be installed are assigned to the ‘install_requires’ variable in setup.py below.
Figure 8: setup.py
The SPI communication protocol is used for the LED Bar. I2C communication protocol for the OLED screen and the heart rate sensor. The ‘hrcalc’ and ‘max30102’ in setup.py are used for the heart rate sensor.
nighttimeParenting.py
The nighttimeParenting.py library can be broken into six main parts. Each of these parts have their own class in nighttimeParenting.py and are shown as blocks below.
Below, the basic design and the decisions that went into each of the six classes in nighttimeParenting.py is described.
micCircuit:
In the micCircuit class there is a function called getPkPkVal(). This finds the peak-to-peak value of the audio output. Another function called trigger() compares this to a threshold value to determine if the baby is awake. A digital value in the low 30’s was found as a good cutoff for the threshold value. The peak-to-peak value is determined by finding the difference between the minimum and maximum in a given time interval. In addition, to peak-to-peak, the micCircuit has some minor functions. For example, there is a function to fetch the digital value from the ADC and there is also a function that converts the digital ADC value into an analog value between 0-3.3V. It was ultimately decided to use peak-to-peak to monitor wakeups due to its straightforward implementation. However, this approach does have some drawbacks. A loud sound that is not a baby crying can trigger a false alarm. Therefore, in the future, it is planned to detect wakeups based on the frequency of a baby crying rather than just the peak-to-peak value.
StereoDecoder:
For the StereoDecoder class, a free, open-source set of Python modules called Pygame was used. This library was designed for developing video games, and its mixer module is meant for loading and playing sounds. Most of the functions of the StereoDecoder class are just wrappers over functions given by this module. Some logic was also taken from Adafruit’s example of our stereo decoder using pygame.
OLED: For the OLED class, the smbus library is used for I2C communication. For graphics and text, adafruit-circuitpython-ssd1306 and Python Imaging Library are used. The two functions of the OLED class adopted much of its code from our fall semester lab, and from the clock example of the luma library.
HRSensor:
In the HRSensor class, there is a function called readSensor() that calls upon max30102.py to sample 100 data points from the heart rate sensor and calls upon hrcalc.py to calculate the BPM and SPo2 levels of the user. This function is then called in two functions called getHR_SPO2() and getHR_SPO2_blocking(). The difference between these two functions is getHR_SPO2_blocking() will not return until valid data is received from the user. The biggest challenge in implementing this class was figuring out how hrcalc.py and max30102.py work. It had to be confirmed that the lower level libraries were being used properly to prevent it from blocking other functions from executing in the application layer.
ledBar:
In the ledBar class there is one function called set_bar_level(). This function takes in a parameter called ‘level’ and illuminates the LED bar based on the value sent to it. Initially, there was a breathe_in() and a breathe_out() function in the infrastructure layer. However, they were removed and implemented in the application layer so the LED bar would not hold onto the spi communication channel for too long. Therefore, the grunt work for this part is done in the application layer.
PhysicalUI:
In the PhysicalUI class there are a few functions each of which address one aspect of the physical user-interface. The toggleVolume() function reads in a value from the ADC and adjusts the volume to a value from 0 to 1 accordingly. The getBrightness() and setBrightness() functions read in a value from the ADC and set the brightness, respectively. Lastly, triggerSOS() determines if the SOS button is pressed and returns true if this occurs. The biggest challenge in implementing the physical UI was the brightness. Initially, there had been one function to control the brightness. However, because the SPI communication is used for the ADC and the I2C communication protocol is used for the OLED, there were problems in the application layer when trying to acquire both the I2C and SPI locks simultaneously. Therefore, it was ultimately decided to split the brightness into two functions: getBrightness() which uses SPI and setBrightness which uses I2C.
Application Layer
Figure 10: State Diagram of app.py
The state diagram provides a basic overview of the application’s functionality and how it responds to events. Below, we will describe the basic design and the decisions that went into each of the four sections of app.py.
Tasks that Run in the Background: There are two main tasks that run in the background, one task to continually monitor the baby and one task to continually calculate the stress levels of the user. These tasks are represented by the threads monitorBaby() and calculateStressLevel(). monitorBaby() continually checks if the microphone circuit is above the threshold value. If the microphone circuit is above the threshold value, monitorBaby() will set the ‘wakeup’ event which will begin the tasks that run in response to a wakeup. Similarly, calculateStressLevel() continually checks if the user’ s BPM and SPO2 is above 110 and below 95%, respectively. If both of these conditions are satisfied, calculateStressLevel() will set the ‘stressHigh’, ‘enableBreathing’, ‘enableMessages’, and ‘EnableMusic’ events. These events start the tasks that run in response to heightened stress levels. Ultimately, it was decided to allow calculateStressLevel() to run in the background instead of running in response to a wakeup event. This is because when no wakeup has occurred, no other thread is using the I2C communication protocol. Also, there will be no valid data if the user is not wearing the Care Sleeve, so calculateStessLevel() running in the background will not cause any other tasks to run in response to it.
Tasks that Run in Response to Wakeup: There is only one task that runs in response to a baby wakeup: wakeupEvent(). First, the wakeupEvent() thread gets the current time as a string and appends this to a log to keep track of wakeups. Next, the thread sends an email to the caregiver notifying them that their baby is awake. In the end, it was decided to use email to notify the caregivers. This is because it was most straightforward to implement in the time allotted for this project and task. However, in the future SMS messages are planned to be used to notify the caregivers of a wakeup event occurring.
Tasks that Run in Response to Stress Level: There are five tasks that run in response to elevated stress levels. These are all encapsulated in threads and are called notifyStressLevels(), messageDisplay(), timeDisplay(), updateBreathing(), and playMusic(). Similar to wakeupEvent(), notifyStressLevels() sends an email to the user notifying them that their stress levels are elevated. messageDisplay() and timeDisplay() display encouraging messages and the time on the OLED screen, respectively. Ultimately, we decided to only show the timeDisplay() if the user did not want to view the encouraging messages. The message display can be switched to the time display in the browser user-interface. Lastly, updateBreathing() and playMusic() begin the breathing exercises on the LED Bar and play music on the stereo decoder, respectively. It was decided that the stress relief would automatically begin when their stress levels are elevated, but it would be up to the user to decide when to end the stress relief. The stress relief can be terminated by using the two dials to turn off the OLED screen, LED Bar, and the music.
Tasks that Run in Response to Browser-Based User-Interface: There are many tasks that run in response to the browser-based user-interface. Most of these tasks run in response to a button being pressed on the webpage. For instance, addMessage() and deleteMessage() run in response to the ‘add’ and ‘delete’ buttons being pressed, respectively. Additionally, pauseMusic() and resumeMusic() run in response to the ‘pause’ and ‘play’ buttons being pressed. In addition to buttons, pauseMessages() and resumeMessages() run in response to the ‘messages’ checkbox. If the checkbox is selected, then the messages will pause. Otherwise, the messages will continue. Lastly, adjustVolume() runs in response to the volume bar being moved. Most of these functions call upon the library layer. However, these functions were not directly implemented in the browser user-interface because we decided that our code should only be able to communicate one layer up or down. Therefore, in this case, app.py serves as a gateway between the user-interface layer and the infrastructure layer.
User-Interface Layer
Finally, the user-interface layer consists of four files: flaskapp.py, index.html, setup.html, and analytics.html. We designed this layer to be clear and easy to use. This layer was inspired by the user experience research that was conducted throughout this semester. Flaskapp.py This file hosts the web server for our websites. This file consists of four functions, which execute based on which page the user clicks on. home_template() and home() execute when the user navigates to ‘/templates/index’ and ‘/’, respectively. setup_template() executes when the user navigates to ‘/templates/setup’. This function communicates with the application layer based on what button is pressed. Lastly, analytics_template() executes when the user navigates to ‘/templates/analytics’. This function was not fully implemented due to time constraints but is supposed to communicate with the application layer to collect the data for analytics. Index.html This file was implemented with the help of bootstrap studio and is responsible for the formatting of the home page of our website.
Figure 11: Features ‘Home’ Page Setup.html This file was implemented with the help of bootstrap studio and is responsible for the formatting of the setup page of our website.
Figure 12: Setup Page Analytics.html This file was implemented with the help of bootstrap studio and is responsible for the formatting of the analytics page of our website.
Figure 13: Analytics Page