Beyond Accessibility: Unleashing the Potential of Home Automation to Foster Independence and Inclusivity for Individuals with Unique Needs
2024 - EPFL (CS-358) MIT - Group project
Helping Hand is an innovative project designed to enhance the independence of individuals with mobility impairments through adaptable home automation. The core of the project is a modular box equipped with a Wi-Fi enabled microcontroller and a plotter like mechanism that allows remote control of various household appliances. This system can be operated via a smartphone or WEB application, making it accessible for users with limited physical capabilities.
Inspired by the needs of Sean, a person with ALS who has limited mobility, Helping Hand aims to facilitate tasks such as opening doors and operating blinds without assistance. The modular box can accommodate different remote controls, allowing users to control multiple devices through a single, user-friendly interface. The project prioritizes accessibility, ease of use, and adaptability, making it a versatile solution for various remote-controlled appliances.
Note: you can check out the original project's proposal: Helping Hand project proposal.
Below is a brief explanation of this repository's structure to provide context and aid readers' comprehension in the following sections of this document.
code
directory: contains all the code for the different parts of the project.
CVserver
: the python server side program responsible for image processing and remote buttons automatic detection and configuration.esp32-cam
: the ESP-32 Cam Arduino code to be able to capture an image using an HTTP request.esp32-controller
: the ESP-32 C6 code that is responsible to control the motors and press the remote's buttons on the main plate.helping_hand
: the Flutter code used to create the application from which we control the device and interact with the different microcontrollers in the system.design
directory: contains all the 3D modeling files and exports along with the laser cut sheets.diagrams
directory: contains all the protocol diagrams specifying how the different parts of the system interact with one another.documentation
directory: contains all the pictures and schematics to be able to reproduce this project.proposal
directory: the LaTeX document used to write the original project proposal.reports
directory: our first meetings reports to agree on what to do together.This section is all about providing instructions and material to be able for anyone to reproduce this project by themselves.
This project is mainly composed of electronic components, laser cut 4mm MDF plates and 3D printed parts.
Here are the steps to follow to build the main body (and the camera module) of the system:
For reference, here are some pictures of the full result assembly:
Here are all the required electronic components:
Wire everything together as indicated in the following schematic:
Note: make sure to leave some length on the cables used on the travelling parts.
For reference, here are some pictures of the resulting cable management (done under the board):
We have four system components that need to interact with eachother in order to function properly. The following diagram summarizes the different intractions between the components:
All the system is driven by a Flutter cross-platform application.
You can install the Flutter dev environment by following the installation instructions (on any OS): https://docs.flutter.dev/get-started/install.
Then navigate to the directory of the application, code/helping_hand
.
You can now launch the application on your platform of choice, on web for example flutter run -d chrome
.
To build for other targets, you can follow the standard Flutter building procedures: https://docs.flutter.dev/deployment.
Here are a few pictures of the application interface:
We use the Arduino IDE to flash the two ESP-32 boards:
code/esp32-cam/esp32-cam.ino
. For more information about this module, please refer to the ESP-32 CAM README.code/esp32-controller/sketch_webServer/sketch_webServer.ino
. For more information about this module, please refer to the ESP-32 C6 README.We used a Python web server (running on any computer or in the cloud) powered by the Flask framework.
Refer to the associated README to setup the python server: Python CVserver Setup README.