layout: default title: Home permalink: /Overview.html
Overview
This project intends to deal with integrating and building a object detection framework and its dectection based on camera and peripheral micro-controller (Arduino BLE 33 mBed) connected to a NVIDIA Jetson Nano acting as a hub. For the purpose of demonstration we would be attaching a low power camera attached to the micro-controller to act as an inference engine built over Machine Learning Framework to interact with Jetson Nano and form a feedback loop to alarm a system of detection of person in frame of camera. The aim of this project is to train a machine learning model and deploy over low-power, low-memory micro-controllers and form feedback system of the Embedded Network Hub (Jetson). This would include bringing up completely independent hardware which includes the controlling unit, sensors, and software bring up, which includes Operating systems, communication protocols, coding styles, and languages.
Using low-level micro-controllers is cheap and provides more opportunity (future goal) to scale the product under development and testing providing both financial and engineering benefits.
NVIDIA Jetson acts as a server for the peripheral boards(clients). The term voting technique indicates that at any given time the reading that is received from the majority of the sensors will be considered for further actions.
Major Goals:
- Yocto to compile meta-tegra.
- Running Tegra-demo-distro with Arduino Setup.
- Setup Camera board and Communication.
- Create custom UART-server, API for integration and communication with externel micro-controller.
- Run AI Image recognition project working with on image on Host/Hub (Jetson) and/or Arduino BLE 33 with Camera.
System works as a hard real time system, which captures camera feed at set time interval irrespective of how much time the sensors take to send data over to Nano. The overaching goal of this project would be create a Hub and Client mechanism such that machine learning can be deployed on micro-controllers with and without GPU's.
Block Diagrams
Hardware Block Diagrams
Software Block Diagrams
Functional Block Diagrams
See Software Block Diagram
API
Target Build System
We intend to use Yocto as the bring up tool to generate Linux image on NVIDIA Jetson Nano.
Hardware Platform
Intended hardware platforms for this project:
Open Source Projects
Previously Used Content
- Yocto
- Signal handlers
- Clocks
- Init Scripts and Rootfs Overlays
New Content
- Yocto Image for Jetson Nano
- Yocto Image for Jetson Docker
- Creating Modules and scripts for Arduino BLE 33
- Custom UART-Server
- Interface for Jetson and Arduino
- Case study for TinyML
- Deploying TinyML
Shared Material
Source Code Organization
- Project Webpage: Dynamically Generated Documentation
- Project Overview: Project Synopsis
- Project Schedule: Project Schedule
- Caleb's Repository: One-click-build script Focused
- Arpit's Repository: Image Development Focused
- Zach's Repository: Driver & Module Code Focused
Group's Overview
Team Project Members
- Caleb: Yocto Image Setup and build for Jetson Tegra, Dockerfile 1-click-image
- Zach: Serial communication Between Subsystems
- Arpit: Machine Learning Framework and TinyML Setup and yocto compatibility.
- Tiny Machine Learning deployment - [Model creation, Neural network training, Tensorflow Micro]
- Computer Vision : Integrate Camera and peripheral to detect Person in frame.
- Code Porting and Tensorflow dependencies.
- { Side Project }: Dummy linux wifi driver for cfg80211 USB Wifi Module
Schedule Page
- Project Schedule: Project Schedule