MishMash hackathon is India’s largest online diversity hackathon. The focus will be to give you, regardless of your background, gender, sexual orientation, ethnicity, age, skill sets and viewpoints, an opportunity to showcase your talent. The Hackathon is Live from 6:00 PM, 23rd March to 11:55 PM, 1st April, 2020
2
stars
12
forks
source link
Midopolarysis- Aiden - XR / Mobility / FinTech/ Deep Tech or Machine Learning / Ed-Tech / Social Impact #119
Midopolarysis- Aiden - XR / Mobility / FinTech/ Deep Tech or Machine Learning / Ed-Tech / Social Impact
ℹ️ Project information
You can select any one theme from - XR / Mobility / FinTech/ Deep Tech or Machine Learning / Ed-Tech / Social Impact
Mobility, FinTech and Ed-Tech are Open Innovation Themes
XR has 3 sub-categories that you can choose from:- Epidemic, Urban, Remake
Deep Tech or Machine Learning is sponsored by UNILEVER and has 3 Problem Statements. If you pick this theme, you need to declare which problem statement you are going to work on
Social Impact has only 1 Problem Statement - So, if you pick this theme you just need to select this Theme and say you will work on the Problem Statement in the Idea Brief Field
Project Name: Aiden
Short Project Description:It is a web app utilising tensorflow.js, browser-based Machine Learning library, to enable accessible physiotherapy for the Visually Impaired and other people as well - talking through exercises by responding to users' postures in real-time.
Deep Tech - Problem Statement - 3: If you have chosen to work on the problem statement - 3 then please submit both models based on the two datasets provided to you.
Deep Tech - Problem Statement - 2: If you have chosen to work on the problem statement - 2 then please provide the reference for your dataset.
Azure Services Used- Kindly mention the Azure Services used in your project.
🔥 Your Pitch
_Technology
Machine Learning - tensorflow.js
Our application uses a tensorflow.js (browser-based) model to make predictions on the state of the current user's pose. It has been trained on a dataset of images created by us (~300 images per pose) to predict whether the position is correct, or incorrect - and what makes it so. We have used Azure Machine Learning Studio, an Azure Machine Learning tool, to train our models in the various physiotherapy poses. Azure Cognitive Services Speech-to-Text API was also used to enable the application to be accessible by the visually impaired. The user can start their exercises via speech in various languages using Azure Translator Speech API remotely and this is more convenient and easier to use for our target audience. The application utilizes Azure Cognitive Services for text-to-speech. This is useful for the visually impaired as they can hear if they are in the right position as the application will tell them to adjust their posture if incorrect. We also use the webcam to track the user's movement which is fed as input to the posenet machine learning model and outputs posture image on the user's body. Key Azure Services that have been used in our product:
Client Folder
The web application is located in the client's folder. The web application consists of two files: index.html and index.js.
Index.html
The index.html contains all the HTML that forms the backbone of the website.
We have used the bootstrap open-source CSS framework for our front-end development.
Index.js
index.js contains the Javascript code for the web application. This works with HTML to add functionality to the site.
Loads the model and metadata and handles image data._
🔦 Any other specific thing you want to highlight?
Azure Services Used.
Azure Storage Services - storing machine learning model ( TF)
Azure Cognitive Services ( Inference )
Text-to-Speech
Speech-to-Text
Custom Vision ( to classify between correct and incorrect images)
Translator
Azure CDN ( three js and other libraries )
Azure Web App with Continuous Deployment
Linux Virtual Machine ( for hosting the website )
Azure CLI ( for deployment)
Azure Cloud Shell (for web app continuous deployment integration)
Azure Pipelines (Continuous deployment feature)
Visual Studio Code ( for all our life <3)
Supportability
This is fully supported on Desktop/Android Google Chrome.
✅ Checklist
Before you post the issue:
[✓] You have followed the issue title format.
[✓] You have mentioned the correct labels.
[✓] You have provided all the information correctly.
Midopolarysis- Aiden - XR / Mobility / FinTech/ Deep Tech or Machine Learning / Ed-Tech / Social Impact
ℹ️ Project information
Project Name: Aiden
Short Project Description: It is a web app utilising tensorflow.js, browser-based Machine Learning library, to enable accessible physiotherapy for the Visually Impaired and other people as well - talking through exercises by responding to users' postures in real-time.
Team Name: Midopolarysis
Team Members: Shivay Lamba @shivaylamba , Rahul Garg @rahulgarg28071998 ,Raghav Dhingra @raghavdhingra , Pulkit Midha @midopooler
Demo Link: (https://youtu.be/KSbFmtkXAg8)
Repository Link(s): https://github.com/rahulgarg28071998/MISHMASH
Presentation Link: https://docs.google.com/presentation/d/1ZK3yC5NOFcbx_t0vHgBenMPLw3xQu-EGsXshJTwyADM/edit?usp=sharing
Deep Tech - Problem Statement - 3: If you have chosen to work on the problem statement - 3 then please submit both models based on the two datasets provided to you.
Deep Tech - Problem Statement - 2: If you have chosen to work on the problem statement - 2 then please provide the reference for your dataset.
Azure Services Used- Kindly mention the Azure Services used in your project.
🔥 Your Pitch
_Technology Machine Learning - tensorflow.js Our application uses a tensorflow.js (browser-based) model to make predictions on the state of the current user's pose. It has been trained on a dataset of images created by us (~300 images per pose) to predict whether the position is correct, or incorrect - and what makes it so. We have used Azure Machine Learning Studio, an Azure Machine Learning tool, to train our models in the various physiotherapy poses. Azure Cognitive Services Speech-to-Text API was also used to enable the application to be accessible by the visually impaired. The user can start their exercises via speech in various languages using Azure Translator Speech API remotely and this is more convenient and easier to use for our target audience. The application utilizes Azure Cognitive Services for text-to-speech. This is useful for the visually impaired as they can hear if they are in the right position as the application will tell them to adjust their posture if incorrect. We also use the webcam to track the user's movement which is fed as input to the posenet machine learning model and outputs posture image on the user's body. Key Azure Services that have been used in our product:
Client Folder The web application is located in the client's folder. The web application consists of two files: index.html and index.js. Index.html The index.html contains all the HTML that forms the backbone of the website. We have used the bootstrap open-source CSS framework for our front-end development. Index.js index.js contains the Javascript code for the web application. This works with HTML to add functionality to the site. Loads the model and metadata and handles image data._
🔦 Any other specific thing you want to highlight?
Azure Services Used.
Azure Storage Services - storing machine learning model ( TF) Azure Cognitive Services ( Inference ) Text-to-Speech Speech-to-Text Custom Vision ( to classify between correct and incorrect images) Translator Azure CDN ( three js and other libraries ) Azure Web App with Continuous Deployment Linux Virtual Machine ( for hosting the website ) Azure CLI ( for deployment) Azure Cloud Shell (for web app continuous deployment integration) Azure Pipelines (Continuous deployment feature) Visual Studio Code ( for all our life <3) Supportability This is fully supported on Desktop/Android Google Chrome.
✅ Checklist
Before you post the issue: