Research about models for gestures

[idosius]

In case we need to recognize gestures, e.g. raising your hands in the air, research how this may be achieved

[idosius]

Regarding facial detection, stumbled upon this: https://github.com/justadudewhohacks/face-api.js

Articles: https://levelup.gitconnected.com/do-not-laugh-a-simple-ai-powered-game-3e22ad0f8166 https://itnext.io/face-api-js-javascript-api-for-face-recognition-in-the-browser-with-tensorflow-js-bcc2a6c4cf07

[idosius]

Hand gestures:

• https://github.com/andypotato/fingerpose
• https://www.youtube.com/watch?v=9MTiQMxTXPE

[idosius]

For body movements, I haven't found a pre-trained model yet. Most likely, we will need to train our own model with video data.

This guy did it with Mortal Kombat: https://blog.mgechev.com/2018/10/20/transfer-learning-tensorflow-js-data-augmentation-mobile-net/

This girl with Street Fighter 2 (see end of video): https://www.youtube.com/watch?v=rwFiFWI23Rw

ChatGPT answer:

To recognize specific body movements or poses using TensorFlow pose detection, you can define custom pose models and train them on the specific movements or poses you want to recognize. Here are the general steps you can follow:

Collect a dataset of images or videos that show the movements or poses you want to recognize. You can use a webcam to capture live data or search for existing datasets online.

Annotate the dataset with labels for the movements or poses you want to recognize. You can use software tools like VGG Image Annotator or LabelImg to annotate your dataset with key points for each pose.

Train a custom pose model using the annotated dataset. You can use pre-trained models like PoseNet or MobileNet to train your custom pose model. For example, you can fine-tune the weights of a pre-trained model using transfer learning to recognize the movements or poses in your dataset.

Test and evaluate the performance of your custom pose model on new data to ensure it can recognize the desired movements or poses.

Once you have trained your custom pose model, you can use it to recognize the specific movements or poses you defined. To do so, you can use the pose estimation algorithm to detect the keypoints in a new image or video frame and then classify the pose based on the positions of those keypoints.

Note that defining custom pose models can require significant amounts of data and computational resources, and may involve some trial and error to optimize the model's performance. However, it can be a powerful way to tailor pose detection to your specific needs.