Add a custom video processing cookbook for fitness training with Inference, Supervision, and GPT integration.

[Matvezy]

Description

Please include a summary of the change and which issue is fixed or implemented. Please also include relevant motivation and context (e.g. links, docs, tickets etc.).

This cookbook demonstrates a custom video processing use-case: creating a video-based fitness trainer. It uses Roboflow Inference to track body keypoints, calculate joint angles, and annotate them on video frames in real-time with Supervision. Additionally, it integrates GPT to provide fitness advice based on video frames and recorded data. The goal of this cookbook is to demonstrate how integrating Inference can enhance foundation models with vision capabilities like GPT 4o for more accurate, data-informed video processing.

List any dependencies that are required for this change.

openai, supervision: Installed automatically in the notebook.

Type of change

Please delete options that are not relevant.

• [x] New feature (non-breaking change which adds functionality)

How has this change been tested, please provide a testcase or example of how you tested the change?

This notebook was tested by running end-to-end on a sample fitness video. The outputs included annotated video frames with joint angles and GPT-generated fitness advice. Since the notebook runs independently, it does not affect other code or dependencies.

Any specific deployment considerations

For example, documentation changes, usability, usage/costs, secrets, etc.

Include a link to the cookbook in documentation for similar use cases (e.g., video processing, Yolo-Pose models, or GPT integrations).

Ensure users are aware of potential costs when using GPT and processing video data.

Docs

• [ ] Docs updated? What were the changes: None

[CLAassistant]

All committers have signed the CLA.

[PawelPeczek-Roboflow]

Hi there, thanks for contribution, we do really appreciate that.

Few comments and I believe we could move forward with this change:

• could you please place the notebook in /examples/community/fitness-gpt-coach directory and add README in new dir, with description that in this catalog people may find community-created examples with clear author attribution for each notebook / example
• could you please add clear disclaimer that the script should not be taken as professional fitness advice, warning users to stay mindfull at the gym
• could you please tell a little bit more about the copyright of the video
• could you clarify why there is --no-check-certificate for wget (maybe we could place the media on our hosting)

Comments about the content of the notebook (may be treated as potential enhancements or just ignored):

• the center of barbel trajectory seems to be important for squats - people would lean more / less forward depending on their femur length and there is nothing arbitrary that can be said about doing or not doing so
• there are different variations of squats, for back- ones one may perform high bar or low bar and the correct movements pattern will differ
• what is particularly important to track in a function of squat depth is a but wink - would be nice to have it analysed + maybe suggested adjustments regarding the stance width

[Matvezy]

Thank you very much for your feedback Paweł! Your comments and suggestions for potential enhancements are great. I will address all of them and make another PR shortly.

[PawelPeczek-Roboflow]

is that you who is squating at the video?

[Matvezy]

is that you who is squating at the video?

Yes, I took that video on purpose so that it wouldn't have any copyright issues. I added the --no-check-certificate since I have the video stored on google drive and in some cases the download without it didn't work. So if we could place the media on your hosting that would be a great solution for that!

[Matvezy]

Hello Paweł,

I just committed a new version with the changes you requested. Your suggestions for potential enhancements are fantastic! Unfortunately, I was only able to implement one of them—the butt wink detection and analysis. For this, I added left and right pelvic tilt angle detection and processing.

The ideas for tracking the barbell's center of trajectory and incorporating different squat variations are excellent too. However, they would require quite a bit of additional code, which might push this notebook beyond the intended simplicity of a cookbook example. For now, I’ve decided to move forward with this version. That said, if any of those ideas are something you'd really like to see, please let me know, and I'll be happy to add them!

Please let me know if you have any thoughts, questions, or concerns. Thank you so much!

[PawelPeczek-Roboflow]

ok, that works - when I have some free time this week I will push the video on our hosting and then that would be simple wget to pull it- I will provide a link, then you can update a notebook and we are good to go

[Matvezy]

Awesome, thank you so much Paweł! I'm ready to make the changes anytime.

[PawelPeczek-Roboflow]

Ok, could you please just !wget 'https://media.roboflow.com/inference/notebooks/community/fitness_gpt_coach/squat_video.mp4' -O squat_video.mp4 - that should work now

also - please attribute you as the author of the example in readme

[PawelPeczek-Roboflow]

and could you also remove gif bin from the repository? I will squash merge such that there should not be a mark of the bins in git history

[PawelPeczek-Roboflow]

ok, URL left to be changed and we are good to go

[Matvezy]

Oh, I thought I changed a URL cell in my notebook, ie it is now !wget 'https://media.roboflow.com/inference/notebooks/community/fitness_gpt_coach/squat_video.mp4' -O squat_video.mp4

I can see it in my most recent "Fixes" commit, is it not displaying for you there? I may be missing something here.

[PawelPeczek-Roboflow]

a ok, did not get displayed in diff

[PawelPeczek-Roboflow]

ok, we are good to go, I will merge and release tmr and will try to get a decent exposure.

Thanks for contribution

[Matvezy]

Awesome, thank you so much Paweł! I really appreciate all of your help and great advice! I'm glad I got to contribute!