search

zkmkarlsruhe / ofxTensorFlow2

TensorFlow 2 AI/ML library wrapper for openFrameworks
Other
109 stars 16 forks source link

Add example_movenet #11

Closed danomatika closed 2 years ago

danomatika commented 2 years ago

Add the example_movenet contribution into develop.

bytosaur commented 2 years ago

cool thanks for the update everyone!

@stephanschulz i know you wanted to look into hand models but this may be interesting for you as well :)

stephanschulz commented 2 years ago

movenet is such a great addition.

Personally I like that movenet and posenet can detect people + their faces even with masks on. The web demo works just fine. But the OF example_movenet does only work when an upper body is part of the video feed. Are there settings that I would need to adjust?

https://storage.googleapis.com/tfjs-models/demos/pose-detection/index.html?model=movenet

Screen Shot 2022-01-24 at 5 12 30 PM

OF example without detection because I am too close to the camera.

Screen Shot 2022-01-24 at 5 14 39 PM
bytosaur commented 2 years ago

Hm can't tell what model is in used in the googleapi example. There are 3 different models of movenet on tfhub. We use the multi person model. I think you need to feed in full body shots.

stephanschulz commented 2 years ago

ah I see.

I went here: https://www.tensorflow.org/hub/tutorials/movenet and click on the first live demo link. I guess it's the "MoveNet: Ultra fast and accurate pose detection model." ?

stephanschulz commented 2 years ago

ok I think I know what you are referring to. You used the movenet/multipose/lightning https://tfhub.dev/s?q=movenet model. While the web demo probably is using movenet/singlepose/lightning.

Do you think it is possible to integrate the single pose model too?

stephanschulz commented 2 years ago

I downloaded the single lightning model and placed it to /Applications/of_v0.11.2_osx_release/addons/ofxTensorFlow2/example_movenet_single/bin/data/model

Then changed these line to: https://github.com/zkmkarlsruhe/ofxTensorFlow2/blob/main/example_movenet/src/ofApp.h#L26-L27

  std::size_t nnWidth = 192; //512;
  std::size_t nnHeight = 192; //288;

Side note: I am not sure why nnHeight had to be 288 for the multi pose version?

Something now gets detected but it is intermittent and not accurate.

https://user-images.githubusercontent.com/1054816/151020502-6c15c663-f8f2-4148-86ca-8ad48ffbc67c.mov

Would you know what else I might need to change?