search

kevinwylder / sbvision

Identify skateboards in images
0 stars 0 forks source link

Pose Estimation Annotations #3

Open kevinwylder opened 4 years ago

kevinwylder commented 4 years ago

Pose estimation is an important direction of this project, how we get to that dataset is another beast. I see two options:

  1. python we use this python port of posenet on the backend with go subprocessing python for uploaded frames.
  2. we directly use posenet through tensorflow.js on the user's browser to extract keyframes.

I'm leaning towards option 2 because it would be nice to have all the data collection process isolated into javascript, leaving python to be purely for training our model. But doing this either expands the scope of what our model needs to learn, or pushes off the inevitability of needing to learn how to do pose estimation in python.

kevinwylder commented 4 years ago

Preliminary results are.... disheartening.

It looks like the dataset used to train PoseNet does not represent skateboarding videos. See for yourself

Screen Shot 2020-03-04 at 9 44 12 PM Screen Shot 2020-03-04 at 9 43 11 PM Screen Shot 2020-03-04 at 9 50 26 PM

These results are far from being able to physically understand the wireframe of the individual, looks like we'll have to take another approach