Open KenR22 opened 1 year ago
Hi Ken,
It's not a problem to have unlabeled frames. The network ignores them during training and it should not bias the model. You don't have to suppress them, that feature exists to get rid of reprojected keypoints that you don't want to have as part of your trainingset. (For example if your subject is completely covered by a wall and the projected points are all on the wall, in that case you don't want the network to get trained on predicting the entirely occluded subject)
I would estimate you need at the very least around 500 labeled frames (meaning roughly 100 frame-sets in your case) to start getting decent results. For accurate tracking most of the time around a 1000 frames are needed. This really depends on the setup and the complexity of the movement though, so it's only a very rough estimate. How many do you have labeled currently?
I think the approach you are currently taking is better than downsampling everything to 360p. If anything I would rather consider upsampling the 360p videos by a factor of two, maybe you could even try playing around with one of the AI upsampling tools that are fairly common these days. I had mixed results playing around with that a bit on other datasets, but it might be worth a shot. (Keep in mind that you'll have to do the same to the calibration videos in that case of course).
One additional troubleshooting step you could take is to let the network create 2D predictions for just a single camera view. If that works well but the 3D predictions look bad it's more likely to be an issue with the configuration of your 3D network, rather than the amount of labeled data.
If you don't mind you can also share the trainingset you created with me (thueser@dpz.eu is my email) and I can have a look and see if there is some configuration that needs to be tweaked, or give you other suggestions if I notice something that can be improved.
Hope this helps and best, Timo
Let me try bumping up the number of labeled frames we have and will reach with our training data later on if that does not work.
Thanks again!
Sounds good! Please let me know if more Annotations improve your results! :)
Hi Timo,
Just wanted to let you know that more annotation helped increase the accuracy of 2D, and 3D is somewhat working.
One part of our subject is detected well, but other parts are not as well. This leads me to the following questions:
Thanks again!
Hi,
Reaching out again with specific questions.
Thanks
Hi Ken,
Sorry for the late reply!
Hope this helps, but please feel free to ask more questions if anything is still unclear!
Best, Timo
Hi again,
here's a link to the manual page that describes the config parameters: https://jarvis-mocap.github.io/jarvis-docs/manual/6_training_hybridnet/
Hope this helps! :)
This is great! I will look into it and let you know if we can get it running Thanks!
Hi Timo,
Currently our system projects the subject on half of the cameras(4 out of 9) at a time. I have some queries regarding this. We did a first pass of the all the models in the hybrid net and it does not look very good.
Thanks again for your help and this great tool!
-Ken