search

spoonsso / dannce

MIT License
214 stars 30 forks source link

COM deviate a lot from animal #133

Closed PIPIhuangy closed 1 year ago

PIPIhuangy commented 2 years ago

Hello! I used 40 labeled frames to predict COM, however I got a bad result. COMs deviate a lot from the animal in nearly all views. 1669383849966 image

I used 8 cameras and in some cases a half of cameras can't capture the animal, but I still used triangulation in hand-label step to triangulate the animal. Could this be contributing to the problem?

Thanks for your help!

spoonsso commented 1 year ago

Are these frames here of 2D COM predictions? Or of 3D COM projections into these cameras? If the latter, then this is probably due to how the animal is missing in half the views. The COM triangulation will use all 8 cameras, so if the animal is missing in 4 of them, the 2D COM in those 4 will be bad, leading to an erroneous 3D prediction.

To check how good your 2D COM predictions are in individual views, you can use the com_debug config parameter, e.g. com_debug: Camera5 will save the 2D COMs overlaid on each frame during prediction along with the 2D output heatmap.

If the 2D COM predictions look good when the animal is in the frame, you can try to use the comthresh config option to throw away the bad views when using dannce-train and dannce-predict. Any 2D COM heatmaps with maxima below comthresh (max thresh is 1.0) will be discarded. To use this comthresh filtering, you need to point to com3d.pickle rather than com3d.mat in your dannce config.

PIPIhuangy commented 1 year ago

Hi, thank you for your reply! These are 3D COM projections into each frames of cameras, and I know what causes the error based on your reply. I will try your advice, thank you!