search

danbider / lightning-pose

Accelerated pose estimation and tracking using semi-supervised convolutional networks.
MIT License
223 stars 32 forks source link

cannot fit PCA with 0 samples < 40 observation dimensions #169

Open marybethcassity opened 2 weeks ago

marybethcassity commented 2 weeks ago

Hi @themattinthehatt Sorry to bother again! I am now encountering the cannot fit PCA with 0 samples < 40 observation dimensions error. I have tracked it down to def _ensure_enough_data(self) in pca.py and am still working on resolving it, but in the mean time figured I would see if you would beat me to it! Thanks!

themattinthehatt commented 2 weeks ago

Here's some more info on the PCA loss: https://lightning-pose.readthedocs.io/en/latest/source/user_guide_advanced/unsupervised_losses.html#pose-pca

One note is that you'll probably want to remove the (what I assume are static) keypoints like camera_frontleft from columns_for_singleview_pca.

Another important thing to note is that right now we have to discard a frame for computing the PCA subspace if any of the keypoints included in columns_for_singleview_pca are missing, i.e. you didn't label the keypoint because it was occluded. Therefore you should limit the keypoints you include in the PCA loss to ones that are typically not occluded, and which you have lots of ground truth labels for. My guess is that you have lots of unlabeled keypoints in your training dataset (which is totally fine) but it means none of the frames are available for computing the PCA loss. So try removing any oft-occluded keypoints from columns_for_singleview_pca and see if that helps matters.

marybethcassity commented 2 weeks ago

Thanks! I remember reading about removing keypoints with a lot of occlusion but didn't realize how important it may be! I'll let you know what I figure out works.