perfanalytics / pose2sim

Markerless kinematics with any cameras — From 2D Pose estimation to 3D OpenSim motion
https://perfanalytics.github.io/pose2sim/
BSD 3-Clause "New" or "Revised" License
271 stars 50 forks source link

Errors while tringulating #110

Closed hmejbri closed 4 months ago

hmejbri commented 5 months ago

I'm using the demo data and im getting these errors while triangulating :

---------------------------------------------------------------------
INFO:root:Triangulation of 2D points for S00_P00_T01_PoseTrial, for all frames.
INFO:root:On Friday 21. June 2024, 10:19:09
INFO:root:---------------------------------------------------------------------
INFO:root:
Project directory: /home/ubuntu/pfe_mocap/S00_P00_Participant/S00_P00_T01_PoseTrial
INFO:root:Interpolation was not possible for person 0. This means that not enough points are available, which is often due to a bad calibration.
INFO:root:
INFO:root:Mean reprojection error for RHip is 7.3 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 0 to 4, 34 to 34, 59 to 59, 75 to 77, 87 to 90 were interpolated.
INFO:root:Mean reprojection error for RKnee is 8.5 px (~ 0.004 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 35 to 36, 47 to 52, 80 to 80, 83 to 90 were interpolated.
INFO:root:  Frames 0 to 12, 21 to 33, 63 to 75 were not interpolated.
INFO:root:Mean reprojection error for RAnkle is 7.9 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 18 to 18, 25 to 33, 35 to 41, 84 to 89 were interpolated.
INFO:root:  Frames 55 to 74 were not interpolated.
INFO:root:Mean reprojection error for RBigToe is 6.7 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 22 to 23, 87 to 89 were interpolated.
INFO:root:  Frames 0 to 17, 25 to 42, 54 to 74 were not interpolated.
INFO:root:Mean reprojection error for RSmallToe is 6.4 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 87 to 89 were interpolated.
INFO:root:  Frames 0 to 17, 22 to 41, 55 to 74 were not interpolated.
INFO:root:Mean reprojection error for RHeel is 7.8 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 19 to 21, 26 to 33, 35 to 41, 47 to 47, 49 to 49 were interpolated.
INFO:root:  Frames 0 to 15, 56 to 89 were not interpolated.
INFO:root:Mean reprojection error for LHip is 7.6 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 0 to 0, 2 to 2, 10 to 10, 40 to 42 were interpolated.
INFO:root:  Frames 44 to 74, 77 to 90 were not interpolated.
INFO:root:Mean reprojection error for LKnee is 7.9 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 0 to 8, 10 to 10, 16 to 16, 64 to 65, 67 to 67, 85 to 85, 87 to 89 were interpolated.
INFO:root:  Frames 27 to 62 were not interpolated.
INFO:root:Mean reprojection error for LAnkle is 10.4 px (~ 0.005 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 0 to 8, 10 to 10, 54 to 54, 56 to 56, 65 to 65, 67 to 67, 86 to 89 were interpolated.
INFO:root:Mean reprojection error for LBigToe is 11.6 px (~ 0.005 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 0 to 89 were not interpolated.
INFO:root:Mean reprojection error for LSmallToe is nan px (~ nan m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 0 to 90 were not interpolated.
INFO:root:Mean reprojection error for LHeel is 2.8 px (~ 0.001 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 0 to 8, 10 to 10, 54 to 54, 56 to 56, 65 to 65, 67 to 67, 89 to 89 were interpolated.
INFO:root:Mean reprojection error for Neck is 8.8 px (~ 0.004 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 54 to 54, 56 to 56, 61 to 62, 64 to 67, 90 to 90 were interpolated.
INFO:root:  Frames 0 to 11, 22 to 44 were not interpolated.
INFO:root:Mean reprojection error for Head is 7.1 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 1 to 1, 3 to 3, 37 to 37, 63 to 63, 65 to 65, 67 to 67, 90 to 90 were interpolated.
INFO:root:  Frames 14 to 35, 45 to 61 were not interpolated.
INFO:root:Mean reprojection error for Nose is 9.4 px (~ 0.004 m), reached with 2.02 excluded cameras. 
INFO:root:  Frames 74 to 74, 77 to 79, 83 to 86, 89 to 90 were interpolated.
INFO:root:  Frames 0 to 33, 46 to 72 were not interpolated.
INFO:root:Mean reprojection error for RShoulder is 7.9 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 12 to 15, 17 to 17, 46 to 48, 54 to 54, 56 to 56, 65 to 65, 67 to 67, 78 to 79, 87 to 89 were interpolated.
INFO:root:  Frames 0 to 10, 30 to 43 were not interpolated.
INFO:root:Mean reprojection error for RElbow is 5.0 px (~ 0.002 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 0 to 8, 60 to 60, 62 to 64, 66 to 66, 82 to 82, 88 to 88, 90 to 90 were interpolated.
INFO:root:  Frames 68 to 78 were not interpolated.
INFO:root:Mean reprojection error for RWrist is 6.4 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 5 to 8, 10 to 10, 16 to 21, 54 to 54, 56 to 56, 65 to 65, 67 to 75, 77 to 80, 82 to 82, 84 to 84, 90 to 90 were interpolated.
INFO:root:Mean reprojection error for LShoulder is 6.8 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 24 to 24, 28 to 28, 48 to 56, 58 to 65, 67 to 67, 85 to 85, 90 to 90 were interpolated.
INFO:root:  Frames 0 to 14 were not interpolated.
INFO:root:Mean reprojection error for LElbow is 7.0 px (~ 0.003 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 0 to 9, 21 to 23, 25 to 33, 79 to 81, 87 to 87, 90 to 90 were interpolated.
INFO:root:  Frames 35 to 77 were not interpolated.
INFO:root:Mean reprojection error for LWrist is 9.2 px (~ 0.004 m), reached with 2.0 excluded cameras. 
INFO:root:  Frames 0 to 4, 88 to 90 were interpolated.
INFO:root:  Frames 11 to 68, 73 to 86 were not interpolated.
INFO:root:
--> Mean reprojection error for all points on all frames is 7.3 px, which roughly corresponds to 3.2 mm. 
INFO:root:Cameras were excluded if likelihood was below 0.3 and if the reprojection error was above 15 px.
INFO:root:Gaps were interpolated with cubic method if smaller than 10 frames.
INFO:root:In average, 2.0 cameras had to be excluded to reach these thresholds.
INFO:root:Camera int_cam_1_img was excluded 74% of the time, Camera int_cam_2_img: 68%, Camera int_cam_3_img: 36%, and Camera int_cam_4_img: 22%.
INFO:root:
3D coordinates are stored at /home/ubuntu/pfe_mocap/S00_P00_Participant/S00_P00_T01_PoseTrial/pose-3d/S00_P00_T01_PoseTrial_0-91.trc.
INFO:root:

INFO:root:Limb swapping was not handled.
INFO:root:Lens distortions were not taken into account.
INFO:root:
Triangulation took 00h00m04s.

This is the output of the calibration :

INFO:root:
--> Residual (RMS) calibration errors for each camera are respectively [0.923, 1.099, 0.92, 1.035] px, 
which corresponds to [0.405, 0.315, 0.384, 0.351] mm.

INFO:root:Calibration file is stored at /home/ubuntu/pfe_mocap/S00_Calibration/Calib_board.toml.
INFO:root:
Calibration took 136.81 s.
davidpagnon commented 5 months ago

Hi, do you get good results when you convert the calibration file? I would suspect a problem in the calibration file but it is a bit hard to debug since you changed the folder structure of Pose2Sim. Do you by any chance have any other toml calibration file in your folder structure? It may not use the right one.

hmejbri commented 5 months ago

No I have no other toml calibration file in my folder structure. When i converted the calibration file I got the same error but a slightly better output, The final animation has some good frames but most of them are wrong.

Screenshot 2024-06-21 163402 Screenshot 2024-06-21 163434 Screenshot 2024-06-21 163712

davidpagnon commented 5 months ago

This is strange, I've never had such issues, and you should not have so many missing frames. Maybe you could remove your environment and all files, and then reinstall it from scratch?

hmejbri commented 5 months ago

by environment you mean pose2sim and openpose ? (i tried opensim on windows and ubuntu and got the same result so its not the problem)

davidpagnon commented 5 months ago

I was thinking of the Pose2Sim environment: conda env remove -n Pose2Sim And then do the full reinstall again. It should not take more than a few minutes

hmejbri commented 5 months ago

I reinstalled the environment but nothing changed.

davidpagnon commented 4 months ago

I still have no idea what's the problem unfortunately :/ If you can wait one or two days, we will release a new version with pose estimation natively supported (faster, more accurate, and easier to install) so it should make the process simpler. If there is still an issue at that point with the Demo data, tell me and we'll try to figure it out!

davidpagnon commented 4 months ago

For your information, he latest Pose2Sim version has been released, with the integration of pose estimation. Do pip install pose2sim --upgrade to access it :)