Problem with Triangulation using EasyMocap - ValueError: cannot reshape array of size 0 into shape (0,newaxis)

[maurice1128]

Dear David, Thank you for the awesome library, I am using 0.4.6 of Pose2Sim. I can use Halpe26 model, calibration(Easy Mocap), personAssociation, but when I tried triangulation, I encountered error: ValueError: cannot reshape array of size 0 into shape (0,newaxis).

I have four cameras my config file for triangulation is reproj_error_threshold_triangulation = 5 # px likelihood_threshold = 0.2 min_cameras_for_triangulation = 3 interpolation = 'cubic' interp_if_gap_smaller_than = 1000000 # Do not interpolate bigger gaps

The weird thing is that not all files encounter this problem, but when I change min_camera = 2, no error occurs. I was wondering what is the possible reason for this. I've read the previous answer which someone had a similar question, but because I'm not using Vicon system, I'm guessing that the reason may be different.

one frame of one camera json file 00000.json

Best regards, Maurice

[davidpagnon]

Hi Maurice,

This error means that Pose2Sim cannot find any frames with a correct triangulation. I can see a few reasons for it:

• Calibration is not good. This is the usual reason, and I have not extensively tested EasyMocap calibration conversion. So this could be it.
• It could also mean that your cameras are not synchronized, and Pose2Sim is trying to triangulate a pose from a camera with an entirely different pose from another one. I unfortunately did not find time to implement a synchronization script, but you can find a draft code or other approaches here
• Another reason would be for me to have not set up the Halpe model right. I have not extensively tested it either, so I cannot exclude some remaining issues there.

In all of these cases, the personAssociation() step should not work. If it does without increasing reproj_error_threshold_association too much, then it could be due to parameters in the triangulation step.

• Did you try to increase reproj_error_threshold_triangulation to 15 or 20?
• You probably decreased likelihood_threshold enough so don't change it.
• Also, increasing the minimum cameras for triangulation means that it won't try to triangulate with less than 3 of your cameras. This can be a bit limiting sometimes, especially with a low reprojection error threshold (and not to be critical, but I am not sure you can obtain such a small reprojection error with an EasyMocap calibration).
• Little tip: instead of increasing interp_if_gap_smaller_than to this extent, you can just set interpolation to "none"

Tell me if this helps, otherwise I'll have a deeper look into your data!

[maurice1128]

Thankyou so much for such great detail. I found out that the projection error threshold were too small, it works well after I change the reproj_error_threshold value. Thankyou again for such a wonderful work you’ve done

David PAGNON @.***>於 2023年12月21日 週四，上午3:32寫道：

Hi Maurice,

This error means that Pose2Sim cannot find any frames with a correct triangulation. I can see a few reasons for it:

• Calibration is not good. This is the usual reason, and I have not extensively tested EasyMocap calibration conversion. So this could be it.
• It could also mean that your cameras are not synchronized, and Pose2Sim is trying to triangulate a pose from a camera with an entirely different pose from another one. I unfortunately did not find time to implement a synchronization script, but you can find a draft code or other approaches here https://github.com/perfanalytics/pose2sim/tree/main?tab=readme-ov-file#camera-synchronization
• Another reason would be for me to have not set up the Halpe model right. I have not extensively tested it either, so I cannot exclude some remaining issues there.

In all of these cases, the personAssociation() step should not work. If it does without increasing reproj_error_threshold_association too much, then it could be due to parameters in the triangulation step.

• Did you try to increase reproj_error_threshold_triangulation to 15 or 20?
• You probably decreased likelihood_threshold enough so don't change it.
• Also, increasing the minimum cameras for triangulation means that it won't try to triangulate with less than 3 of your cameras. This can be a bit limiting sometimes, especially with a low reprojection error threshold (and not to be critical, but I am not sure you can obtain such a small reprojection error with an EasyMocap calibration).
• Little tip: instead of increasing interp_if_gap_smaller_than to this extent, you can just set interpolation to "none"

Tell me if this helps, otherwise I'll have a deeper look into your data!

— Reply to this email directly, view it on GitHub https://github.com/perfanalytics/pose2sim/issues/58#issuecomment-1865024170, or unsubscribe https://github.com/notifications/unsubscribe-auth/A35OXDVC2HW4WHAJGEB3P6LYKM4NBAVCNFSM6AAAAABA4KBBAOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRVGAZDIMJXGA . You are receiving this because you authored the thread.Message ID: @.***>