Open JustHere27 opened 1 day ago
Hi,
From what I can see here, Caliscope and Pose2Sim don't provide the same pose estimation output.
Pose2Sim supports Caliscope calibration so you can use it for your calibration if you prefer; but other than that I would encourage you to try both tools independently from the pose estimation stage, and see what best fits your needs.
@mprib may be able to tell you more about it!
Actually, I just realized that although the calibration correctly loads in the Blender addon: https://github.com/davidpagnon/Pose2Sim_Blender ,
there is a little bug in the recap messages of personAssociation and triangulation. I'm fixing it now and will release a new version by the end of the day, that you will be able to install with pip install Pose2Sim -U
NB: Since there is only one person in the scene, you can skip this personAssociation.
I just released the fix, would you mind telling me if it works on your end?
P.S.: I misread this message and thought you had skipped pose estimation. I was wrong, you did things correctly! (although you can still skip personAssociation() )
Pose2Sim.poseEstimation() # I skipped the calibration as I am using the config.toml fom the sample data
Thanks for the quick and clear response.
I tried it again after running pip install Pose2Sim -U
, however I got another error during triangulation.py
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[1], line 4
2 Pose2Sim.poseEstimation()
3 Pose2Sim.synchronization()
----> 4 Pose2Sim.triangulation()
5 Pose2Sim.filtering()
6 Pose2Sim.markerAugmentation()
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\Pose2Sim.py:375, in triangulation(config)
372 logging.info(f"Project directory: {project_dir}")
373 logging.info("---------------------------------------------------------------------\n")
--> 375 triangulate_all(config_dict)
377 end = time.time()
378 elapsed = end-start
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\triangulation.py:956, in triangulate_all(config_dict)
936 c3d_paths = [convert_to_c3d(t) for t in trc_paths]
938 # # Reorder TRC files
939 # if multi_person and reorder_trc and len(trc_paths)>1:
940 # trc_id = retrieve_right_trc_order(trc_paths)
(...)
954
955 # Recap message
--> 956 recap_triangulate(config_dict, error_tot, nb_cams_excluded_tot, keypoints_names, cam_excluded_count, interp_frames, non_interp_frames, trc_paths)
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\triangulation.py:300, in recap_triangulate(config_dict, error, nb_cams_excluded, keypoints_names, cam_excluded_count, interp_frames, non_interp_frames, trc_path)
298 calib_file = glob.glob(os.path.join(calib_dir, '*.toml'))[0] # lastly created calibration file
299 calib = toml.load(calib_file)
--> 300 cam_names = np.array([calib[c].get('name') for c in list(calib.keys())])
301 cam_names = cam_names[list(cam_excluded_count[0].keys())]
302 error_threshold_triangulation = config_dict.get('triangulation').get('reproj_error_threshold_triangulation')
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\triangulation.py:300, in <listcomp>(.0)
298 calib_file = glob.glob(os.path.join(calib_dir, '*.toml'))[0] # lastly created calibration file
299 calib = toml.load(calib_file)
--> 300 cam_names = np.array([calib[c].get('name') for c in list(calib.keys())])
301 cam_names = cam_names[list(cam_excluded_count[0].keys())]
302 error_threshold_triangulation = config_dict.get('triangulation').get('reproj_error_threshold_triangulation')
AttributeError: 'int' object has no attribute 'get'
I ran:
from Pose2Sim import Pose2Sim
Pose2Sim.poseEstimation()
Pose2Sim.synchronization()
Pose2Sim.triangulation()
Pose2Sim.filtering()
Pose2Sim.markerAugmentation()
Pose2Sim.kinematics()
Hi, sorry about that, I tried to bring a quick fix to something that jumped to mind, but without testing it... I'll properly test it tonight and keep you updated!
Thank you for the mention to bring this to my attention @davidpagnon. I had been unaware of the OpenPose format, just defaulting to my preference for tabular tidy-style data. Obligatory xkcd.
I've opened an issue to jog my memory to revisit this as time permits: https://github.com/mprib/caliscope/issues/650
@davidpagnon No worries, I am happy that you were willing to help.
Hi again,
I had another look at it and upgraded Pose2Sim, it should not throw any error anymore.
Now, here are two more issues:
poseEstimation() does not work well with images when they are rotated. Until today, I either filmed videos upright, or used cameras that detected their own rotation (phone, action camera...), so I did not have this problem.
I had a look and there is a button in CaliScope GUI to rotate videos. This is then specified in the calibration file as rotation_count
. I have not implemented such a feature, but I guess it would be a good idea!
In the meantime, you can rotate your videos and your calibration file with the script below.
Results are still far from perfect. Here are my current guesses:
Either way, if you ever have a chance to test Pose2Sim from a Caliscope calibration on other data, I would be quite interested in knowing how it goes!
# ROTATE VIDEOS AND CALIBRATION FILES
# pip install moviepy
# Go to your videos folder and run this script
import os
from pathlib import Path
import toml
import numpy as np
import cv2
from moviepy.editor import *
from Pose2Sim.common import rotate_cam
videos_files = ['port_1.mp4', 'port_2.mp4', 'port_3.mp4']
rotation = [-90, -90, -90]
calib_file = '../calibration/config.toml'
# Rotate videos
for vid, rot in zip(videos_files, rotation):
Path.rename(Path(vid), f'{vid}.old')
clip = VideoFileClip(f'{vid}.old')
clip = clip.rotate(rot)
clip.write_videofile(vid)
clip.close()
# Rotate calibration file
calib = toml.load(calib_file)
new_calib = calib.copy()
Path(calib_file).rename(Path(calib_file).with_suffix('.toml.old'))
cal_keys = [c for c in calib.keys()
if c not in ['metadata', 'capture_volume', 'charuco', 'checkerboard']
and isinstance(calib[c],dict)]
for cam, rot in zip(cal_keys, rotation):
new_calib[cam]['size'] = calib[cam]['size'][::-1] if rot==90 or rot==-90 else calib[cam]['size']
K = np.array(calib[cam]['matrix'])
K[0,0], K[1,1] = (K[1,1], K[0,0]) if rot==90 or rot==-90 else (K[0,0], K[1,1])
K[0,2], K[1,2] = (calib[cam]['size'][0]-K[1,2], K[0,2]) if rot == -90 \
else (K[1,2], calib[cam]['size'][1]-K[0,2]) if rot == 90 \
else (calib[cam]['size'][1]-K[0,2], calib[cam]['size'][0]-K[1,2])
new_calib[cam]['matrix'] = K.tolist()
r,t = rotate_cam(calib[cam]['rotation'], calib[cam]['translation'], ang_z=-np.radians(rot))
r = np.squeeze(cv2.Rodrigues(r)[0])
new_calib[cam]['rotation'], new_calib[cam]['translation'] = r.tolist(), t.tolist()
with open(new_calib_file, 'w') as f:
toml.dump(calib, f)
I figured out where the problem was!
After all, it was not about handling Caliscope files, nor about distortion (although I should probably tackle it one day or another). The issue was in the script for rotating the calibration file; more specifically, in the optical center K[0,2], K[1,2]
.
I edited my above answer with the correction.
EDIT: reedited it now.
I tried to run an analysis with the caliscope sample data. I am unclear on what is causing the error.
Any assistance would be greatly appreciated.
It says that
calib_cam1['matrix'][0][0]
causedTypeError: 'int' object is not subscriptable
. However, in calibration config.toml[cam_1]
had:matrix = [[1086.1181825737008, 0.0, 946.8128791033895], [0.0, 1080.8281918966268, 585.7453083609911], [0.0, 0.0, 1.0]]
I tried running :
It stopt in
Pose2Sim.personAssociation()
with this stacktrace:this was my file structure was:
my calibration config.toml was :
I took the my root config.toml from Demo_SinglePerson. All I changed was
convert_from
in[calibration.convert]
: