Closed peterlololsss closed 1 month ago
Hi, at what stage do your get the error? Can you copy paste the whole message?
A few thoughts in the meantime:
Hi, After I successfully calibrate and poseEstimate(the pose folder is there with json files), I got the error from [synchronization, personassociation and triangulation], following is the error message:
---------------------------------------------------------------------
Camera synchronization
On Tuesday 16. July 2024, 10:52:49
---------------------------------------------------------------------
Project directory: C:\Users\Ivan\Desktop\single_trial
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
Cell In[4], line 1
----> 1 Pose2Sim.synchronization()
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\Pose2Sim.py:307, in synchronization(config)
304 logging.info("---------------------------------------------------------------------")
305 logging.info(f"\nProject directory: {project_dir}")
--> 307 synchronize_cams_all(config_dict)
309 end = time.time()
310 elapsed = end-start
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\synchronization.py:295, in synchronize_cams_all(config_dict)
293 except:
294 raise ValueError(f'No json files found in {pose_dir}. Make sure you run Pose2Sim.poseEstimation() first.')
--> 295 pose_listdirs_names = sort_stringlist_by_last_number(pose_listdirs_names)
296 json_dirs_names = [k for k in pose_listdirs_names if 'json' in k]
297 json_dirs = [os.path.join(pose_dir, j_d) for j_d in json_dirs_names] # list of json directories in pose_dir
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\common.py:355, in sort_stringlist_by_last_number(string_list)
352 def sort_by_last_number(s):
353 return int(re.findall(r'\d+', s)[-1])
--> 355 return sorted(string_list, key=sort_by_last_number)
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\common.py:353, in sort_stringlist_by_last_number.<locals>.sort_by_last_number(s)
352 def sort_by_last_number(s):
--> 353 return int(re.findall(r'\d+', s)[-1])
IndexError: list index out of range
In [5]: Pose2Sim.personAssociation()
---------------------------------------------------------------------
Associating persons for single_trial, for all frames.
On Tuesday 16. July 2024, 10:54:12
---------------------------------------------------------------------
Project directory: C:\Users\Ivan\Desktop\single_trial
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
Cell In[5], line 1
----> 1 Pose2Sim.personAssociation()
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\Pose2Sim.py:354, in personAssociation(config)
351 logging.info("---------------------------------------------------------------------")
352 logging.info(f"\nProject directory: {project_dir}")
--> 354 track_2d_all(config_dict)
356 end = time.time()
357 elapsed = end-start
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\personAssociation.py:672, in track_2d_all(config_dict)
670 # 2d-pose files selection
671 pose_listdirs_names = next(os.walk(pose_dir))[1]
--> 672 pose_listdirs_names = sort_stringlist_by_last_number(pose_listdirs_names)
673 json_dirs_names = [k for k in pose_listdirs_names if 'json' in k]
674 try:
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\common.py:355, in sort_stringlist_by_last_number(string_list)
352 def sort_by_last_number(s):
353 return int(re.findall(r'\d+', s)[-1])
--> 355 return sorted(string_list, key=sort_by_last_number)
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\common.py:353, in sort_stringlist_by_last_number.<locals>.sort_by_last_number(s)
352 def sort_by_last_number(s):
--> 353 return int(re.findall(r'\d+', s)[-1])
IndexError: list index out of range
In [6]: Pose2Sim.triangulation()
---------------------------------------------------------------------
Triangulation of 2D points for single_trial, for all frames.
On Tuesday 16. July 2024, 10:54:20
---------------------------------------------------------------------
Project directory: C:\Users\Ivan\Desktop\single_trial
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
Cell In[6], line 1
----> 1 Pose2Sim.triangulation()
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\Pose2Sim.py:400, in triangulation(config)
397 logging.info("---------------------------------------------------------------------")
398 logging.info(f"\nProject directory: {project_dir}")
--> 400 triangulate_all(config_dict)
402 end = time.time()
403 elapsed = end-start
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\triangulation.py:782, in triangulate_all(config_dict)
780 # 2d-pose files selection
781 pose_listdirs_names = next(os.walk(pose_dir))[1]
--> 782 pose_listdirs_names = sort_stringlist_by_last_number(pose_listdirs_names)
783 json_dirs_names = [k for k in pose_listdirs_names if 'json' in k]
784 n_cams = len(json_dirs_names)
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\common.py:355, in sort_stringlist_by_last_number(string_list)
352 def sort_by_last_number(s):
353 return int(re.findall(r'\d+', s)[-1])
--> 355 return sorted(string_list, key=sort_by_last_number)
File ~\miniconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\common.py:353, in sort_stringlist_by_last_number.<locals>.sort_by_last_number(s)
352 def sort_by_last_number(s):
--> 353 return int(re.findall(r'\d+', s)[-1])
IndexError: list index out of range
It seems to me the three error messages are identical, though I cannot figure out which part I did goes wrong.
To answer your thought:
Thank you for your reply!:)
Hi, Looking at your error, I can see two reasons for it to crash:
But I could be missing something. Can you tell me the structure of your directory?
You can use this function for example:
def list_files(startpath):
for root, dirs, files in os.walk(startpath):
level = root.replace(startpath, '').count(os.sep)
indent = ' ' * 4 * (level)
print('{}{}/'.format(indent, os.path.basename(root)))
subindent = ' ' * 4 * (level + 1)
for f in files:
print('{}{}'.format(subindent, f))
list_files(pose_dir)
Sidenote: extrinsic calibration based on a checkerboard is generally less accurate, because you calibrate on a single plane and on a small surface. So even if your reprojection error is low on the checkerboard points, you could obtain higher reprojection errors on triangulation. However, if you do need to use a checkerboard, it is usually best to let the corners be detected automatically (unless it clearly looks like they are wrongly detected). You always get a few pixels off by clicking, while the automatic detection has a subpixel accuracy.
Hi, sorry for the late reply
Followign is the folder structure,(Very long due to json files.)
C:\Users\Ivan\miniconda3\python.exe D:\PyCharm\PyCharmProjects\PoseEstimation\List_files.py
single_trial/
Config.toml
logs.txt
Calibration/
Calib_board.toml
Object_points.trc
extrinsics/
logi_1/
extrinsic_image_1_0000.png
logi_2/
extrinsic_image_1_0000.png
logi_3/
extrinsic_image_1_0000.png
Osmo/
extrinsic_image_1_0000.png
intrinsics/
logi_1/
intrinsic_image_0000.jpg
to
intrinsic_image_0053.jpg
logi_2/
intrinsic_image_0000.jpg
to
intrinsic_image_0053.jpg
logi_3/
intrinsic_image_0000.jpg
to
intrinsic_image_0053.jpg
Osmo/
intrinsic_image_0000.jpg
to
intrinsic_image_0053.jpg
pose/
logi_1_pose.mp4
logi_2_pose.mp4
logi_3_pose.mp4
osmo_pose.mp4
logi_1_json/
logi_1_000000.json
to
logi_1_000348.json
logi_2_json/
logi_2_000000.json
to
logi_2_000348.json
logi_3_json/
logi_3_000000.json
to
logi_3_000348.json
osmo_json/
osmo_000000.json
to
osmo_000348.json
videos/
logi_1.mp4
logi_2.mp4
logi_3.mp4
osmo.mp4
Process finished with exit code 0
My confusions are:
Thank you for your data!
I finally pinned your problem: the Osmo camera does not have a number, so my sorting method fails. I'll push the fix in the next release, but in the meantime you can edit sort_stringlist_by_last_number
in Pose2Sim/common.py
(do pip show pose2sim
to check where the package is located), and replace it by:
def sort_stringlist_by_last_number(string_list):
'''
Sort a list of strings based on the last number in the string.
Works if other numbers in the string, if strings after number. Ignores alphabetical order.
Example: ['json1', 'zero', 'js4on2.b', 'aaaa', 'eypoints_0000003.json', 'ajson0', 'json10']
gives: ['ajson0', 'json1', 'js4on2.b', 'eypoints_0000003.json', 'json10', 'aaaa', 'zero']
'''
def sort_by_last_number(s):
numbers = re.findall(r'\d+', s)
if numbers:
return (False, int(numbers[-1]))
else:
return (True, s)
return sorted(string_list, key=sort_by_last_number)
Or you can just rename your camera so that it has a number in it.
Then, triangulation will not work, because your calibration is not valid (all rotations and translations are set to zero).
Config.toml
file and set:
calculate_extrinsics = true
extrinsics_method = 'board'
Now the questions you asked per email:
Additionally I have a questions regarding to scene measurement: In my understanding, the measure steps are:(Still takes a table as an example) 1.I choose the origin point 2.Based on the reference of 8 points in the graph from Pose2Sim.calculation(), I found the respective 8 points in the space around the table(Or the corners/edges of the table) 3.Using a measurer to measure the distance from the 8 points to the origin I chose, in (X,Y,Z) axies coordiates 4.Type the Axies coordiates into the config file. 5.Run Pose2Sim calibration Is my understanding correct?
Not exactly: you need to put your table in the scene, and to measure its coordinates. For example, for a table that is 2 m long, 1 m deep, and 80 cm heigh, it could be:
object_coords_3d = [[0.0, 0.0, 0.0],
[2.0 , 0.0, 0.0],
[0.0, 1.0, 0.0],
[2.0, 1.0 , 0.0],
[0.0, 0.0, 0.8],
[2.0 , 0.0, 0.8],
[0.0, 1.0, 0.8],
[2.0, 1.0 , 0.8]]
Then you replace the coordinates in Config.toml by these. You need to set extrinsics_method = 'scene'
, and then you just follow the instructions and click the corresponding points in the image! Try it and tell me if anything is still unclear.
Thank you for your help! Synchronizations work now after modifying the function:)
And as you said, triangulation doesn't work for the calibration I sent you. But before I recalibrate using scene measurement, I also ran all the stages for the one other trial I calibrated using easymocap, triangulation works however the opensim's run button (After I loaded respective files)grayed out for the .trc file unless I MarkerAugmented it, yet the resulting motion is still horrendous.
I blamed this to my calibration, I would try again with new scene measurement calibration in coming days, I would update to you as soon as I finish the work.
Again, thank you for your help!
Hi, these days I ran some tests and am facing new challenges:(((
the Pose2Sim process is completed however the resulting opensim model animation is twitching and glitching here and there,
Regarding to this I have following concerns:
When open a model in opensim, is it the best to open the model with respective pose estimation model?( In my case is COCO_133)
The next problem is when loading the .trc file from Pose2Sim, opensim won't let me scale the model with it as shown in the image(No matter was it marker agumented/filtered) Yet if I change the model to body25_scaled or LSTM , it is runnable, but after the ik loaded and run the joint angle is all wrong and the skeleton is twitching. Screenshot 2024-07-23 214159
Sometimes opensim would only run 50frames and that's it, not a single frame more...
I also tried pose estimating with HALPE_26 along with HALPE_26 model in Opensim_Setup, it is 'runanble' but glitchy and twiching, also (refering to point 3) 50 frames.
I am kinda at loss of the direction to look for where the problem is. I recalibrated with scene measurements, and the error rate was around 10mm ---(note: when typing to this point, i realize when calibrating extrinsic parameter I chose 8 reference points which is fewer than recommended 10 points, I would recalibrate tomorrow to see if anything improved)
Besides point 5, is there any advice on where i should look into:)
Triangulation Info : --> Mean reprojection error for all points on all frames is 8.9 px, which roughly corresponds to 13.5 mm. Cameras were excluded if likelihood was below 0.3 and if the reprojection error was above 15 px. Gaps were interpolated with cubic method if smaller than 10 frames. Larger gaps were filled with the last valid value. In average, 0.69 cameras had to be excluded to reach these thresholds. Camera logi_2 was excluded 55% of the time, Camera logi_1: 8%, and Camera logi_3: 6%.
note: I delete the additional osmo camera since its inclusion would result logi_1 and logi_2 to be excluded completely, I may need to readjust its position, I wonder that could be that the osmo's ultrawide view has already make poseEstimation have many errors, which negatively affects the performance of Triangulation .
Thank you for your help along the way! :))))
Hi, I'm not exactly sure what the problem is.
After checking your triangulation info, it does not look bad but you should still try to visualize them. Logi_2 is excluded more than half of the time though, you may want to check its calibration. If you had to delete the additional Osmo, it may be that its calibration was not great. Can you see large distortions of straight lines? I started implementing distortion correction but did not find time to complete it. If you really need it, you can use openCV to undistort the video, and then perform calibration and triangulation again (unless there is an option to film in "Linear" mode like on GoPros).
Hey there,
After the recalibrated, (also adjusted the camera position to 30 degree angels), the twiching is vastly imporved, I would not say it is perfect but neither was my calibration, so hooray? hooray!
coco133 opensim model still cannot load the trc file however LSTM model works, I would try other models(like HALPE_26) later to see if that is some compatibility issue from coco133 maybe?
About the 50 frames, there is a time range setting in opensim which i completely ignored before, it automatically capped the frame. Once changed the issue is gone.
The triangulation info has also improved: Mean reprojection error for all points on all frames is 8.1 px, which roughly corresponds to 15.3 mm. Cameras were excluded if likelihood was below 0.3 and if the reprojection error was above 15 px. Gaps were interpolated with cubic method if smaller than 10 frames. Larger gaps were filled with the last valid value. In average, 0.24 cameras had to be excluded to reach these thresholds. Camera logi_2 was excluded 12% of the time, Camera logi_4: 9%, Camera logi_1: 2%, and Camera logi_3: 1%.
It is a real surprise after trying to figure out those issues during these days, it cannot done without your help so here's my gratitude :))
I'm glad it works now! I'll have to have a look if there is an issue with the Coco_133 model.
Can you tell me what failed with Body_133? I cannot reproduce the issue.
Hi,
It seems there are markers not found shown in the screenshots between .trc files and both ik and scale setups.(Red areas in the screenshots).
Once I unclicked those problem markers or change scaling factor manually, it is runnable then.
Okay, thanks! I'll release a new version soon, but I think the error should be fixed if you change lines 375 and 376 in skeletons.py:
Node("Reye", id=2),
Node("Leye", id=1),
Can you tell me if it works?
Yep that worked. But noted it is REye with the capital E or it wont. thank you for your help:)
Thank you for the feedback!
Hi there! Currently I'm at V1.95, and during running steps after poseEstimation,(i.e. synchronization(), personAssociation(), triangulation(), markerAugmentation() ) I got no other than Error: List index out of range.
At first I thought it was due to high calibration error, (log showing Residual (RMS) calibration errors for each camera are respectively [2.219, 1.948, 2.032, 2.019] px, which corresponds to [12.057, 11.336, 10.647, 10.072] mm.) However, I recalculated by using easymocap this time, with error rate around 0.2 px, the error persist.
I also tried :
They all failed unfortunately while list index out range shouting at me:(
Looking forward to your reply! Thx for reading the issue:)