Question on using your own data (from opencap to opensim).

[djinayon]

Hi again @davidpagnon, I'm currently following through the 'Use your own data' part of the repository and I'm quite confused on the workflow and unsure if I am doing it right. I have some questions I'd like to ask:

• First, if I understood it correctly, in the screenshot below, the copied folder must be in the \pose2sim folder? What happens if I don't place it in the pose2sim folder?

• Second, I'm kinda confused on the folder structure in general. When starting from scratch, say having only two videos of a single person walking and let's say I copied the single person demo as the case for simplicity. I just have to follow the folder structure right? Is there a need to delete some folders/generated files from the copied folder? If yes, which folders? Also where should I place the videos and what format should they be in?

• Third, when trying the calibration part where for opencap, I copied the pickle file in the calibration folder of the copied folder and tweaked the Config.toml accordingly and I get this error. Is this a known issue?

• Lastly, if I could trouble you for one more thing, is there perhaps a youtube video/channel I could watch that might help me gain a bit more understanding of the workflow (or how to create one) when using pose2sim?

Thank you for taking your time to read this and I really appreciate the hard work you've put in here.

[davidpagnon]

Hi,

• no you can copy the folder to wherever you like, rename it, etc. You don't even have to copy it as long as you follow the right structure
• The videos need to be put in the videos folder, and can be in any format: you just have to specify it in the Config.toml file.
• it is telling you that it is looking for data (image size) in the pickle file and that it cannot find it. Most probably, you did not put the right calibration file in. Is there any other pickle calibration file in your OpenCap project? The other option would be for you to do the calibration from scratch
• unfortunately there is not video yet, although it is something I'd like to do. But currently there are edits to the code and new options almost every month so it's not quite worth it yet...

[djinayon]

Hi, thank you for your response! I'm pretty sure I copied the right calibration file but I'll check. If it still shows the error, guess I'll start from scratch. Also, looking forward to the video once everything's in place. Good luck and thanks a bunch for the work you do!

[davidpagnon]

If it does not work, do you mind sending me your OpenCap calibration file so that I can have a look?

[djinayon]

Hi again @davidpagnon, I tried the suggestions you mentioned and got it to work, I copied the wrong pickle file (haha). I managed to do the calibration and pose estimations. However when I try to synchronize, I encountered this problem. I've tried solutions mentioned in https://stackoverflow.com/questions/41994485/how-to-fix-could-not-find-or-load-the-qt-platform-plugin-windows-while-using-m, but nothing seems to work for me. Any idea what causes this or how to solve it? As the thread mentioned, it probably has something to do with the directory of the Qt, but I'm lost haha.

[djinayon]

Hi again @davidpagnon, I tried the suggestions you mentioned and got it to work, I copied the wrong pickle file (haha). I managed to do the calibration and pose estimations. However when I try to synchronize, I encountered this problem. I've tried solutions mentioned in https://stackoverflow.com/questions/41994485/how-to-fix-could-not-find-or-load-the-qt-platform-plugin-windows-while-using-m, but nothing seems to work for me. Any idea what causes this or how to solve it? As the thread mentioned, it probably has something to do with the directory of the Qt, but I'm lost haha.

I tried running the single person demo and oddly enough the problem in the synchronization showed (which shouldn't happen because I already succeeded in the demo). So I tried removing the environment altogether and started from the beginning (creating the environment and doing the single person demo). It seems to have solved the problem, I don't know how though.

Now, I tried running my trial (me walking filmed with two cameras from opencap) and the synch problem is now gone. But another error showed:

What does this error tell me?

[davidpagnon]

Hi, I'm glad you fixed the qt issue.

As the new error says, chances are that your issue is related to calibration or synchronization:

• Did you convert the calibration data from OpenCap? If not, what were the error values? did you take enough space to calibrate?
• Did you check the best practices for synchronization? N.B.: Works best when:
  • only one participant must be in the scene (set approx_time_maxspeed and time_range_around_maxspeed accordingly)
  • the participant is at a roughly equal distance from all cameras
  • the capture should be at least 5 seconds long

[djinayon]

Hi, thanks for responding.

• Yes I did change the config to convert from OpenCap. I'm not sure what you mean by taking enough space to calibrate though, I just used the calibration files from OpenCap directly. Here's the calibration result:
• I'm the only person present in my video and the cameras are roughly the same distance too and the vid is longer than 5 seconds. Maybe it's from the quality of my video? I'll try using another vid and see if it works out.

[davidpagnon]

It's hard to tell if there is anything wrong with the calibration file but let's assume it is good. What are the correlation values when you synchronize? Did you play with these parameters to get the highest possible correlation value?

keypoints_to_consider = ['RWrist'] # 'all' if all points should be considered, for example if the participant did not perform any particicular sharp movement. In this case, the capture needs to be 5-10 seconds long at least
                           # ['RWrist', 'RElbow'] list of keypoint names if you want to specify a list of keypoints with sharp vertical motion.
approx_time_maxspeed = 'auto' # 'auto' if you want to consider the whole capture (default, slower if long sequences)
                           # [10.0, 2.0, 8.0, 11.0] list of times (seconds) if you want to specify the approximate time of a clear vertical event for each camera
time_range_around_maxspeed = 2.0 # Search for best correlation in the range [approx_time_maxspeed - time_range_around_maxspeed, approx_time_maxspeed  + time_range_around_maxspeed]

[djinayon]

It's hard to tell if there is anything wrong with the calibration file but let's assume it is good. What are the correlation values when you synchronize? Did you play with these parameters to get the highest possible correlation value?

keypoints_to_consider = ['RWrist'] # 'all' if all points should be considered, for example if the participant did not perform any particicular sharp movement. In this case, the capture needs to be 5-10 seconds long at least
                           # ['RWrist', 'RElbow'] list of keypoint names if you want to specify a list of keypoints with sharp vertical motion.
approx_time_maxspeed = 'auto' # 'auto' if you want to consider the whole capture (default, slower if long sequences)
                           # [10.0, 2.0, 8.0, 11.0] list of times (seconds) if you want to specify the approximate time of a clear vertical event for each camera
time_range_around_maxspeed = 2.0 # Search for best correlation in the range [approx_time_maxspeed - time_range_around_maxspeed, approx_time_maxspeed  + time_range_around_maxspeed]

I haven't played around those yet, I used the same parameters as the single demo for now, but I'll try. Here's the synchronization result from using the same parameters as the single demo:

[davidpagnon]

Sorry I did not see your answer! The synchronization looks good... Did you figure it out?

[djinayon]

Hi, thank you for replying. I've been working on other stuff so I haven't tried other things here. My professor got it to work with a trial of his own, the error is probably from my opencap data. I will be consulting with him within this week. Thanks again!

[davidpagnon]

Okay! I'm closing it then, don't hesitate to reopen the issue if needed!