search

JARVIS-MoCap / JARVIS-AnnotationTool

AnnotationTool to create multi-view annotations for the JARVIS 3D Markerless Pose Estimation Toolbox
https://jarvis-mocap.github.io/jarvis-docs/
GNU Lesser General Public License v2.1
24 stars 6 forks source link

Charuco Calibration Not Working #16

Open KenR22 opened 1 year ago

KenR22 commented 1 year ago

Hi,

We have shifted from Anipose to Jarvis, but we have calibration in Anipose format for our previous recording. I could not get the CHaruco to get working on Jarvis. I have attached our calibration video and charuco board parameter(under the toml file) here. I was wondering if you could guide me to the problem.

https://drive.google.com/file/d/1ZKJEH7MikZAsPpsFvqcfZMbWz0jbG9RG/view?usp=sharing

Let me know if the link does not work.

Thanks, Ken

JARVIS-MoCap commented 1 year ago

Hi,

Thank you for sharing the example! The link requires you to give me permission to access the files, I send a request. I'll have a look at the data as soon as I have access.

Timo

KenR22 commented 1 year ago

You should have the access now.

Thanks, Ken

timohueser commented 1 year ago

Sorry it took me so long to get back to the issue! I just tried running the calibration on the videos you provided but could not figure out what version of the ChArUco markerset you are using. Could you please tell me what pattern this is/how you generated it?

Thanks and best Timo

KenR22 commented 1 year ago

Hi,

This board was generated using anipose. https://github.com/lambdaloop/anipose/blob/dbebebba1e438f563f373245c2e546ece118fc65/anipose/anipose.py#L284 I think they used OpenCV here^.

The parameters we used can be found in the Calibration_TIMO/config.toml file

Let me know if you need anything else.

Thanks again! Ken

KenR22 commented 1 year ago

Hi,

Also I would like to mention that anipose parameters are in the folder but it does not match with jarvis format. If you could guide how to convert them, that should also work for us. Thanks

JARVIS-MoCap commented 1 year ago

Hey, sorry it took me a while to figure this one out! It seems like OpenCV changed the way they arrange the ChArUco markers somewhere between version 4.5 and version 4.7. There are also some changes to the Python commands in the newer version (e.g CharucoBoard_create was renamed to just CharucoBoard), so I assume there are some changes under the hood of the OpenCV aruco module that cause this. Unfortunately that means that you can not use the calibration recordings you have with the JARVIS, since it uses a newer OpenCV version.

I attached a Anipose to Jarvis conversion script that I used some time age. It does NOT work on your calibration files directly though, since your format seems to be different to what I was using. Also note that JARVIS does not allow fisheye calibration, so it might be a bit tricky to correctly convert the distortion values.

Sorry I can't give you a more satisfying answer. Please let me know if you have any other questions and I'm happy to help out with getting the conversion working!

Timo

KenR22 commented 1 year ago

Thanks for your time on this! I was wondering if you would be willing to give a pointer to make my own calibration for Jarvis preferably on python. I have got to the point I can do charuco detection on previous opencv but struggling with the bundle adjustment part.

karihoffman commented 1 year ago

Hey Timo,

Thanks so much for your response to this! We're in a bind trying to finish the conversion and Ken could use help finding the specific next step (bundle adjustment). These are invaluable data for us, which is why he's putting in the effort needed to use the old charuco (we can't recollect these neural learning data).

Anything you can do to point him in the right direction would be appreciated!