search

mprib / caliscope

Multicamera Calibration + Pose Estimation --> Open Source Motion Capture
https://mprib.github.io/caliscope/
BSD 2-Clause "Simplified" License
172 stars 10 forks source link

How to Stereo Calibrate more than 2 cameras? #341

Closed aviralchharia closed 1 year ago

aviralchharia commented 1 year ago

Hi, I was trying 3D human pose estimation using $3$ cameras: i.e., first finding 2D keypoints from each of the $3$ camera frames and triangulating to get the 3D pose. In such a case, how to get the camera extrinsic parameters?

Specifically, suppose for $3$ cameras, should we use $[C1, C2]$, $[C2, C3]$ pairs or should we use $[C1, C2]$, $[C1, C3]$ to find the extrinsic matrix of the cameras?

How to use the GUI for the same?

mprib commented 1 year ago

Hi @aviralchharia. Just saw this sitting here in the issues. Sorry for the delay in noticing it. You bring up a big question. For more than two cameras you can no longer rely on opencv functions and must use a process called "bundle adjustment". This is an optimization that relies on SciPY. It's a can of worms to get into, but hopefully just having the word "bundle adjustment" can help get you started down the google/ChatGPT rabbit hole.

In a couple weeks I'll start rolling out a rough version of this project and when I have the README updated and an intro tutorial put together, I will send a notice your way.