luxonis / depthai

DepthAI Python API utilities, examples, and tutorials.
https://docs.luxonis.com
MIT License
938 stars 232 forks source link

[Feature-Request] Multi-camera stereo system for large baseline setup #766

Open LukeKcy opened 2 years ago

LukeKcy commented 2 years ago

Start with the why:

This feature would be useful as it would allow for running multiple cameras which would assist on giving better accuracy at further distances.

Move to the what:

A script that can be used to run multiple cameras for accurate object detection/tracking in 3D spaces at large distances (wide baseline between the cameras) and match the object being detected across the devices

Move to the how:

Possibly using the RGB camera from the Oak D devices and using them find the distance of objects, user FPSychotic on the discord suggested that a rolling color sensor wouldn't help but instead suggested using the stereo cameras on the devices.

The devices would also need to be calibrated together while looking at the same screen to get the extrinsics of the cameras in relation to each other

Erol444 commented 2 years ago

Multi camera calibration (of extrinsics) would be useful for this feature as well: https://github.com/luxonis/depthai/issues/747 And we could potentially use dynamic calibration (example here) for the mutli-OAK-cam stereo perception. Thoughts? cc @szabi-luxonis

SzabolcsGergely commented 2 years ago

Yeah, that could be used @Erol444