search

facebookresearch / ocean

Ocean is the in-house framework for Computer Vision (CV) and Augmented Reality (AR) applications at Meta. It is platform independent and is mainly implemented in C/C++.
https://facebookresearch.github.io/ocean/
MIT License
633 stars 58 forks source link

How to align the coordinate systems of the external camera and the Quest's camera? #14

Closed AplusX closed 3 months ago

AplusX commented 3 months ago

Thanks for providing the Ocean framework!

Since the Quest does not allow users to access built-in camera data, how can we align the coordinate systems of the external camera and Quest's camera?

I didn't find any relevant information in the development documentation (https://facebookresearch.github.io/ocean/docs/introduction/)

atom2-source commented 3 months ago

There isn't a official calibration yet but seeing passthrough and the view from the external camera you can align them really close by eye there's alot of cv tools for rotation etc it's something you should only have to do once if the camera is secured and maybe check it every update.

AplusX commented 3 months ago

There isn't a official calibration yet but seeing passthrough and the view from the external camera you can align them really close by eye there's alot of cv tools for rotation etc it's something you should only have to do once if the camera is secured and maybe check it every update.

That's true. But it is almost impossible to align them accurately just by the user's eyes. Note that to enable Quest passthrough mode, the extra camera needs to support ARM64 architecture. I am trying to align Realsense with Quest.

atom2-source commented 3 months ago

I'll be doing it weekend with my quest pro I figure close enough is good enough a few buttons for the cv image manipulation so you can control it while wearing would be my start and look at a target of some kind.it won't be super accurate but I figure if I can't see a difference a user of the application won't image segmentation might be a bit of a ask with eye alignment only a display that showed how much you have altered the image from the webcam would be another good place to start so you can do eye alignment write it down then you have a really good starting point for image correction.

enpe commented 3 months ago

This is a great suggestion. And, yes, it's already on our radar. Please bare in mind that the support for external USB cameras is still very new. So, not everything will be immediately available.

AplusX commented 3 months ago

This is a great suggestion. And, yes, it's already on our radar. Please bare in mind that the support for external USB cameras is still very new. So, not everything will be immediately available.

May I know if Meta has a plan to release the access permission of Quest camera?