Is GNM works for any camera?

[james-yoo]

There are many types of cameras in the robotics domain. In case of different camera is used in the training and deployment, is GNM work well? Is there any reason to recommend a wide-angle RGB camera in deployment?

[robodhruv]

Hi @james-yoo , The GNM training data certainly has a diverse variety of cameras used for data collection (and across different datasets, we did not enforce common cameras), but the data sources unanimously used cameras with a large FoV. The main motivation behind using a wide-angle camera is because the large FoV helps localization and improve distance estimates. This is generally true for all situations, and a larger FoV almost certainly improves collision avoidance as well as mitigate partial observability concerns (as opposed to something with a narrow FoV like a RealSense).

We did all our experiments with commercial fisheye cameras with two properties -- large FoV and low latency. You can also try narrower FoV and it might work, but possibly a bit worse. If you want to use a spherical camera, this might be a bit difficult since they are a lot more OOD with respect to the training data.

If you have a strong case to use other types of camera (e.g., RealSense or a spherical camera), please check out our concurrent research on using geometric priors to enable generalization to diverse camera parameters.

[james-yoo]

@PrieureDeSion, Great. I will check ExAug. Thank you!