search

facebookresearch / habitat-sim

A flexible, high-performance 3D simulator for Embodied AI research.
https://aihabitat.org/
MIT License
2.64k stars 424 forks source link

Define Fisheye camera model with distortion #1820

Open raoqiang opened 2 years ago

raoqiang commented 2 years ago

🚀 Feature

api for add custum camera model with distortion like fisheye camera；to simulate the fisheye camera and get the sementic segmenation and depth

Motivation

simulation for large fov camera and simulation the fisheye camera which widely used in industrial instruments

dhruvbatra commented 2 years ago

@Skylion007

Skylion007 commented 2 years ago

@raoqiang we actually do support fisheye cameras and have for a while. The python bindings are merged into this PR and you see how to construct the corresponding camera sensor there: https://github.com/facebookresearch/habitat-sim/pull/1168

raoqiang commented 2 years ago

@Skylion007 thanks for your reply, I have tried the double sphere fisheye camera model to generate fisheye corlor image and depth image, but I found that the depth is not correct and I tested the depth to recontructed the depth to cloud, the center part is correct but the edge is wrong image

image

the fragment is the src/shader/doubleSphereCamera.frag the double sphere parameter I used is this "cam.intrinsics": [ { "fx": 312.99588161843817, "fy": 312.9214941782706, "cx": 638.7854720110728, "cy": 514.5354880999439, "xi1": -0.17918626779269324, "xi2": 0.590840604914911 } ],

there is an obviously mistake on the depth image image

how can I get the correct depth from fisheye camera model scerely wish for your reply !

aclegg3 commented 2 years ago

@Skylion007 any thoughts on this? Is this a bug?

ZivKidd commented 2 years ago

met the same problem in equirectangular depth image, it seems like a cube shaped depth map is added to the depthmap, maybe the cubemap to fisheye and equirectangular transform is wrong image image