search

microsoft / AirSim

Open source simulator for autonomous vehicles built on Unreal Engine / Unity, from Microsoft AI & Research
https://microsoft.github.io/AirSim/
Other
16.47k stars 4.59k forks source link

Modelling fisheye cameras #3012

Open tharindurmt opened 4 years ago

tharindurmt commented 4 years ago

I am trying to record camera images from four cameras which have a very high field of view of 194 degrees, which AirSim does not support as of yet (if the FoV is more than 180 degrees the images will be flipped vertically. Even if the FoV is set to be just below 180 degree there is no fisheye effect).

AirSim cameras operate a little different to other sensors and as per my understanding, it uses SceneCaptureComponent2D of UE to generate the images using PIPCamera.

So, I was wondering what would be the most suitable way to implement fisheye model. One option would be to change the camera projection model of UE. Is there any other way to achieve this in AirSim side (other than the trivial solution of stitching the output of multiple cameras. This is not an option for me as I need four fisheye cameras with 194 FoV, which means that I will have to create at least 20 cameras with 90 FoV).

Related issues: #2129, #2379

tharindurmt commented 4 years ago

@jonyMarino @rajat2004 Is it possible use PIPCamera with UE LensDistortion plugin? It provides a camera matrix and its parameters can be changed accordingly. I tried adding it to the PIPCamera blueprint as in the following figure (I am completely new to UE). Material and a render target used in this are created following this Global Shader as a Plugin UE tutorial (steps 1 to 3 of Blueprint Setup section). I was under the impression that this transformation will affect any content passed through PIPCamera blueprint. However, this seems to have no effect on captured images. I can't figure out what's wrong with my approach. Could you please help me out?

Lensdist

petergerten commented 4 years ago

@tharindurmt Did you get further with this? I try to get a 220deg fisheye camera to work.

tharindurmt commented 4 years ago

@petergerten Unfortunately, no, I was not able to get this to work. I have a very limited knowledge of UE and I wasn't even sure what i was doing. But from my research, the best bet would be to use post-processing materials. I couldn't figure out a way to integrate the post-processing material to AirSim.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had activity from the community in the last year. It will be closed if no further activity occurs within 20 days.