search

lgsvl / simulator

A ROS/ROS2 Multi-robot Simulator for Autonomous Vehicles
Other
2.29k stars 781 forks source link

Get the raw data of the camera before game engine (ISP) processing #1910

Open yumianhuli2 opened 2 years ago

yumianhuli2 commented 2 years ago

Hello! How do I get raw camera data that is not processed by the game engine?Just like the raw data of the real camera before ISP processing. Can I get it directly from a script inside the simulator? Thank you！

EricBoiseLGSVL commented 2 years ago

All camera data is processed by the engine. Are you talking about preprocessed data? Before lighting effects or other passes?

yumianhuli2 commented 2 years ago

All camera data is processed by the engine. Are you talking about preprocessed data? Before lighting effects or other passes?

Yes, it's the raw data from the optical signal to the electrical signal(without White balance, Auto exposure...). Then the raw data is displayed on the screen after the ISP process/engine process. So from a real simulation point of view I'm going to get this raw data. Thank U！

EricBoiseLGSVL commented 2 years ago

There is no optical lens in the simulator. What are you trying to accomplish? The camera render pass is processed to match the camera settings. You can look to HDRP docs to add a hook before post processing is done or alter the code and remove all effects. We also have a simplified rendering mode in settings that removes all hdrp post processing effects.

yumianhuli2 commented 2 years ago

There is no optical lens in the simulator. What are you trying to accomplish? The camera render pass is processed to match the camera settings. You can look to HDRP docs to add a hook before post processing is done or alter the code and remove all effects. We also have a simplified rendering mode in settings that removes all hdrp post processing effects.

Getting the raw data is to compare the results we get through the ISP to see if they are satisfactory, not any post-processing in the game engine. This allows us to evaluate our ISP applications. So we want to get results that don't go through any of the game engine's piplines(such as built-in, URP, HDRP). This interface is provided in other autonomous driving or ADAS professional simulation software, such as VTD, PRESCAN, SCANeR.This is significant.

So how do I make sure I get the raw data I want without going through any of Unity's rendering pipelines? Could you plz provide an exact document or URL or sample program？ Thank U very much！