search

Field-Robotics-Japan / UnitySensors

ROS/ROS2 enabled Sensor models (Assets) on Unity
Apache License 2.0
191 stars 28 forks source link

Support of RGB and Depth Camera Sensor for URP ? #67

Open panagelak opened 8 months ago

panagelak commented 8 months ago

Hello i really like this package and the sensors it provides i was looking for something like this a long time.

Since URP will become the default render pipeline, will you find time to support in the future? ( even HDRP )

I think they only sensors that needs updating is the RGB and Depth Camera sensor to support all the pipelines, i am not very good with shaders otherwise i would help.

Thank you

Autumn60 commented 7 months ago

I tried URP. https://github.com/Autumn60/UnitySensors/tree/feature/urp_test

The RGB camera could be executed with only minor changes. (https://github.com/Autumn60/UnitySensors/commit/2aee2addf4ceb7f094823a676f420ada6ae933ce)

RGBD camera and LiDAR are under investigation.

panagelak commented 7 months ago

@Autumn60 i can also confirm that with this change in the RGBCameraSensor.cs it works ( 1 instead of 0 in the RenderTexture line 27). The VLP-16 lidar also works.

The Depth camera and pointcloud do not work since the Color2Depth shader is not updated.

Furthermore the depth image is compressed using jpeg format and i can decompress it like a normal image ( bgr8 encoding ) but the more correct way of doing that would be for the depth image to use 32FC1 format. Maybe this format is more difficult to compress/decompress though, and an option to publish the uncompressed depth image directly with the 32FC1 format should be added.

Using the 32FC1 format together with a camera info message ( generated by the robotics hub CameraInfoGenerator ), can be used to reconstruct the point cloud on the ros side ( with color or not ) as another option and for use cases where the depth image is only required ros expects this format

RyodoTanaka commented 6 months ago

@Autumn60 @panagelak Thank you for your discussion. I also trying to use this repository under URP environment. As your discussion, I also make it work. To make it work, I needed to set up not only source codes but several settings. Therefore, I am thinking should we create new repository such as "UnitySensorsURP".

I would like to ask your opinion.

Autumn60 commented 6 months ago

I agree with @RyodoTanaka 's suggestion. Separating repositories seems like a good idea.
However, it seems difficult to share updates about other sensors.

If Unity Technologies makes URP the standard, it might be a good idea to leave the current branches as an archive and change the UnitySensors project to support URP.