search

Field-Robotics-Japan / UnitySensors

ROS/ROS2 enabled Sensor models (Assets) on Unity
Apache License 2.0
208 stars 32 forks source link

How to use the package? #159

Closed sawan-kcl closed 4 months ago

sawan-kcl commented 4 months ago

Hi, I used this package about 3 years ago and got great success with it. Now I'm trying to use it again but seems a lot has changed. Can you tell me how to get it working? After installing package from git URL, I don't get any option to 'import' it in my project and hence I'm not able to use prefabs. I tried downloading the package as a zip and opening scene in unity but that ran into same error as mentioned in #157

please could you help me out?

Thanks.

Autumn60 commented 4 months ago

@sawan-kcl Thank you for coming back to UnitySensors again! Could you tell me what version (or URL) you used for the installation?

sawan-kcl commented 4 months ago

Hi, Thanks for your response.

https://github.com/Field-Robotics-Japan/UnitySensors is what i used which i believe is version 2.0.4. Also, I'm using ROS2 and the error #157 pops up as soon as i try to change my protocol to ROS2 in ROS settings window.

Autumn60 commented 4 months ago

@sawan-kcl

Sorry for the late reply.

Changes in PR #158 that fixes #157 were not merged to version 2.0.4. To use version 2.0.4, the same fix as in #157 must be made. Alternatively, you can use feature/commonPublisher branch, which is the version under development.

I will later release version 2.0.5 which fixes those issues.

sawan-kcl commented 4 months ago

@Autumn60 Thank you for your response. I did try using feature/commonPublisher branch but I couldn't get any data in RViZ. At this point I'm completely lost if I can even use this package. Given the circumstances, please could you tell me what would be the best way for me to use this package?

Autumn60 commented 4 months ago

@sawan-kcl I updated master branch for fixing the issue. Could you try master branch?

The URL used for installation is as follows.

sawan-kcl commented 4 months ago

@Autumn60 I tested it. The error msg in #157 does not pop up but the other issues remain. I'll list them down below:

  1. No option to 'import' the package after installing, thus the prefabs don't show up in assets folder. when i try a search in 'All Prefabs' option under 'Favorites', I can see them and that's how I've been using them so far.

  2. When trying to display images in Rviz2 (both from RGB and RGBD camera), RviZ2 crashes with following error: terminate called after throwing an instance of 'image_transport::TransportLoadException' what(): Unable to load plugin for transport 'image_transport/compressed_sub', error string: According to the loaded plugin descriptions the class image_transport/compressed_sub with base class type image_transport::SubscriberPlugin does not exist. Declared types are image_transport/raw_sub Aborted (core dumped)

  3. When trying to display pointcloud2 data from RGBD, RViZ2 doesn't crash but i can't see any cloud ( I checked the frame_id ).

  4. apparently the IMU works.

  5. GPS throws following error in Unity: NullReferenceException: Object reference not set to an instance of an object UnitySensors.Sensor.GNSS.GNSSSensor.UpdateSensor () (at ./Library/PackageCache/com.frj.unity-sensors@dc59e69b7f/Runtime/Scripts/Sensors/GNSS/GNSSSensor.cs:29) UnitySensors.Sensor.UnitySensor.Update () (at ./Library/PackageCache/com.frj.unity-sensors@dc59e69b7f/Runtime/Scripts/Sensors/UnitySensor.cs:36)

  6. When i used the package back in 2021 (with ROS 1), from what i remember, I had to build and modify some Velodyne ROS packages to get the LiDAR data. I don't know if that's still the case. Any documentation on this would be very helpful. Then i can test out LiDAR pointclouds as well.

Also, please let me know if there are any documentations available to get the other sensors working as well.

sawan-kcl commented 4 months ago

Any updates on this?

Autumn60 commented 4 months ago

@sawan-kcl Thank you for the detailed report on each element.

I am not good at English, so I will answer in bullet points.

  1. This is expected behaviour. Prefabs included in the package do not appear in the Asset list.
  2. The name of the topic may be suffixed with ‘/compressed’ to ensure that the image is displayed correctly.
  3. under investigation
  4. Thank you for testing!
  5. under investigation
  6. Currently, the data is output as PointCloud2, so there is no need to prepare or modify the ROS package.
Autumn60 commented 4 months ago
  1. GNSSSensor requires an object _coordinateSystem to be set up to define the coordinate system in order to be used. For specific use cases, check the Demo_GNSS_ros scene.

https://github.com/Field-Robotics-Japan/UnitySensors/blob/b1a50503bb38017f902cb232b489de2de334d62b/Assets/UnitySensors/Runtime/Scripts/Sensors/GNSS/GNSSSensor.cs#L12-L13

sawan-kcl commented 4 months ago

@Autumn60 Thank you for your response. Please also investigate number 2. It's not a topic naming convention problem. I checked. I'm using ROS humble with Ubuntu 22.04 and Unity Editor version 2022.3.30f1.

Autumn60 commented 4 months ago
  1. There could be multiple causes. Can you confirm that the topic is flowing successfully with rostopic echo /pointcloud (or ros2 topic echo /pointcloud) ?
sawan-kcl commented 4 months ago
  1. yes i can see data with ros2 topic echo /camera/depth/points but getting the buffer overflow while trying to display it in rviz2.

[rviz]: Message Filter dropping message: frame 'camera_frame' at time 34.786 for reason 'discarding message because the queue is full'

sawan-kcl commented 4 months ago
  1. GNSSSensor requires an object _coordinateSystem to be set up to define the coordinate system in order to be used. For specific use cases, check the Demo_GNSS_ros scene.

https://github.com/Field-Robotics-Japan/UnitySensors/blob/b1a50503bb38017f902cb232b489de2de334d62b/Assets/UnitySensors/Runtime/Scripts/Sensors/GNSS/GNSSSensor.cs#L12-L13

while trying to open the scene, it says 'not allowed to open a scene in read only package.'

Autumn60 commented 4 months ago

@Autumn60 Thank you for your response. Please also investigate number 2. It's not a topic naming convention problem. I checked. I'm using ROS humble with Ubuntu 22.04 and Unity Editor version 2022.3.30f1.

I was able to verify the behavior by modifying the topic name in the CameraInfo message.

image

Autumn60 commented 4 months ago
  1. GNSSSensor requires an object _coordinateSystem to be set up to define the coordinate system in order to be used. For specific use cases, check the Demo_GNSS_ros scene.

https://github.com/Field-Robotics-Japan/UnitySensors/blob/b1a50503bb38017f902cb232b489de2de334d62b/Assets/UnitySensors/Runtime/Scripts/Sensors/GNSS/GNSSSensor.cs#L12-L13

while trying to open the scene, it says 'not allowed to open a scene in read only package.'

I had forgotten that a Scene imported as a package cannot be opened. Can you recreate the scene using the following screenshots?

Set a GeoCoordinateSystem object for the CoordinateSystem in the GNSSSensor script component.

image

image

sawan-kcl commented 3 months ago

Hi, I just wanted to let you know that while trying to make my own sensor plugins, I was able to solve this problem. The main issue was the dependency on ROS2 package image-transport-plugins which can be installed with sudo apt install ros-${ROS_DISTRO}-image-transport-plugins and also the 'camera_info' topic for both RGB and Depth camera was mismatched which can be solved by the naming convention you suggested above.

I'm very happy that i got it working finally. I haven't tested the GPS yet but i'll let you know if i do.

Thank you.