search

stereolabs / zed-ros-wrapper

ROS wrapper for the ZED SDK
https://www.stereolabs.com/docs/ros/
MIT License
447 stars 391 forks source link

Simulation input from gazebo. #190

Closed radnvlad closed 4 years ago

radnvlad commented 6 years ago

So, I would like to simulate input coming from Gazebo into the zed-ros-wrapper. Is this possible?

According to http://wiki.ros.org/zed-ros-wrapper , it should be possible to play back recordings so it is possible inputting "virtual data", but can I somhow generate that data realtime from gazebo?

Best, Vlad R.

aarontan-git commented 6 years ago

Did you figure this out? @RealityVoid

radnvlad commented 6 years ago

Well, yes and no. Zed-ros-wrapper uses the zed sdk to interface with the HW - and I did not manage to make that work. Instead I just used a gazebo plugin to get the point cloud out of the simulation. It's not the same thing and I'm not using a zed-ros-wrapper so I had to do some extra processing on the point cloud (things that would come right off the bat from zed-ros-wrapper, such as odometry) but it worked for my application.

I'm pretty sure you can somehow write a gazebo plugin to emulate the zed sdk somehow, but i'm not sure if you have enough information to easily make this and not sure it's worth it.

renanmb commented 6 years ago

it is possible, you just need to add the plugin to .sdf , set up the .launch, and configure the urdf and meshes. I am running into the same trouble, the MIT racecar project ( https://github.com/mit-racecar/racecar-simulator ) have the zed set up but there is no documentation. I am looking for finding someone that setup the plugin, since the plugin simulates the sensors outputs to work it need to be as similar as possible to the device being simulated. This is the big problem.

Looking forward.

radnvlad commented 6 years ago

@renanmb is there a plugin for the zed camera? If not, I think one specifically for the ZED camera is needed in order to work with the ZED SDK. Otherwise, you need to modify the wrapper to take some other source other than the ZED SDK as input. But of course, loosing the ZED SDK means some stuff that is happening there is going to behave a different way - I'm most interested in the odometry part.

At least that is my understanding of it, I might be wrong though.

Myzhar commented 6 years ago

We are working on this... not sure that Gazebo can be interfaced with ZED SDK to simulate stereo acquisition.

radnvlad commented 6 years ago

Well, we can surely simulate an Kinect from Gazebo. So it is possible, but from what I can tell, not with the current ZED SDK

@Myzhar - do you have your work somewhere on an open branch?

jim-rothrock commented 5 years ago

After examining the zed-ros-wrapper source code, it looks like the easiest way to add Gazebo support would be to use the ZED SDK's streaming system. The ZED ROS node would receive the stream over IP, as is currently supported. The Gazebo plug-in would simulate two cameras and an IMU, which should be straightforward. The missing piece would be an addition to the ZED SDK. Here is pseudocode for the addition:

// Open a stream and return its file descriptor. int sl::openStream(string ipAddr, int port)

// Close the stream. void sl::closeStream(int fd)

// Write the left image, the right image, and the IMU data, simulated in a Gazebo plug-in, // to the stream. void sl::writeZedMData(int fd, const sl::Mat& leftImg, const sl::Mat& rightImg, const sl::IMUData& imuData, Time timestamp)

The Gazebo plug-in would be responsible for converting its image and IMU data into the ZED SDK format used by writeZedMData().

Myzhar commented 5 years ago

Hi @jim-rothrock thank you for your suggestion. We will take it in account for future developments. Today the ZED SDK cannot accept any kind of image couples as input source. It is designed and tuned to work explicitly with images generated by the two sensors placed on ZED cameras. Using external images, mostly if generated by an "artificial simulator", cannot guarantee that the resulting depth processing is the same as the "real" one. The results of a simulation must be the most similar as possible to the result you expect in the real world, this is the main motivation why we are not yet providing this "simple" feature.

deep-city-robotics commented 3 years ago

Hi guys, has there been a solution to this?

Myzhar commented 3 years ago

Hi @deep-city-robotics, unluckily nothing changed. The ZED SDK is closed and it cannot accept inputs from other sources than real cameras, SVOs, or network streams generated by the ZED SDK itself.

mstampa commented 3 years ago

This is quite unfortunate. Are there any plans to offer Gazebo support in the future? I'd certainly appreciate that.

For my purposes, a dedicated Gazebo plugin for the ZED SDK doesn't have to be a perfect match to reality. Presumably it will still be more accurate than any model based on stock depth camera plugins I'll have to set up and use in the meantime.

hichamhendy commented 1 year ago

Is there any news on this??

Myzhar commented 1 year ago

We are working on the Gazebo plugin, but ROS integration will be initially only available for ROS2. ROS1 is near EOL so new plugins/features will only be released for the new platform.

santoshbalaji commented 1 year ago

hi is there ros2 gazebo plugin for accessing zed sdk available at the moment ? I am using zed2i camera with body tracking features in real time. So looking for solution to have this feature in simulation as well

Myzhar commented 1 year ago

No, the work on the Gazebo plugin has been temporarily postponed because the obtained results were not satisfying in terms of quality.

nrkumar93 commented 1 year ago

This link seems like it simulates Zed2 on Gazebo... I did not test yet. https://www.leorover.tech/integrations/erc-gazebo

ahmetsaglam commented 1 year ago

This link seems like it simulates Zed2 on Gazebo... I did not test yet. https://www.leorover.tech/integrations/erc-gazebo

They are using an existing plugin for depth (and point cloud), which is libgazebo_ros_openni_kinect and libgazebo_ros_camera for the rest of camera stuff. Depending on what you need, they may be enough to simulate the ZED.

altineller commented 3 months ago

This link seems like it simulates Zed2 on Gazebo... I did not test yet. https://www.leorover.tech/integrations/erc-gazebo

They are using an existing plugin for depth (and point cloud), which is libgazebo_ros_openni_kinect and libgazebo_ros_camera for the rest of camera stuff. Depending on what you need, they may be enough to simulate the ZED.

The leorover seems to be ros1 only.

I just need a generic 3d point cloud generator for gazebo simulator, using ros2. It can even get its position from gazebo world, and publish it, basically as long as it generates the same type of data as zed2i, maybe plus some gaussian noise, I am fine with it, no matter how the data is generated.

Is there a previosly make method for this?