search

jr-robotics / robo-gym-robot-servers

Repository containing Robot Servers ROS packages
https://sites.google.com/view/robo-gym
MIT License
29 stars 22 forks source link

Questions for creating new costumed robot server #7

Closed coral26 closed 2 years ago

coral26 commented 3 years ago

Under 'example_robot_server' in the ros_bridge.py file:

  1. I don't see 'control_period' (like what you have in the UR robot server). Is there no need for this variable?
  2. Is it true that under 'get_state' I define the observations and this is where I need to define the images from the robot camera as observations?
matteolucchi commented 3 years ago

I don't see 'control_period' (like what you have in the UR robot server). Is there no need for this variable?

https://github.com/jr-robotics/robo-gym-robot-servers/blob/bd00e4662a1af59384c8d5049450f9a0179711e7/example_robot_server/src/example_robot_server/ros_bridge.py#L92-L93

The sleep time between one command and the next one is hardcoded. In case you wish to be able to change that then you should use a variable like it is done for the other robot servers.

Is it true that under 'get_state' I define the observations and this is where I need to define the images from the robot camera as observations?

Yes, in get_state() is where you collect all the information that you want to send over to the Environment and pack it in a grpc message.

images from the robot camera

Beware that currently there is no support for images sent as a grpc message, so you might have to look into how to 'pack' an image into a grpc message. gRPC uses proto3, this might be helpful.

Cheers, Matteo

TheGreatLudini commented 3 years ago

Beware that currently there is no support for images sent as a grpc message, so you might have to look into how to 'pack' an image into a grpc message.

I was able to adapt the gRPC state message in order to transmit images, and I thought it would be helpful to share: If you are using NumPy and have your image as a numpy array, you can convert it to raw bytes using the np.tobytes() method. Then you add a 'bytes' field to your state message, see here. In your gym environment, you can convert these raw bytes back to a numpy array using the np.frombuffer() method. Furthermore, if you are sending your camera image via ROS by a sensor_msgs/Image message, you can convert it to an OpenCV image with the cv_bridge package in your ROS bridge. Then, you can convert it to a numpy array. I hope this helps anyone who tries to send images via gRPC.

matteolucchi commented 3 years ago

@TheGreatLudini thank you very much for this contribution! It's really great to see other people expanding the framework! Thanks for sharing this with us!

coral26 commented 3 years ago

Thank you very much @matteolucchi and @TheGreatLudini for your answers. @TheGreatLudini , your guidance on how to adjust the gRPC message to transit images from ROS is very helpful!