search

robotology / icub-models

Official URDF and SDF models of the iCub humanoid robot.
Creative Commons Attribution Share Alike 4.0 International
32 stars 33 forks source link

iCubGazeboV3/iCubGenova09: issues with realsense camera #69

Closed Nicogene closed 3 years ago

Nicogene commented 3 years ago

In the latest delivered model (https://github.com/robotology/icub-models-generator/pull/172)

The depth camera seems not to be generated in the urdf.

The rgb camera is generated but it gives this error:

[DEBUG] GazeboYarpCamera Plugin: sensor scoped name is  default::iCub::chest::realsense_chest_rgb
[WARNING] |yarp.device.grabberDual| Period parameter not found, using default of 0.03 s
[WARNING] |yarp.device.grabberDual| 'capabilities' parameter not found or misspelled, the option available are COLOR(default) and RAW, using default
[INFO] |yarp.os.Port| Port /icubSim/depthCamera_rgb/camera:o/rpc active at tcp://172.19.239.69:10045/
[INFO] |yarp.os.Port| Port /icubSim/depthCamera_rgb/camera:o active at tcp://172.19.239.69:10046/
[INFO] |yarp.dev.PolyDriver| Created device <gazebo_camera>. See C++ class GazeboYarpCameraDriver for documentation.
[WARNING] |yarp.device.grabberDual| Targets has not IVisualParamInterface, some features cannot be available
[INFO] |yarp.dev.PolyDriver| Created wrapper <grabberDual>. See C++ class ServerGrabber for documentation.
[INFO] Loaded GazeboYarpCamera Plugin correctly
[ERROR] Unable to get the iFrameGrabberImage interface from the device
Nicogene commented 3 years ago

Today I investigated this problem

and I think our problem is very similar to this one https://answers.ros.org/question/232534/gazebo-camera-frame-is-inconsistent-with-rviz-opencv-convention/

I tried to remove all the rpy components(setting them to 0) and the camera looked correctly to the horizon.

My suspect is that gazebo doesn't treat the camera like a "special" sensor and expects that the z is upward etc The options are 2:

  1. Use the correct frame and make the gazebo camera look in the z direction(I don't know how)
  2. Fake the frame in order to make gazebo happy

I dislike the second option since the frame in the cad would be different to the one in the reality

@traversaro and/or @fiorisi I am missing something? I checked the cer-sim sdf and there are rotations, but I don't know how those frames were defined

cc @vvasco @xEnVrE

traversaro commented 3 years ago

Just to understand @Nicogene, on the base of which documentation of realsense we are currently placing the frame in the model?

Nicogene commented 3 years ago

With @salvi-mattia we extracted the frames from the intel datasheet.

traversaro commented 3 years ago

With @salvi-mattia we extracted the frames from the intel datasheet.

Which page? : )

Nicogene commented 3 years ago

I didn't find a clear reference, but I deduced it from pages 65-66 and from the fact that considers the z as depth

traversaro commented 3 years ago

I didn't find a clear reference, but I deduced it from pages 65-66 and from the fact that considers the z as depth

But this leaves the ambiguity on the X/Y direction, right? In any case, I saw that Intel provides official URDF (https://github.com/IntelRealSense/realsense-ros), STEP models ( https://dev.intelrealsense.com/docs/cad-files ) and API docs ( https://dev.intelrealsense.com/docs/projection-in-intel-realsense-sdk-20 ), so I think somewhere we can get this info. (BTW: which realsense model do we have mounted on iCub3? )

traversaro commented 3 years ago

I tried to remove all the rpy components(setting them to 0) and the camera looked correctly to the horizon.

That is quite strange. The Gazebo/SDF documentation (http://sdformat.org/spec?ver=1.5&elem=sensor#sensor_camera) is not complete on this, but as far as I understand I would say that the view direction of the camera is indeed the positive Z . If you set all the RPY angles to 0 0 0, then the camera should look in the Z directory of the chest, i.e. UP . Can you post the SDF model (or at least the elements related to the chest link and the realsense frame) that you obtain by converting the URDF model as it is, and the one that you manually fixed by setting RPY to 0 0 0, and in your experience is working correctly?

pattacini commented 3 years ago

But this leaves the ambiguity on the X/Y direction, right?

For an optical sensor the convention states that:

traversaro commented 3 years ago

But this leaves the ambiguity on the X/Y direction, right?

For an optical sensor the convention states that:

* z-axis goes along the optical axis (pointing outward)

* x-axis points rightward (wrt to the sensor's image plane)

* y-axis points downward

I totally agree (it is coherent also with REP-103: https://www.ros.org/reps/rep-0103.html#suffix-frames) but if this convention was used in choosing the frame it should be explicitly specified.

Nicogene commented 3 years ago

Chatting with @traversaro it emerged that probably there is a lack in the sdf specifications, it seems not clear where the optical axis of the gazebo camera is placed, from this http://gazebosim.org/tutorials?tut=camera_distortion&cat=sensors seems that the optical axis is along the the x-axis instead of the z-axis as we expected.

We should check if there are open issues for this, and in case not, we should open one.

In the meanwhile, I think that the only option is to do like @fiorisi did for cer-sim (see https://github.com/robotology/cer-sim/commit/86c709d48a8fcb0d2af4bdcb1bd306eb158ae9b3)

traversaro commented 3 years ago

In the meanwhile, I think that the only option is to do like @fiorisi did for cer-sim (see robotology/cer-sim@86c709d)

I think that this is indeed a good strategy, because it permits to keep the sensor pose to be the expected one, and always have the camera pose to be the place were we convert to the REP-103 optical convention and the Gazebo camera convention.

We should check if there are open issues for this, and in case not, we should open one.

Let me know if you prefer to open the issue or if you prefer that I open it myself.

cc @xEnVrE

traversaro commented 3 years ago

Chatting with @traversaro it emerged that probably there is a lack in the sdf specifications, it seems not clear where the optical axis of the gazebo camera is placed,

To clarify, this is the documentation we were referring to: http://sdformat.org/spec?ver=1.8&elem=sensor#sensor_camera

Nicogene commented 3 years ago

Let me know if you prefer to open the issue or if you prefer that I open it myself.

I can open it but I have just to know where to open it :D

BTW seems that now it works fine ! https://github.com/robotology/icub-models-generator/pull/180#issuecomment-736557556

traversaro commented 3 years ago

Let me know if you prefer to open the issue or if you prefer that I open it myself.

I can open it but I have just to know where to open it :D

The appropriate issue tracker is https://github.com/osrf/sdformat, that hosts both the SDF specification and the C++ reference implementation of the SDF parser.

Nicogene commented 3 years ago

The issue on the sdformat repo has been opened(https://github.com/osrf/sdformat/issues/425), I think as soon as the PR is merged we can close this issue