search

robotology / stereo-vision

Repository containing apps for stereo vision
https://robotology.github.io/stereo-vision/
GNU General Public License v2.0
19 stars 19 forks source link

SFM on icubSim #5

Closed martz21 closed 8 years ago

martz21 commented 8 years ago

Hello!

Is SFM working on icubSim? I tried the following steps but it doesn't seem to give the expected result. launch yarpserver and iCub_SIM launch iKinGazeCtrl --from configSim.ini, launch SFM --robot icubSim So far, everything seems ok, however if I try to open a yarpview and connect to SFM launch yarpview --name /viewer and then yarp connect /SFM/disp:o /viewer nothing appears (it's just black). I tried to lauch yarp rpc /SFM/rpc >>calibrate but nothing seems to happen. Am I doing something wrong or forgetting some steps? Thanks in advance for your help!

Best, Martina

GiuliaP commented 8 years ago

Hi Martina,

I think that the only things that you might have missed, as I read from your description, is connecting the input ports of SFM with the iCub's cameras and launching the SFM also with the parameter --from icubSimEyes.ini. If you contact me, I can send you the .xml files that I used in order to have the SFM module running in simulation (I do not know how to post them here). These are the equivalent of the iCub Startup, Calib Cameras and SFM applications that I use on the real robot. Here is a screenshot: of course the quality of the disparity is bad due to the fact that the images are not correctly rectified, and poor of key-points to be matched in the calibration.

sfm_simulation

Best, Giulia

pattacini commented 8 years ago

For starting up basic modules with the simulator I'd recommend to rely on this xml available from within the repo: https://github.com/robotology/icub-main/blob/master/app/iCubStartup/scripts/icubSimStartup.xml.template.

gabriellapizzuto commented 7 years ago

I followed the same procedure as Martina and added the following to the icubSimStartup.xml:

<module>
    <name>SFM</name>
    <parameters>--robot icubSim --from icubSimEyes.ini</parameters>
    <node>localhost</node>
</module>

<module>
    <name>yarpview</name>
     <parameters>--name /matchViewer</parameters>
     <node>localhost</node>
 </module>

 <module>
     <name>yarpview</name>
     <parameters>--name /dispViewer</parameters>
     <node>localhost</node>
 </module>

 <connection>
     <from>/SFM/match:o</from>
     <to>/matchViewer</to>
     <protocol>tcp</protocol>
  </connection>

  <connection>
      <from>/SFM/disp:o</from>
      <to>/dispViewer</to>
      <protocol>tcp</protocol>
  </connection>

However, I still get the black screen when I try to view /SFM/disp:o and /SFM/match:o. Am I missing something else?

GiuliaP commented 7 years ago

Hi @gabriellap , do you connect the iCub's cameras to the SFM's input ports (left and right images)? I am asking since in the .xml you pasted I don't see these two connections.

mingshi1214 commented 3 weeks ago

Hi, I'm trying to use stereo-vision on the icubSim on Gazebo. I followed all the steps above and connected iCub's cameras to the SFM's inport ports. However, when starting up SFM, it required me to have sfm.ini from calibration

No local calibration file found in /home/mingshi/.local/share/yarp/contexts/cameraCalibration/SFM.ini ... Using Kinematics and Running SFM once.

Since I am using this in simulation, I'm not sure how to calibrate without the physical robot. I've tried running the stereoCalib module with --robot flag as icubSim and --from flag as icubSimEyes.ini with no luck.

stereoCalib --robot icubSim --from icubSimEyes.ini

[ERROR] |yarp.device.remote_controlboard| Problem connecting to /icub/head/stateExt:o, is the remote device available?

Can someone help me with this or can I get a dummy SFM.ini file somewhere?

pattacini commented 2 weeks ago

Hi @mingshi1214

There's actually no need to calibrate the stereo-rig in simulation as it is an ideal model, after all, unless you'd really aim to simulate the effects of impairments.

To get to know how to use the depth sensor with the iCub in Gazebo, you could refer to https://github.com/robotology/icub-gazebo-grasping-sandbox where point clouds of objects lying on a table are retrieved using depth.

mingshi1214 commented 2 weeks ago

Hi @pattacini

Q1: Is there a way for me to get constant disparity maps from the stereo-vision module on icubSim gazebo? My goal is to visualize the constant disparity map calculations and play around with the stereo-vision gui but it seems like it requires me to run calibration to retrieve the SFM.ini file.

Right now I am currently running these steps below but it results in 

No local calibration file found in /home/mingshi/.local/share/yarp/contexts/cameraCalibration/SFM.ini ... Using Kinematics and Running SFM once.

with black screens for dispViewer and matchViewer.

Q2: I tried looking at the sandbox code example. I see that it uses the left camera's depthImage as depth for the pointcloud. I was wondering if the depthImage in the left camera was obtained though stereo matching or through a depth sensor? If it is through matching, why is the quality of this depthImage so much better than the disparity maps other users have linked through their issues?

yarpserver
gazebo tutorial_joint-interface.sdf 
yarpmotorgui --robot icubSim (I then move the left hand into view)

yarpview --name /view/left 
yarp connect /icubSim/cam/left/rgbImage:o /view/left
yarpview --name /view/right
yarp connect /icubSim/cam/right/rgbImage:o /view/right
yarpview --name /matchView
yarpview --name /dispView
yarpview --name /depthLeft

iKinGazectrl --robot icubSim
SFM --robot icubSim

yarp connect /SFM/disp:o /dispViewer
yarp connect /SFM/match:o /matchViewer
yarp connect /icubSim/cam/left/depthImage:o /depthLeft

This is what I am able to see: image

pattacini commented 2 weeks ago

🔲 Q1

SFM has been devised for working on the robot with real images and not virtual images, which are in turn full of artifacts that can make disparity computation quite a tough job. That's mainly the reason why the final outcome is not good.

At any rate, it's not a problem if SFM doesn't find the calibration file, which is mandatory for a real robot. On a simulated robot, SFM will connect to iKinGazeCtrl to get adjusted to the simulated stereo-rig, which is meant to be exact.

See:

This self-calibration is performed only once though, meaning that if you move the eyes you'll lose it[^1]. Therefore, configure the eyes and only then launch SFM.

Also, it's important to launch iKinGazeCtrl correctly as per this list of command-line options:

Thereby, the correct signature is[^2]:

iKinGazeCtrl --context gazeboCartesianControl --from iKinGazeCtrl.ini

🔲 Q2

. I was wondering if the depthImage in the left camera was obtained though stereo matching or through a depth sensor? If it is through matching, why is the quality of this depthImage so much better than the disparity maps other users have linked through their issues?

It's a virtual sensor, indeed. See:

[^1]: Same thing on a physical robot. [^2]: To this end, make sure you're aligned with a SW distro that is decently recent.