search

wg-perception / linemod

An OR pipeline based on LINE-MOD from OpenCV
47 stars 54 forks source link

Detection don't respond #31

Closed xMutzelx closed 8 years ago

xMutzelx commented 8 years ago

When I try to detect objects I get the following output and nothing happens after this:

rosrun object_recognition_core detection -c  `rospack find object_recognition_linemod`/conf/detection.ros.ork
[ INFO] [1463665199.331471619]: Initialized ROS. node_name: /object_recognition_server
[ INFO] [1463665199.714861103]: System already initialized. node_name: /object_recognition_server
[ INFO] [1463665199.724349959]: Subscribed to topic:/camera/depth_registered/camera_info [queue_size: 1][tcp_nodelay: 0]
[ INFO] [1463665199.727949582]: Subscribed to topic:/camera/rgb/camera_info [queue_size: 1][tcp_nodelay: 0]
[ INFO] [1463665199.731266661]: Subscribed to topic:/camera/depth_registered/image_raw [queue_size: 1][tcp_nodelay: 0]
[ INFO] [1463665199.734579789]: Subscribed to topic:/camera/rgb/image_rect_color [queue_size: 1][tcp_nodelay: 0]

It never publishes something. After like 2 min of waiting the command get the following error message:

[ INFO] [1463665094.353870696]: Initialized ROS. node_name: /object_recognition_server
[ INFO] [1463665094.716635842]: System already initialized. node_name: /object_recognition_server
[ INFO] [1463665094.729798651]: Subscribed to topic:/camera/depth_registered/camera_info [queue_size: 1][tcp_nodelay: 0]
[ INFO] [1463665094.738068257]: Subscribed to topic:/camera/depth_registered/image_raw [queue_size: 1][tcp_nodelay: 0]
[ INFO] [1463665094.747247293]: Subscribed to topic:/camera/rgb/camera_info [queue_size: 1][tcp_nodelay: 0]
[ INFO] [1463665094.754698440]: Subscribed to topic:/camera/rgb/image_rect_color [queue_size: 1][tcp_nodelay: 0]
Traceback (most recent call last):
  File "/home/marcel/catkin_ws/src/object_recognition_core/apps/detection", line 24, in <module>
    run_plasm(args, plasm)
  File "/opt/ros/indigo/lib/python2.7/dist-packages/ecto/opts.py", line 85, in run_plasm
    sched.execute(options.niter)
ecto.CellException:            exception_type  CellException
[cell_name] = pipeline1

[function_name] = process_with_only_these_inputs

[type] = std::bad_alloc

[what] = std::bad_alloc

[when] = While triggering param change callbacks

This is my output from training:

rosrun object_recognition_core training -c `rospack find object_recognition_linemod`/conf/training.ork 
Training 1 objects.
computing object_id: 8e7cd947fb153e19c0437207ec001034
Info,  T0: Load /tmp/fileTzFANj.obj
Info,  T0: Found a matching importer for this file format
Info,  T0: Import root directory is '/tmp/'
Error, T0: OBJ: No object detected to attach a new mesh instance.
Skipping one or more lines with the same contents
Info,  T0: Entering post processing pipeline
Info,  T0: RemoveRedundantMatsProcess finished. Removed 0 redundant and 1 unused materials.
Info,  T0: Points: 0, Lines: 0, Triangles: 3, Polygons: 0 (Meshes, X = removed)
Info,  T0: CalcTangentsProcess finished. Tangents have been calculated
Info,  T0: JoinVerticesProcess finished | Verts in: 1536 out: 478 | ~68.9%
Info,  T0: Cache relevant are 3 meshes (512 faces). Average output ACMR is 1.005859
Info,  T0: Leaving post processing pipeline
texture loaded.
texture loaded.
texture loaded.
Deleting the previous model 8e7cd947fb153e19c0437207ec001ea3 of object 8e7cd947fb153e19c0437207ec001034

I also saw #30, but I am using the tutorial coke.stl

I am using the latest version of ORK, Ros indigo, Ubuntu 14.04 and OpenCV 2.4.11

Thank you very much

JimmyDaSilva commented 8 years ago

It probably means one of these topics is not published.

/camera/depth_registered/camera_info
/camera/depth_registered/image_raw
/camera/rgb/camera_info
/camera/rgb/image_rect_color

If these are not the topics you want to work with, you need to change them in the detection.ros.conf file

xMutzelx commented 8 years ago

The command rostopic list displays all of these topics, so I think they are being published. Is there another possibility to check those topics? Thank you for your response

JimmyDaSilva commented 8 years ago

rostopic hz /topic_name show the rate the topic is published at rostopic echo/topic_name print the topic incoming data

You should go through the ros tutorials ...

xMutzelx commented 8 years ago

Thank you for your reply. Sorry, I am new to ORK and ROS, you are right, I should repeat them. I tried those commands and here is my output:

Tested topics: /camera/depth_registered/camera_info /camera/depth_registered/image_raw /camera/rgb/camera_info /camera/rgb/image_rect_color

rostopic hz /topic_name: Avg. rate is at 29.67 in all four topics (min: 0.005s max: 2.829s std dev: 0.09491s window: 905) And 2 messages were lost (no new messages).

rostopic echo /topic_name: _/camera/rgb/image_rectcolor:

header: 
  seq: 1274
  stamp: 
    secs: 1463987328
    nsecs: 727463616
  frame_id: camera_rgb_optical_frame
height: 480
width: 640
encoding: bgr8
is_bigendian: 0
step: 1920
data: [66, 70, 76, 68, 71, 76, 68, 70, 77, 66, 70, 78, 64, 71, 79, 68, 72, 79, 71, 71, 79, 73, 69, 78, 73, 70, 78, 71, 71, 77, 69, 71, 77, 66, 71, 75, 69, 70, 75, 71, 69, 75, 70, 69, 74, 68, 69, 76, 69, 69, 77, 71, 69, 78, 69, 69, 78, 65, 69, 78, 63, 70, 78, 65, 70, 78, 66, 71, 78, 68, 70, 78, 70, 69, 80, 70, 70, 81, 70, 70, 82, 70, 71, 82, 71, 71, 81, 70, 71, 80, 68, 72, 80, 68, 73, 80, 70, 72, 80, 72, 72, 80, 73, 72, 80, 73, 73, 79, 72, 74, 79, 71, 73, 79, 71, 72, 80, 71, 72, 81, 70, 72, 82, 68, 74, 82, 66, 75, 81, 64, 75, 81, 65, 75, 81, 66, 74, 81, 66, 74, 82, 65, 74, 82, 66, 74, 83, 69, 74, 83, 72, 73, 82, 76, 74, 82, 78, 75, 82, 68, 74, 82, 59, 73, 83, 64, 73, 83, 69, 73, 82, 67, 72, 81, 65, 72, 82, 68, 71, 83, 72, 70, 83, 74, 70, 82, 77, 71, 83, 76, 72, 83, 73, 72, 83, 70, 71, 83, 68, 69, 82, 67, 67, 80, 66, 66, 79, 65, 67, 78, 64, 68, 77, 64, 67, ...

_/camera/rgb/camerainfo:

header: 
  seq: 1407
  stamp: 
    secs: 1463987329
    nsecs: 927471539
  frame_id: camera_rgb_optical_frame
height: 480
width: 640
distortion_model: plumb_bob
D: [0.1735518030360792, -0.2736813074553146, 0.001852840373772948, 0.001706412488158114, 0.0]
K: [516.5592149732722, 0.0, 311.4192807752578, 0.0, 515.9815870409795, 254.3926612299159, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [529.7174072265625, 0.0, 311.735959372345, 0.0, 0.0, 529.3607177734375, 254.5049444462711, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi: 
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: False
---

_/camera/depth_registered/imageraw:

header: 
  seq: 1460
  stamp: 
    secs: 1463987331
    nsecs: 610183708
  frame_id: camera_rgb_optical_frame
height: 480
width: 640
encoding: 16UC1
is_bigendian: 0
step: 1280
data: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...

Later in the sequence there are also other numbers than zero.

_/camera/depth_registered/camerainfo:

header: 
  seq: 1436
  stamp: 
    secs: 1463987330
    nsecs: 778194272
  frame_id: camera_rgb_optical_frame
height: 480
width: 640
distortion_model: plumb_bob
D: [0.1735518030360792, -0.2736813074553146, 0.001852840373772948, 0.001706412488158114, 0.0]
K: [516.5592149732722, 0.0, 311.4192807752578, 0.0, 515.9815870409795, 254.3926612299159, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [529.7174072265625, 0.0, 311.735959372345, 0.0, 0.0, 529.3607177734375, 254.5049444462711, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi: 
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: False
---
JimmyDaSilva commented 8 years ago

I am sorry @xMutzelx . It seems the topics are published correctly so I don't see why the node is not running. I am not a maintainer here, just an advanced user.

As there is no errors published on the terminal to help us debug this, I advise you get the linemod package from the sources (this GitHub repo) and add some prints to figure out yourself what is going on here.

Are you sure about the mesh you are using ? Maybe it is the same issue as #30 . To make sure you can use the following commands to remove the data from the database and then re-train the coke.stl. That you are sure there is only the coke mesh that is trained, and there is only one trained data for the coke file.

rosrun object_recognition_core object_search.py rosrun object_recognition_core object_delete.py OBJECT_ID

xMutzelx commented 8 years ago

No problem @JimmyDaSilva. Thank you for your help! I tested some different meshes and I always have the same problem. BUT when I delete the linemod training data completly and try to run linemod again, I get the following output:

[ INFO] [1464865988.543218649]: Initialized ROS. node_name: /object_recognition_server
[ INFO] [1464865988.813210823]: System already initialized. node_name: /object_recognition_server
[ INFO] [1464865988.820188239]: Subscribed to topic:/camera/depth_registered/camera_info [queue_size: 1][tcp_nodelay: 0]
[ INFO] [1464865988.823587755]: Subscribed to topic:/camera/rgb/camera_info [queue_size: 1][tcp_nodelay: 0]
[ INFO] [1464865988.826767248]: Subscribed to topic:/camera/depth_registered/image_raw [queue_size: 1][tcp_nodelay: 0]
[ INFO] [1464865988.830132147]: Subscribed to topic:/camera/rgb/image_rect_color [queue_size: 1][tcp_nodelay: 0]
No models found for method "LINEMOD"
[ INFO] [1464865989.072141996]: publishing to topic:/recognized_object_array

As it seems, there is a problem with the linemod training data and not with the topics. I already checked my database and all id's and references seems to be okay.

I checked my database and I found something strange: Linemod Training data contains:

Ts, Rs and Ks only contains this: 

%YAML:1.0
matrices:
   []

Is this right or should it be different?

xMutzelx commented 8 years ago

I fixed my problem. Here are my answers link.

kunliu-kelvin commented 8 years ago

Hello, @xMutzelx. Do you know the reason now?

I also use linemod to train and detect the coke can. But there is not the problem you mentioned. I use ubuntu 1404, opencv 2.4.11, linemod (now latest commit: 35aebde), the lateset whole ork until today. I'll write the process.

Before training, I use mesh_add.py to only add this file "coke.obj" to the couchDB database.

Then, I run: rosrun object_recognition_core training -c /home/kun/catkin_ws_ork/src/ork_linemod/conf/training.ork . When I go to check the model document in the couchDB datebase. It looks maybe right and is different from your results:

distances
24.5 KB, text/x-yaml
detector
4.1 MB, text/x-yaml
Ts
0.8 MB, text/x-yaml
Rs
1.6 MB, text/x-yaml
Ks
0.6 MB, text/x-yaml

At last, I run command to detect: rosrun object_recognition_core detection -c /home/kun/catkin_ws_ork/src/ork_linemod/conf/detection.ros.ork. It runs, and the results of detection looks normal but with wrong detections as only one coke can is in the view:

......
[ INFO] [1466337474.397303023] [/object_recognition_server]: "Subscribed to topic:/camera/rgb/image_rect_color [queue_size: 1][tcp_nodelay: 0]"
Reading the mesh file original.obj
Info,  T0: Load .obj
Info,  T0: Found a matching importer for this file format
Info,  T0: Import root directory is './'
Error, T0: OBJ: Unable to locate material file ./coke.obj.mtl
Error, T0: OBJ: failed to locate material material_0, skipping
Error, T0: OBJ: failed to locate material material_1, skipping
Error, T0: OBJ: failed to locate material material_2, skipping
Info,  T0: Entering post processing pipeline
Info,  T0: Points: 0, Lines: 0, Triangles: 3, Polygons: 0 (Meshes, X = removed)
Info,  T0: CalcTangentsProcess finished. Tangents have been calculated
Info,  T0: JoinVerticesProcess finished | Verts in: 1536 out: 494 | ~67.8%
Info,  T0: Cache relevant are 3 meshes (512 faces). Average output ACMR is 1.035156
Info,  T0: Leaving post processing pipeline
Loaded 4bccd0f996dd42e5d508a13ef7037f98 with the number of samples 5084

4bccd0f996dd42e5d508a13ef7037f98 99.0079 icp 0.138474,  ratio 0 or 0
matches  35 / 1 / 58
[ INFO] [1466337477.271069272] [/object_recognition_server]: "publishing to topic:/recognized_object_array"
4bccd0f996dd42e5d508a13ef7037f98 99.4048 icp 0.137057,  ratio 0 or 0
4bccd0f996dd42e5d508a13ef7037f98 92.4603 icp 0.100726,  ratio 0 or 0.330789
matches  56 / 2 / 56
4bccd0f996dd42e5d508a13ef7037f98 99.0079 icp 0.135333,  ratio 0 or 0
4bccd0f996dd42e5d508a13ef7037f98 94.8413 icp 0.100522,  ratio 0 or 0.379902
......

What I really wonder are the errors occurring above, as follows:

Error, T0: OBJ: Unable to locate material file ./coke.obj.mtl
Error, T0: OBJ: failed to locate material material_0, skipping
Error, T0: OBJ: failed to locate material material_1, skipping
Error, T0: OBJ: failed to locate material material_2, skipping

Do you know what these are talking about?

xMutzelx commented 8 years ago

Hi @kliu-kelvin, I know this errors and it is not really relevant. You are using .obj files and these want textures for the detection. I know 2 possibilities to fix it:

  1. You can copy this textures in your directory (in my case it is /tmp and yours is ./) and the errors should disappear.
  2. You can use the coke.stl file (same folder as coke.obj) and it will work too.

I recommend to use the .stl files. If you later want to train objects with reconstruction then these also will be .stl files. Sorry for my late respond.

kunliu-kelvin commented 8 years ago

Fine, thank you! Find I need more knowledge about 3D model files.

On 2016年06月21日 17:12, xMutzelx wrote:

Hi @kliu-kelvin https://github.com/kliu-kelvin, I know this errors and it is not really relevant. You are using .obj files and these want textures for the detection. I know 2 possibilities to fix it:

  1. You can copy this textures https://github.com/wg-perception/ork_tutorials/tree/master/data textures in your directory (in my case it is /tmp and yours is ./) and the errors should disappear.
  2. You can use the coke.stl file (same folder as coke.obj) and it will work too.

I recommend to use the .stl files. If you later want to train objects with reconstruction then these also will be .stl files.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wg-perception/linemod/issues/31#issuecomment-227384278, or mute the thread https://github.com/notifications/unsubscribe/AJAvb-fjB2sQHBDtdmIKE7zq5fKbmUfVks5qN6sBgaJpZM4IiRWt.