Running DOPE Error with cracker model

[astronaut71]

Hi

Hi Im using Ubuntu 16.04, ROS kinetic and having CUDA 10 and cudnn 7.10. Then using the ZED camera and the the following configuration

topic_camera: "/zed/zed_node/rgb/image_rect_color"
topic_camera_info: "/zed/zed_node/rgb_raw/camera_info"
topic_publishing: "dope_zed"
input_is_rectified: False   # Whether the input image is rectified (strongly suggested!)
downscale_height: 400      # if the input image is larger than this, scale it down to this pixel height

Then the camera info like this

image_width: 640
image_height: 480
camera_name: dope/webcam
camera_matrix:
  rows: 3
  cols: 3
  data: [679.4, 0, 621.9, 0, 679.4, 349.6, 0, 0, 1]
distortion_model: plumb_bob
distortion_coefficients:
  rows: 1
  cols: 5
  data: [0, 0, 0, 0, 0]
rectification_matrix:
  rows: 3
  cols: 3
  data: [1, 0, 0, 0, 1, 0, 0, 0, 1]
projection_matrix:
  rows: 3
  cols: 4
  data: [679.4, 0, 621.9, 0, 0, 679.4, 349.6, 0, 0, 0, 1, 0]

When running the roslaunch dope dope.launch while Model loaded in 283.750207901 seconds. Running DOPE... (Listening to camera topic: '/zed/zed_node/rgb/image_rect_color') got this ERROR

[ERROR] [1586609943.985973]: bad callback: <bound method Subscriber.callback of <message_filters.Subscriber object at 0x7f917d862750>>
Traceback (most recent call last):
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/topics.py", line 750, in _invoke_callback
    cb(msg)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 75, in callback
    self.signalMessage(msg)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 57, in signalMessage
    cb(*(msg + args))
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 224, in add
    self.signalMessage(*msgs)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 57, in signalMessage
    cb(*(msg + args))
  File "/home/admini/catkin_deep_object_pose/src/dope/nodes/dope", line 248, in image_callback
    self.config_detect
  File "/home/admini/catkin_deep_object_pose/src/dope/src/dope/inference/detector.py", line 251, in detect_object_in_image
    out, seg = net_model(image_torch)
  File "/home/admini/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/admini/.local/lib/python2.7/site-packages/torch/nn/parallel/data_parallel.py", line 112, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/home/admini/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/admini/catkin_deep_object_pose/src/dope/src/dope/inference/detector.py", line 91, in forward
    out1 = self.vgg(x)
  File "/home/admini/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/admini/.local/lib/python2.7/site-packages/torch/nn/modules/container.py", line 91, in forward
    input = module(input)
  File "/home/admini/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/admini/.local/lib/python2.7/site-packages/torch/nn/modules/conv.py", line 301, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: CUDNN_STATUS_MAPPING_ERROR

Any help?My objects that I want to detect must be same color and size as in config_pose.yaml?
Thanks

[xdw-123]

I have the same problem. Maybe you need to upgrade your torch and torch vision

[astronaut71]

yes done it. then is ok .. load the model but nothing detect. Must be the object I want to detect same size (in cm) and color as in the config file? So what if my cracker box is bit bigger and different color then in the config file, means not gonna detect?

[xdw-123]

Well, as like as two peas, you need to check the model and your realistic model, which can be seen in the same way.

[astronaut71]

my realistic model not exactly the same as the model. So is it possible to detect something similar the model? lets say different color and bit bigger size then the model?

[xdw-123]

Maybe, but the advice is the same.

[astronaut71]

oh ok ok .got it

[astronaut71]

also is very slow. almost not real time. Im looking for real time 6D pose estimation for real data. Which are much more bigger then the models and with varies dimensions.

[xdw-123]

How do you plan to do real 6D attitude data? We can discuss it together。

[astronaut71]

well either use it end to end detector or, use some detector like YOLO (darknet_ros) that give me the 3d coordinate of the detected bounding box, and use that for ROI and further point cloud processing. So from there u have to do clustering, and PCA and get the oriented bounding box( positions and orientation). Im doing the second approach now , as the end to detector I guess are more on research stage. So Im feeding the point clouds with the centroid points (xmin, xmax, ymain, ymax, zmin, zmax) from the detector and then perform PCA.

[avinashsen707]

https://github.com/avinashsen707/AUBOi5-D435-ROS-DOPE

Checky results, i got it like this

[astronaut71]

So you tried only with realistic box exactly same as the model right? Any different realistic objects did you try?

[avinashsen707]

So you tried only with realistic box exactly same as the model right? Any different realistic objects did you try?

I mad a 3D printed box of exact sizes and applied its cover images. So far now, i in the developing stage of dataset for the realistic object for my project domain