search

wenbowen123 / iros20-6d-pose-tracking

[IROS 2020] se(3)-TrackNet: Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains
Other
384 stars 66 forks source link

problem in getting inference on novel objects using python predict_ros.py inside docker #60

Closed monajalal closed 1 year ago

monajalal commented 1 year ago

From what I understand, I need to have two versions running, one with roscore and one that runs the inference.

However, when I run roscore, the other one exits or vice versa,

root@ard-gpu-01:/# roswtf
Unable to load plugin [tf.tfwtf] from package [tf]. Exception thrown: [dynamic module does not define init function (init_tf2)]
No package or stack in context
================================================================================
Static checks summary:

No errors or warnings
================================================================================

ROS Master does not appear to be running.
Online graph checks will not be run.
ROS_MASTER_URI is [http://localhost:11311]
root@ard-gpu-01:/# roscore
... logging to /root/.ros/log/9da9ce58-ff1b-11ed-a6d1-6c2408935949/roslaunch-ard-gpu-01-106.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://ard-gpu-01:45991/
ros_comm version 1.12.17

SUMMARY
========

PARAMETERS
 * /rosdistro: kinetic
 * /rosversion: 1.12.17

NODES

Traceback (most recent call last):
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/__init__.py", line 308, in main
    p.start()
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/parent.py", line 279, in start
    self.runner.launch()
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/launch.py", line 657, in launch
    self._setup()
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/launch.py", line 632, in _setup
    launched = self._launch_master()
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/launch.py", line 397, in _launch_master
    validate_master_launch(m, self.is_core, self.is_rostest)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/launch.py", line 83, in validate_master_launch
    if not rosgraph.network.is_local_address(m.get_host()):
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosgraph/network.py", line 175, in is_local_address
    local_addresses = ['localhost'] + get_local_addresses()
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosgraph/network.py", line 217, in get_local_addresses
    import netifaces
ModuleNotFoundError: No module named 'netifaces'
root@ard-gpu-01:/# pip install netifaces
Collecting netifaces
  Downloading netifaces-0.11.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (32 kB)
Installing collected packages: netifaces
Successfully installed netifaces-0.11.0
WARNING: You are using pip version 20.3.3; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3.6 -m pip install --upgrade pip' command.
root@ard-gpu-01:/# roscore
... logging to /root/.ros/log/b75643e0-ff1b-11ed-b152-6c2408935949/roslaunch-ard-gpu-01-123.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://ard-gpu-01:42693/
ros_comm version 1.12.17

SUMMARY
========

PARAMETERS
 * /rosdistro: kinetic
 * /rosversion: 1.12.17

NODES

auto-starting new master
process[master]: started with pid [133]
ROS_MASTER_URI=http://ard-gpu-01:11311/

setting /run_id to b75643e0-ff1b-11ed-b152-6c2408935949
process[rosout-1]: started with pid [146]
started core service [/rosout]

also, netifaces was not installed in the docker.

By the way, do you have instructions for native setup (not docker setup) not using ros?

I want to see how the inference on novel objects works. And I assume novel here means both novel poses, novel objects, and novel classes, am I right?

root@ard-gpu-01:/home/se3_tracknet# python predict_ros.py 
Unable to register with master node [http://localhost:11311]: master may not be running yet. Will keep trying.

Can you show a minimal example of how to use your pretrained model to get 6D pose of novel objects?

monajalal commented 1 year ago

so, I am trying to run natively, for any potential novel test object of interest that you have. Could you please share the complete command along with the necessary artifacts such as dataset_info.yaml?

(tracknet) mona@ard-gpu-01:~/iros20-6d-pose-tracking$ python predict.py 
ckpt_dir: None
dataset_info_path None/../dataset_info.yml
Traceback (most recent call last):
  File "/home/mona/iros20-6d-pose-tracking/predict.py", line 633, in <module>
    with open(dataset_info_path,'r') as ff:
FileNotFoundError: [Errno 2] No such file or directory: 'None/../dataset_info.yml'
monajalal commented 1 year ago

your current dataset_info.yaml shows:

camera:   #Intrinsic params
  height: 480
  width: 640
  focalX: 1.066778000000000020e+03
  focalY: 1.067487000000000080e+03
  centerX: 3.129868999999999915e+02
  centerY: 2.413109000000000037e+02

train_samples: 200000               # Number of training samples
val_samples: 2000                      # Number of validation samples
max_translation: 0.02                # max possible translation in meter bewteen consecutive images in your video
max_rotation: 15                      #max possible rotation in degree bewteen consecutive images in your video
boundingbox: 10                        # Bounding box padding percentage. No need to change
resolution: 176                          # Resolution of the image
distribution: gauss              # no need to change this
renderer: pyrenderer

models:
  0:   # Index in rendered segmentation image
    model_path: /home/se3_tracknet/object_models/bunny/1.ply  # Path to your CAD model

blender:
  range_x: [-0.3,0.3]
  range_y: [-0.3,0.3]
  range_z: [0.4,0.9]
  env_light_range: [0.3,5]
  env_light_color: [[0,0.05], [0,0.05], [0,0.05]]    #color type float
  max_lamp_num: 3
  lamp_brightness: [0.1,1]
  lamp_colors: [[0.5,1], [0.5,1], [0.5,1]]
  lamp_pos_range: [[-3,3], [-3,3], [-2,0]]

texture_folders:  # Images to use as background during syn data generation
  [
    '/media/bowen/e25c9489-2f57-42dd-b076-021c59369fec/DATASET/dtd/images/**/*.jpg',
    '/media/bowen/e25c9489-2f57-42dd-b076-021c59369fec/DATASET/ETH_Synthesizability/**/*.jpg',
  ]

and inside predict.py we have train_data_path arg. What should I set it to in order to use your inference data of novel objects in order to see a demo of your code?

Also not sure if this is related, but I get this error:

(tracknet) mona@ard-gpu-01:~/iros20-6d-pose-tracking$ python produce_train_pair_data.py 
Computing object width
Traceback (most recent call last):
  File "/home/mona/iros20-6d-pose-tracking/produce_train_pair_data.py", line 231, in <module>
    completeBlender()
  File "/home/mona/iros20-6d-pose-tracking/produce_train_pair_data.py", line 158, in completeBlender
    mesh = trimesh.load(dataset_info['models'][0]['model_path'])
  File "/home/mona/anaconda3/envs/tracknet/lib/python3.9/site-packages/trimesh/exchange/load.py", line 116, in load
    ) = parse_file_args(file_obj=file_obj,
  File "/home/mona/anaconda3/envs/tracknet/lib/python3.9/site-packages/trimesh/exchange/load.py", line 630, in parse_file_args
    raise ValueError('string is not a file: {}'.format(file_obj))
ValueError: string is not a file: /home/se3_tracknet/object_models/bunny/1.ply
wenbowen123 commented 1 year ago

By novel object, do you mean the objects other than YCB? If so, you need to first generate the synthetic and train the model on it.