Closed rickstaa closed 5 years ago
Hi @rickstaa,
Sorry to hear that you're having problems with the ROS setup. As a disclaimer, we do not use ROS internally as there is significant communications overhead, and ROS has turned out to be unnecessarily involved in general haha. Thus, the ROS package was really just created in case others wanted to use it, and thus it is a bit buggy due to a lack of rigorous testing. I can share with you what I normally do to get it running:
ROS Kinetic has been installed system-wide on the development machine.
1) Create a new virtual env (I would stick to Python2.7 here because I actually never got around to testing the ROS capabilities on Python3. I will do so when I get a chance.). This virtual environment should have access to system-wide packages. (I believe it's the default behaviour, but I also think that it can be disabled.)
2) Install all packages through the appropriate requirements.txt
. (You are correct that this needs to be done beforehand since ROS normally uses rosdep
for dependencies but we didn't want to go with that since some libraries aren't supported in their registry. I will update the installation instructions.)
3) Run catkin make
and source devel/setup.bash
.
4) You might also have to pip install
some libraries here and there like rospkg
and rospy
. This is just due to a lack of thoroughness on my part. I will update the installation instructions for this too.
Once you try this, let me know if there are still any issues and we can go from there!
Thanks, Vishal
@visatish Thanks a lot for your answer. I will do some testing this afternoon and report the result. What do you use internally to compute the grasps and control the robot? My plan was now to build a C++ ROS node that spawns your python 3 detection algorithm as a new process and communicate through a shared memory address or UTP sockets.
@rickstaa Internally we simply query the Python policy. The returned actions have a property called grasp
which has a function pose()
that returns the grasp pose in 3D camera space. You will need to transform to 3D robot space using a known transformation that can be calibrated with a chessboard. So you would execute something like this in code:
T_grasp_camera = action.grasp.pose()
T_grasp_robot = T_camera_robot * T_grasp_camera
For controlling the robot, we use an ABB YuMi and have an open-source Python library here. You will probably have to find something similar for the Panda. Unfortunately, we don't have the bandwidth to help with motion planning itself.
@visatish I did some testing within a python2.7 virtual environment. Ilke you stated your ROS package seems to work inside a python2.7 virtual environment. While following your steps I, however, encountered some errors. For documentation purposes I reported the errors and the solutions to these errors below:
Errors << gqcnn:make /catkin_ws/logs/gqcnn/build.make.000.log
Traceback (most recent call last):
File "/opt/ros/kinetic/share/gencpp/cmake/../../../lib/gencpp/gen_cpp.py", line 41, in <module>
import genmsg.template_tools
File "/opt/ros/kinetic/lib/python2.7/dist-packages/genmsg/template_tools.py", line 39, in <module>
import em
ImportError: No module named 'em'
make[2]: *** [/catkin_ws/devel/.private/gqcnn/include/gqcnn/BoundingBox.h] Error 1
make[2]: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "/opt/ros/kinetic/share/gencpp/cmake/../../../lib/gencpp/gen_cpp.py", line 41, in <module>
import genmsg.template_tools
File "/opt/ros/kinetic/lib/python2.7/dist-packages/genmsg/template_tools.py", line 39, in <module>
import em
ImportError: No module named 'em'
make[2]: *** [/catkin_ws/devel/.private/gqcnn/include/gqcnn/Action.h] Error 1
Traceback (most recent call last):
File "/opt/ros/kinetic/share/gencpp/cmake/../../../lib/gencpp/gen_cpp.py", line 41, in <module>
import genmsg.template_tools
File "/opt/ros/kinetic/lib/python2.7/dist-packages/genmsg/template_tools.py", line 39, in <module>
import em
ImportError: No module named 'em'
make[2]: *** [/catkin_ws/devel/.private/gqcnn/include/gqcnn/Observation.h] Error 1
make[1]: *** [CMakeFiles/gqcnn_generate_messages_cpp.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [all] Error 2
Solved by running pip install empy
After sourcing the ROS setup.bash
file and launching the ROS package using the roslaunch gqcnn grasp_planning_service.launch
command I get the following errors:
Failed to load Python extension for LZ4 support. LZ4 compression will not be available.
Traceback (most recent call last):
File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 46, in <module>
from perception import (CameraIntrinsics, ColorImage, DepthImage, BinaryImage,
File "/catkin_ws/devel/lib/python3/dist-packages/perception/__init__.py", line 35, in <module>
exec(__fh.read())
File "<string>", line 8, in <module>
File "/catkin_ws/src/perception/perception/camera_intrinsics.py", line 14, in <module>
from .image import DepthImage, PointCloudImage
File "/catkin_ws/src/perception/perception/image.py", line 11, in <module>
import cv2
ImportError: /opt/ros/kinetic/lib/python2.7/dist-packages/cv2.so: undefined symbol: PyCObject_Type
[gqcnn/grasp_planner-2] process has died [pid 11905, exit code 1, cmd /catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py __name:=grasp_planner __log:=/root/.ros/log/b6dee622-9815-11e9-a237-98e7f48e3bc4/gqcnn-grasp_planner-2.log].
log file: /root/.ros/log/b6dee622-9815-11e9-a237-98e7f48e3bc4/gqcnn-grasp_planner-2*.log
WARNING:root:autolab_core not installed as catkin package, RigidTransform ros methods will be unavailable
Traceback (most recent call last):
File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 46, in <module>
from perception import (CameraIntrinsics, ColorImage, DepthImage, BinaryImage,
File "build/bdist.linux-x86_64/egg/perception/__init__.py", line 8, in <module>
File "build/bdist.linux-x86_64/egg/perception/camera_intrinsics.py", line 14, in <module>
File "build/bdist.linux-x86_64/egg/perception/image.py", line 27, in <module>
File "/usr/local/lib/python2.7/dist-packages/skimage/__init__.py", line 167, in <module>
from .util.dtype import (img_as_float32,
File "/usr/local/lib/python2.7/dist-packages/skimage/util/__init__.py", line 6, in <module>
from .apply_parallel import apply_parallel
File "/usr/local/lib/python2.7/dist-packages/skimage/util/apply_parallel.py", line 8, in <module>
import dask.array as da
File "/usr/local/lib/python2.7/dist-packages/dask/array/__init__.py", line 5, in <module>
from .core import (Array, block, concatenate, stack, from_array, store,
File "/usr/local/lib/python2.7/dist-packages/dask/array/core.py", line 29, in <module>
from . import chunk
File "/usr/local/lib/python2.7/dist-packages/dask/array/chunk.py", line 73, in <module>
nancumprod = npcompat.nancumprod
AttributeError: 'module' object has no attribute 'nancumprod'
[gqcnn/grasp_planner-2] process has died [pid 27812, exit code 1, cmd /catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py __name:=grasp_planner __log:=/root/.ros/log/278e867a-9820-11e9-a237-98e7f48e3bc4/gqcnn-grasp_planner-2.log].
log file: /root/.ros/log/278e867a-9820-11e9-a237-98e7f48e3bc4/gqcnn-grasp_planner-2*.log
This problem is explained in this topic. The solution is to add the virtual environment site-packages
folder before the ROS dist-packages
folder. This can be done running the following code:
_venv_site_packages=$(find "$VIRTUAL_ENV/$2" -iname site-packages)
export PYTHONPATH="$_venv_site_packages:$PYTHONPATH"
To later undo this the following command can be used:
export PYTHONPATH=$(echo $PYTHONPATH | sed -e "s:$_venv_site_packages\:::")
Traceback (most recent call last):
File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 410, in <module>
grasping_policy = CrossEntropyRobustGraspingPolicy(policy_cfg)
File "/catkin_ws/src/gqcnn/gqcnn/grasping/policy/policy.py", line 726, in __init__
GraspingPolicy.__init__(self, config)
File "/catkin_ws/src/gqcnn/gqcnn/grasping/policy/policy.py", line 310, in __init__
metric_type, self._metric_config)
File "/catkin_ws/src/gqcnn/gqcnn/grasping/grasp_quality_function.py", line 1303, in quality_function
return GQCnnQualityFunction(config)
File "/catkin_ws/src/gqcnn/gqcnn/grasping/grasp_quality_function.py", line 931, in __init__
self._gqcnn = get_gqcnn_model().load(self._gqcnn_model_dir)
File "/catkin_ws/src/gqcnn/gqcnn/model/tf/network_tf.py", line 218, in load
gqcnn.init_mean_and_std(model_dir)
File "/catkin_ws/src/gqcnn/gqcnn/model/tf/network_tf.py", line 245, in init_mean_and_std
except FileNotFoundError:
NameError: global name 'FileNotFoundError' is not defined
This error is not present when I supply the model_name:=GQCNN-4.0-PJ
argument with my roslaunch command.
System information
Describe what you are trying to do
I am trying to test your GQCNN ROS package on an Emika Franka Panda robot.
Problem description
Before applying it on the real robot I first am testing out the GQCNN package inside a gazebo simulation. To do this I tried launching the ROS node both in a python2 and python3 environment.
Python 2.7.15 virtual environment (virtualenv package)
When I try to launch the ROS node using the
roslaunch gqcnn grasp_planning_service.launch model_name:=GQCNN-4.0-PJ
command inside the pyhton 2.7 enviroment I get the following error:Due to this error I assumed I needed to install the packages present inside the
setup.py
file. I, therefore, created a python 2.7.15 virtual environment, sourced the environment and executed the following commands:python -m pip install -r gqcnn/requirements/p2/gpu_requirements.txt
However, if while in the enviroment if I want to build the ROS package again I get the following error:
This is is not strange since the rospackages are installed on the global python 2.7.15 interperter while I'm inside a virtual enviroment. I therefore deactivated the virtual env and build the ROS package again. When I then try to launch the ROS package from within in the virtual enviroment I get the following error:
This again is caused by the rospackages being installed in the global python interperter. This behavoir is explained on this topic. I tried to solve this by running the following command:
However, I now got a lot of other errors. So I'm unsure if I'm using the package in the right way?
python 3.7.3 conda environment
In conda I got similar errors due to the fact that the ROS packages are installed in the global python 2.7 interpreter. I tried solving some of them using the
pip install rospkg
command. However, due to the fact that the ROS node is written in python 2.7 (no python3.7 support for ROS kinetic), I get the following error:Build all catkin_packages (Autolab_core, perception, visualization, gqcnn & meshrender) in global 2.7 python
I also tried building all berkleyautomation packages using
catkin build
and installing all the python requirements of each packages using the setup.py files. In this case I also keep running into dependencies and errors. After running thesetup.py
files the following packages appeared to be missing:After solving for all the dependencies I get the following error:
I have yet to solve this issue but I first wanted to check if I'm not installing the GQCNN package the wrong way.
Question
Can you maybe elaborate on how the ROS package should be used? Do I need to install the python packages in order to use the ROS package? And if so in which environment do I need to install the python packages? And in that case which python version do I need to use? Can I use it with the python 3.7 packages out of the box or do I need to use the multiprocessing library and the socket library in order to achieve this?
Thanks a lot in advance,