Eval ROS-industrial's ai pick libraries

[130s]

Motivation https://github.com/kinu-garage/essay_in_idleness/issues/215

CoS

• [ ] Build, dry run. I/O, difficulty
• [ ] Grasp Planning: I/O, difficulty, how generic?
• [ ] Workcell: I/O, difficulty, how generic?

[130s]

Some shady info found on FAQ:

Workcell Builder

Q. I have my own object_description folders for existing objects that I want to to load into the scene. How do I load it into the Workcell Builder? A. It is currently not possible to load your own objects into the scene. You need to create it in the GUI. Object loading features will be included in the future versions of this package.

I'm unsure what this means. Hope this doesn't mean existing models defined in an existing .sdf file cannot be used.

Grasp Planner

Q. Is it possible to do a side grasp rather than a top down grasp? A. Yes, but your camera would then be required to then face the side which you want to grasp.

??? Camera view angle at the to-be-grasped object restricted?

[130s]

Looking at the diagram https://easy-manipulation-deployment-docs.readthedocs.io/en/latest/about/overview.html, I guess some epd* topics are requirement for EMD, so might be easier to also use EPD.

According to https://easy-manipulation-deployment-docs.readthedocs.io/en/latest/emd_packages/grasp_planner/grasp_planner_input.html#grasp-planner-input, it's either pointcloud2 or epd* topics, not necessarilly both.

https://easy-manipulation-deployment-docs.readthedocs.io/en/latest/emd_packages/grasp_planner/grasp_planner_input.html#greater-hardware-requriements-epd-workflow

Due to the higher hardware limitations for deploying of deep learning models, hardware requirements will be higher for the EPD workflow. Check the easy_perception_deployment documentation for a more comprehensive hardware requirement specifications.

Concerning and also links are broken. Fix suggested https://github.com/tanjpg/emd_docs/pull/1

[130s]

Build failure

``` s# colcon build --parallel-workers 4 --packages-up-to emd_grasp_planner Starting >>> emd_msgs Finished <<< emd_msgs [0.37s] Starting >>> emd_grasp_planner [Processing: emd_grasp_planner] --- stderr: emd_grasp_planner ** WARNING ** io features related to pcap will be disabled ** WARNING ** io features related to png will be disabled ** WARNING ** io features related to libusb-1.0 will be disabled /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/end_effectors/finger_gripper.cpp: In member function ‘Eigen::Vector3f FingerGripper::get_gripper_pla ne(std::shared_ptr&, std::shared_ptr&, const Vector3f&, const GraspObject&)’: /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/end_effectors/finger_gripper.cpp:1362:35: warning: unused parameter ‘finger_sample_1’ [-Wunused-parameter] 1362 | std::shared_ptr & finger_sample_1, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~ /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/end_effectors/finger_gripper.cpp:1363:35: warning: unused parameter ‘finger_sample_2’ [-Wunused-para meter] 1363 | std::shared_ptr & finger_sample_2, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~ In file included from /opt/ros/galactic/include/rclcpp/client.hpp:40, from /opt/ros/galactic/include/rclcpp/callback_group.hpp:23, : /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp: In member function ‘void grasp_planner::GraspScene::extract_objects_epd(const st d::vector >, std::allocator > > >&)’: /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp:334:84: error: cannot convert ‘std::__cxx11::basic_string’ to ‘const char*’ 334 | RCLCPP_INFO(LOGGER, "EPD detected " + std::to_string(this->grasp_objects.size()) + " objects."); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~ || |std::__cxx11::basic_string In file included from /opt/ros/galactic/include/rmw/types.h:28, from /opt/ros/galactic/include/rcl/types.h:20, from /opt/ros/galactic/include/rcl/log_level.h:22, from /opt/ros/galactic/include/rcl/arguments.h:21, from /opt/ros/galactic/include/rcl/context.h:28, from /opt/ros/galactic/include/rcl/guard_condition.h:26, from /opt/ros/galactic/include/rclcpp/executor.hpp:30, from /opt/ros/galactic/include/rclcpp/executors/multi_threaded_executor.hpp:26, from /opt/ros/galactic/include/rclcpp/executors.hpp:21, from /opt/ros/galactic/include/rclcpp/rclcpp.hpp:156, from /opt/ros/galactic/include/tf2_ros/buffer_interface.h:43, from /opt/ros/galactic/include/tf2_ros/buffer.h:36, from /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp:28, from /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp:17: /opt/ros/galactic/include/rcutils/logging.h:479:16: note: initializing argument 4 of ‘void rcutils_log(const rcutils_log_location_t*, int, const char*, const char*, ...)’ 479 | const char * format, | ~~~~~~~~~~~~~^~~~~~ /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp: In member function ‘void grasp_planner::GraspScene::extract_objects(const typena me T::ConstSharedPtr&) [with T = sensor_msgs::msg::PointCloud2_ >; typename T::ConstSharedPtr = std::shared_ptr > >]’: /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp:340:57: warning: unused parameter ‘msg’ [-Wunused-parameter] 340 | const sensor_msgs::msg::PointCloud2::ConstSharedPtr & msg) | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~ In file included from /opt/ros/galactic/include/rclcpp/client.hpp:40, from /opt/ros/galactic/include/rclcpp/callback_group.hpp:23, from /opt/ros/galactic/include/rclcpp/any_executable.hpp:20, from /opt/ros/galactic/include/rclcpp/memory_strategy.hpp:25, from /opt/ros/galactic/include/rclcpp/memory_strategies.hpp:18, from /opt/ros/galactic/include/rclcpp/executor_options.hpp:20, from /opt/ros/galactic/include/rclcpp/executor.hpp:36, from /opt/ros/galactic/include/rclcpp/executors/multi_threaded_executor.hpp:26, from /opt/ros/galactic/include/rclcpp/executors.hpp:21, from /opt/ros/galactic/include/rclcpp/rclcpp.hpp:156, from /opt/ros/galactic/include/tf2_ros/buffer_interface.h:43, from /opt/ros/galactic/include/tf2_ros/buffer.h:36, from /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp:28, from /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp:17: /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp: In member function ‘void grasp_planner::GraspScene::setup(std::string)’: /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp:550:53: error: cannot convert ‘std::__cxx11::basic_string’ to ‘const char*’ 550 | RCLCPP_INFO(LOGGER, "Listening to: " + topic_name + "..."); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~ | | | std::__cxx11::basic_string In file included from /opt/ros/galactic/include/rmw/types.h:28, from /opt/ros/galactic/include/rcl/types.h:20, from /opt/ros/galactic/include/rcl/log_level.h:22, from /opt/ros/galactic/include/rcl/arguments.h:21, from /opt/ros/galactic/include/rcl/context.h:28, : /opt/ros/galactic/include/rcutils/logging.h:479:16: note: initializing argument 4 of ‘void rcutils_log(const rcutils_log_location_t*, int, const char*, const char*, ...)’ 479 | const char * format, | ~~~~~~~~~~~~~^~~~~~ In file included from /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp:17: /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp: In instantiation of ‘grasp_planner::GraspScene::GraspScene(const SharedPtr&) [with T = sensor_msgs::msg::PointCloud2_ >; rclcpp::Node::SharedPtr = std::shared_ptr]’: /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp:667:31: required from here /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp:220:42: warning: ‘grasp_planner::GraspScene > >::viewer’ will be initialized after [-Wreorder] 220 | pcl::visualization::PCLVisualizer::Ptr viewer; | ^~~~~~ /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp:125:27: warning: ‘rclcpp::Node::SharedPtr grasp_planner::GraspScene > >::node’ [-Wreorder] 125 | rclcpp::Node::SharedPtr node; | ^~~~ /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp:80:3: warning: when initialized here [-Wreorder] 80 | GraspScene(const rclcpp::Node::SharedPtr & node_) | ^~~~~~~~~~ /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp: In instantiation of ‘grasp_planner::GraspScene::GraspScene(const SharedPtr&) [with T = epd_msgs::msg::EPDObjectLocalization_ >; rclcpp::Node::SharedPtr = std::shared_ptr]’: /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp:669:31: required from here /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp:220:42: warning: ‘grasp_planner::GraspScene > >::viewer’ will be initialized after [-Wreorder] 220 | pcl::visualization::PCLVisualizer::Ptr viewer; | ^~~~~~ /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp:125:27: warning: ‘rclcpp::Node::SharedPtr grasp_planner::GraspScene > >::node’ [-Wreorder] 125 | rclcpp::Node::SharedPtr node; | ^~~~ /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp:80:3: warning: when initialized here [-Wreorder] 80 | GraspScene(const rclcpp::Node::SharedPtr & node_) | ^~~~~~~~~~ /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp: In instantiation of ‘grasp_planner::GraspScene::GraspScene(const SharedPtr&) [with T = epd_msgs::msg::EPDObjectTracking_ >; rclcpp::Node::SharedPtr = std::shared_ptr]’: /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/src/grasp_scene.cpp:670:31: required from here /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp:220:42: warning: ‘grasp_planner::GraspScene > >::viewer’ will be initialized after [-Wreorder] 220 | pcl::visualization::PCLVisualizer::Ptr viewer; | ^~~~~~ /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp:125:27: warning: ‘rclcpp::Node::SharedPtr grasp_planner::GraspScene > >::node’ [-Wreorder] 125 | rclcpp::Node::SharedPtr node; | ^~~~ /cws/src/ros-industrial/easy_manipulation_deployment/easy_manipulation_deployment/emd_grasp_planner/include/emd/grasp_planner/grasp_scene.hpp:80:3: warning: when initialized here [-Wreorder] 80 | GraspScene(const rclcpp::Node::SharedPtr & node_) | ^~~~~~~~~~ make[2]: *** [CMakeFiles/grasp_planning_interface.dir/build.make:63: CMakeFiles/grasp_planning_interface.dir/src/grasp_scene.cpp.o] Error 1 make[1]: *** [CMakeFiles/Makefile2:101: CMakeFiles/grasp_planning_interface.dir/all] Error 2 make: *** [Makefile:141: all] Error 2 --- Failed <<< emd_grasp_planner [45.7s, exited with code 2] Summary: 1 package finished [46.2s] 1 package failed: emd_grasp_planner 1 package had stderr output: emd_grasp_planner ```

Is this happening b/c I'm building on Galatic while the recommended ros distro is foxy?

Looks like it. This doesn't happen on foxy.