[WIP][Spot] Spot's person following demo

[sktometometo]

This demo enables Spot to follow person.

https://user-images.githubusercontent.com/9410362/112797926-1ae3b180-90a7-11eb-892a-21e9df53716e.mp4

Prerequities

This demo requires packages below.

• [spot_ros]() with this patch
• [common_msgs]() with this patch
• [jsk_recognition]() with this patch and this patch
• [jsk_spot_startup]() with this patch

How to run

Before running this demo, please launch and prepair a controller.

• jsk_spot_bringup.launch
• object_detection_and_tracking.launch
• multi_object_detector.launch

And then, please run

roslaunch spot_person_follower demo.launch

After this, you can start following behavior by pressing L2 button of the controller. Spot will follow the nearest person at the time of pressing. If you want to stop the behavior, please press the L2 button again.

[k-okada]

@sktometometo where is the node to publish /spot_recognition/bbox_array, /spot_recognition/tracking_labels ?

Cc: @tongtybj

[tongtybj]

@sktometometo

Please add the node of human tracking in your launch file, and I strongly recommend to clean up you commits. Comments like "fix bugs", "update" are trivival. Please specify what kind of bugs or updates you address. Also please add description for this PR.

BTW, I am very interesting with this demo. Can I try today or tomorrow?

[sktometometo]

@k-okada @tongtybj This demo requires outputs of rect_array_in_panorama_to_bounding_box_array node in object_detection_and_tracking.launch and deep_sort_tracker_node.py in multi_object_tracker.launch. Currently, both launch files are in jsk_spot_startup packgae.

Sorry for dirty commit history, this is a develop branch. Now this demo works. So I will clean up commits and add descriptions.

I will go to the lab in today's evening.

[tongtybj]

@sktometometo

I also plan to go to lab today. Can you show this demo at that time?

[sktometometo]

@tongtybj Yes, I can show you current demo. No problem.

[k-okada]

@sktometometo do you have a bag file for this demo? @tongtybj is creating tracking node and he will create sample launch files that publish 3D person / human / cara trajectory.

[sktometometo]

I think rosbag files in this grive directory can be used.

In order to reproduce recognition fuction of Spot, you need 2 ROS workspaces because these rosbag files only contains outputs of sensor data and currently object detection is done by coral_usb_ros First one can be created with rosinstall file in my PR ( README.md file describes how to create a workspace for spot). Second one have to include coral_usb_ros, jsk_robot with my PR, jsk_perception with my PR about rect_array_in_panorama_to_bounding_box_array.

In order to play, please run three launch files.

# In the first workspace
roslaunch jsk_spot_startup play.launch rosbag:=<absolute path to rosbag>

# In the second workspace
roslaunch jsk_spot_startup object_detection_and_tracking.launch

# In the first workspace
roslaunch jsk_spot_startup multi_object_tracker.launch

The third launch file requires CUDA and chainer <= 6.7.0

[tongtybj]

@sktometometo

Thank you for your kind instruction.

So, from my understanding, object detection (from coral_usb_ros) is used to provide the first bounding box to feed and start deep_sorted_tracker, right? And, why you need two ros worksapce, it that coral_sub_ros is based on python3, right?

I am developing a new tracker: https://github.com/tongtybj/detr, and I am going to try this new tracker with your rosbag data first. Hope my new tracker can outperform the deep_sort_tracker.

[sktometometo]

@tongtybj rect_array_in_panorama_to_bounding_box_array and deep_sort_tracker runs independently since deep_sort_tracker use only object detection’s result in 2D image plane. Integration of multi object tracking and 3D geometric information is done in this demo.

[sktometometo]

and yes, coral_usb_ros requires python3 and we needs to create 2 ROS workspaces

[k-okada]

@sktometometo I think you can create bag file that includes images and /edgetpu_human_pose_estimator/output/poses, so that @tongtybj can skip workspace settings.

[sktometometo]

@k-okada Ok I wilv creat it later

[sktometometo]

Moved to https://github.com/jsk-ros-pkg/jsk_demos/pull/1343