How to deploy this code on a real machine?

[mmahdavian]

Hi @Pradur241. What do you mean by real machine?

[Pradur241]

Hi @mmahdavian I saw in the paper that you have verified the effect of this algorithm on a robot, and would like to know how to replicate this work?

[mmahdavian]

@Pradur241 First, You need to have a robot connected to a laptop or boards such as Jetson to load and run the robot and the model. You also need a camera to receive RGB data. Later you can use a 3D pose extractor to detect 3D human pose. We used Zed2 camera which already provides the human 3D pose. The camera can be attached to the robot or used as a 3D observer. Remember if you use the camera on the robot you need to either use multiple cameras or an actuator underneath the camera to keep the human inside the viewing angle all the time. We used the camera as a 3rd person. Then you need to have the robot pose using the odometery provided by robot encoders or any other methods. When you have the human, robot and camera poses you can find relative pose of human with respect to the robot. Then you can feed a sequence of the poses to the trained model and predict the human future pose and trajectory. Later you can use a planner such as TEB to move the robot in front of the human and keep following the human.

[Pradur241]

@mmahdavian Thanks for your reply. One more question, do I need to change the h36mdataset function when using this code for the robot?

[mmahdavian]

@Pradur241 The dataset is not relevant to the application of the model. You can use the model for your robot without changing the h36m dataset functions. Only if you need to change the number of body joints or something similar, you might need to change some parts of the code.

[Pradur241]

@mmahdavian Thanks !!!!! I will try it later!