af-a commented 3 years ago

Summary of changes

This pull request addresses introduces a recognize_gesture_action action, which involves two stages:

Gesture classification: classifying an observed gesture according to a set of known gesture examples (based on the method presented in this paper)
Gesture processing: reasoning about the classified gesture; for example, determining the object being pointed to after observing a pointing gesture (Note: so far the processing of only this gesture has been implemented).

The primary inspiration for this action is the robocup "What is that" task, which is handled within the action by processing a pointing gesture. This processing involves finding the object most likely being pointed to by a person (see the functionalities in PointingGestureRecognizer). The approach relies on the openpose library for determining the pose of the pointing person, and the native fasterrcnn_resnet50_fpn Pytorch object detector. In addition, the gesture classifier relies on the fastdtw library.

Note: The GestureClassifier is particularly still a work in progress, and is subject to further improvements.