Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction

[Fujiry0]

Paper: http://openaccess.thecvf.com/content_CVPR_2020/papers/Mohamed_Social-STGCNN_A_Social_Spatio-Temporal_Graph_Convolutional_Neural_Network_for_Human_CVPR_2020_paper.pdf

Short summary: Social Spatio-Temporal Graph Convolutional Neural Network (Social-STGCNN), which substitutes the need of aggregation methods by modeling the interactions as a graph.

Overcome the limitation of existing methods

• Recurrent architectures
  • Parameter inefficient and expensive in training
• Aggregation layers
  • The aggregation in feature states is neither intuitive nor direct in modeling interactions between people, as the physical meaning of feature states is difficult to interpret.
  • Since the aggregation mechanisms are usually based on heuristics like pooling, they could fail in modeling interactions between pedestrians correctly.

Details

• Method: Supervised deep neural networks
• Input: A set of N pedestrians in a scene with their cor-responding observed positions
• Output: Predicted distribution
• The Social-STGCNN model consists of two main parts:
  • ST-GCNN Conduct spatio-temporal convolution operations on the graph representation of pedestrian trajectories to extract features.
  • TXP-CNN Takes these features as inputs and predicts the future trajectories of all pedestrians as a whole.

GitHub: https://github.com/abduallahmohamed/Social-STGCNN

[jayakornv]

I just read this and I find the explanation to be terrible... I don't understand how to obtain the mean and covariance of the assumed Gaussian from the model. Also, it seems the second network with convolution seems to mean convolution in 2D space (as the code use Conv2d), but I don't understand the meaning of this tensor that needs the conv2d as it is not explained in the paper.