search

benedekrozemberczki / pytorch_geometric_temporal

PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models (CIKM 2021)
MIT License
2.61k stars 367 forks source link

Classification of a sequence of graphs #178

Closed Borzyszkowski closed 2 years ago

Borzyszkowski commented 2 years ago

Hello!

In my project, I work on motion classification and was wondering if pytorch geometric temporal could be used for this application. Namely, my dataset consists of a sequence of graphs such that each graph represents a hand pose, where joints are given as nodes and bones are given as edges. The graph is static with temporal signal (3 dynamic features in each of 21 nodes).

I implemented a GCN for graph classification using pytorch geometric, which takes the following data representation: Data(x=[21, 3], edge_index=[2, 40], y=[1])

Now, I would like to use the temporal relationship between the graphs and classify their whole sequence instead of a single graph. However, I encountered the following issues: -StaticGraphTemporalSignal requires to define targets (labels) for each graph, which is in conflict with my case -I could not pass entire sequence of graphs to the neural network, eg. DCRNN. For instance, examples given on the website calculate loss for each graph separately, and not for the sequence of graphs as a whole.

My approach is quite similar to the work presented in this paper and I was wondering if using pytorch geometric temporal for this case would be possible. Any advice will be appreciated!

Best, Bartek

Borzyszkowski commented 2 years ago

I managed to run ST-GCN and 2s-AGCN, according to the papers [1 and 2] respectively.

I used the following graph: Data(x=[4000, 21, 3], edge_index=[4000, 2, 40], y=[4000, 1]) where in my case 4000 is a fixed size of a sequence.

gbastiandillo commented 1 year ago

Hi @Borzyszkowski , how're you? Can you share how many group of sequences did u used? (explained in other way, if each sequence it's a video, how many videos did u used?) Do you think can share your final model architecture?

Thanks!