search

shlizee / Predict-Cluster

Repository for PREDICT & CLUSTER: Unsupervised Skeleton Based Action Recognition
Other
106 stars 23 forks source link

Action to predict #22

Closed A7777-gp closed 3 years ago

A7777-gp commented 3 years ago

Hello, after dimensionality reduction through encoder, how to connect the features with actions after dimensionality reduction?

sukun1045 commented 3 years ago

I am a little bit confused with your question. What do you mean by connecting the features with actions? We first forward propagate the sequence to the RNN encoder and use the last hidden state as the feature. I guess the dimensionality reduction you mentioned refers to the encoder of the autoencoder, right? we use the lower dimension feature as the final representation of the action. You can either use KNN classification to evaluate the representation or use T-SNE plot to visualize them.

A7777-gp commented 3 years ago

Thank you very much for your answer.I mainly have the following questions: 1. Sequences generate features of lower dimensions through the encoder of the autoencoder. How do you classify actions according to these features?2. Why do you want to view invariant transformation of data?

A7777-gp commented 3 years ago

Can this be applied in pose estimation? Or it can be applied in person re-identification?

sukun1045 commented 3 years ago

@A7777-gp 1. we use the simple K-nearest neighbor (k = 1) method to classify the actions because it won't require any new linear layer or weights. 2. The reason to do view-invariant transformation is to reduce the variance of different viewpoints and we find that is necessary for our unsupervised approach.

We haven't tried our approach to pose estimation or person re-identification problem and it is unclear how exactly to do that. However, the idea behind our work is quite general (using a regeneration task to extract useful latent features), we definitely encourage you to explore various problems.

Thank you

A7777-gp commented 3 years ago

谢谢哥，五一快乐！😀~