search

PaddlePaddle / models

Officially maintained, supported by PaddlePaddle, including CV, NLP, Speech, Rec, TS, big models and so on.
Apache License 2.0
6.9k stars 2.91k forks source link

How to add flow attention cluster for the attention cluster model? #5177

Open p1n0cch10 opened 3 years ago

p1n0cch10 commented 3 years ago

Hi, I am using the attention cluster model for video classification. The original paper shows that the model uses the rgb attention, the flow attention, and the audio attention. However, there are only the rgb attention and the audio attention in the code.

Can anyone tell me how to add the flow attention cluster for the model? Many thanks.

chajchaj commented 3 years ago

Thanks for your attention. Because of efficiency，our open source version doesn't use flow data, which is diff from the paper. We'll contact the author of the paper to ask if there is an original implementation, and release it as soon as possible if exists. BTW, we sincerely recommend the new repo https://github.com/PaddlePaddle/PaddleVideo, based on Paddle2.0, including video recognition, action localization, and temporal action detection tasks.

chajchaj commented 3 years ago

For now, you can extract the flow feature by using https://github.com/yjxiong/dense_flow.