spatial-temporal relative positional embedding?

facebookresearch / mvit

Code Release for MViTv2 on Image Recognition.

Apache License 2.0

391 stars 46 forks source link

spatial-temporal relative positional embedding? #4

Closed JunweiLiang closed 2 years ago

JunweiLiang commented 2 years ago

Any suggestions on how to implement spatial-temporal relative positional embeddings? I'm trying to extend based on the cal_rel_pos_spatial function in attention.py

Thanks, Junwei

lyttonhao commented 2 years ago

Hi, you can follow cal_rel_pos_spatial to add a separate temporal rel pos embedding. Our official spatial-temporal rel pos embedding will be released in PySlowFast (our video codebase) soon together with the MViTv2 video models.