search

facebookresearch / fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
MIT License
30.16k stars 6.37k forks source link

[Linformer] Pre-Trained Weights #2795

Open patrickvonplaten opened 3 years ago

patrickvonplaten commented 3 years ago

Thanks a lot for adding the official code for Linformer to FairSeq! Are you also planning on releasing some pre-trained weights for the model?

myleott commented 3 years ago

@sinonwang, @madian9

madian9 commented 3 years ago

Thank you for your interest. We plan to release it, but we don't have ETA yet.

yuxiang-wu commented 3 years ago

@madian9 Is there any update on releasing the pretrained weights for Linformer?

NielsRogge commented 3 years ago

@madian9 @sinonwang @myleott I've a PR on the HuggingFace Transformers library that implements Linformer, in both PyTorch and Tensorflow. It would be great if we we can equip these models with pre-trained weights.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale. If this issue is still affecting you, please leave any comment (for example, "bump"), and we'll keep it open. We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!

mcgowra2 commented 2 years ago

@madian9 Any updates on if the pretrained weights may be released soon?

Yfeng-44 commented 2 years ago

Hi, do this thread have any news?

aamir-s18 commented 8 months ago

@sinonwang, @madian9 @myleott any updates?