Pre-train model - Githubissues

OpenNLPLab / cosFormer

[ICLR 2022] Official implementation of cosformer-attention in cosFormer: Rethinking Softmax in Attention

Apache License 2.0

176 stars 25 forks source link

Pre-train model #7

Open csorujian opened 2 years ago

csorujian commented 2 years ago

In the paper，it mentioned that the work of the bidirectional language modeling pre-train has been done. Are you planning on releasing some pre-trained weights for the model?