Do you plan on re-training with newer BERT models such as ALBERT?

EmilyAlsentzer / clinicalBERT

repository for Publicly Available Clinical BERT Embeddings

MIT License

673 stars 135 forks source link

Do you plan on re-training with newer BERT models such as ALBERT? #13

Closed Weilin37 closed 4 years ago

Weilin37 commented 4 years ago

Since ALBERT has SOTA performance + it is much smaller in model size, I was wondering if you planned on retraining based on ALBERT rather than vanilla BERT.

EmilyAlsentzer commented 4 years ago

Hi, Weilin. Yes, there are folks in our group who are working on extending clinical BERT to ALBERT. Stay tuned!

MostHumble commented 5 months ago

Hi, Weilin. Yes, there are folks in our group who are working on extending clinical BERT to ALBERT. Stay tuned!

Hi Emiliy, any updates on this?