search

allenai / scibert

A BERT model for scientific text.
https://arxiv.org/abs/1903.10676
Apache License 2.0
1.47k stars 214 forks source link

When were the most recent publications of pre-training data included? #116

Open kaisugi opened 3 years ago

kaisugi commented 3 years ago

I know that SciBERT is pre-trained by the Semantic Scholar corpus. I also know that the Semantic Scholar corpus is not publicly available.

I am wondering how many new papers are included in the pre-training data. For example, are papers from ACL 2018 included? The Semantic Scholar Corpus paper was published in 2018 or so, so I'm guessing that's right around the borderline between having a paper...