-
There is code python3 /workspace/bookcorpus/download_files.py in BooksDownloader. But I can't find this script.
-
# UniLM checkpoint
wget -O bert_save.tar.gz "https://onedrive.live.com/download?cid=E5364FD183A1F5BB&resid=E5364FD183A1F5BB%212016&authkey=AB5-lxzCkgpfLhg"
The above link is no longer anymore. Tha…
-
Hi Sosuke,
Thanks a lot for the wonderful work! I expect to obtain the bookcorpus dataset with your crawler, but I failed to crawl the articles owing to some network errors. I am afraid that I cann…
-
Since I am using the remote LINUX machine without root and container. I am trying to download the BookCorpus data and find it doesn't work. How can i find the "/workspace/bookcorpus/download_files.py"…
-
I'm following [this](https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT/biobert) guide. At step 2 I'm required to execute
```
$ bash scripts/docker/build.sh
…
mneri updated
4 years ago
-
Hi,
Thank you for the great work. I was wondering if the model you have for download (model-base-uncased) has already been pre-trained or do we need to do pre-training ourselves?
Thanks!
-
## ❓ Questions & Help
I am trying to train distilbert with different architecture. If you can share the text dump for the pre-training, it would be great. Thanks!
-
Hello,
Are you planning on releasing English pre-trained versions of Albert in the future?
Thank you,
-
The base model has a very powerful performance !
What are the training steps / batch size / learning rate for the base model ? Is that all same with the large model ?
Do you have any other corpus fo…
-
Hi,
In `Pre-training VL-BERT` section, you've highlighed some tasks on which model was trained.
1. Pretraining on Conceptual Conceptions
2. Masked Language Modeling with Visual Clues
3. Mask…