a bug fix and adding minimal tokens/article to prepare_wiki.sh

n-waves / multifit

The code to reproduce results from paper "MultiFiT: Efficient Multi-lingual Language Model Fine-tuning" https://arxiv.org/abs/1909.04761

MIT License

282 stars 56 forks source link

a bug fix and adding minimal tokens/article to prepare_wiki.sh #49

Closed cahya-wirawan closed 4 years ago

cahya-wirawan commented 4 years ago

Fastai changed the argument name of load_data() function from 'fname' to 'file'. This PR will fix the bug in pretrain_lm.py which still use fname in load_data() function. It add also a question about minimal tokens/article to prepare_wiki.sh with the default value of 100.

PiotrCzapla commented 4 years ago

Thank you! Fyi. I've rewritten the repository to clean up the dead code, use fastai sentencepiece and to make it more consistent in the way we handle hyper-parameters. The work is on multifit branch. I'm going to change to release this as 1.0 soon, so what is in the master will be archived.