Closed boss-chanon closed 1 year ago
can use train_extremely_large_corpus for train tokenizer large corpus
train_extremely_large_corpus
large_corpus
output_path
Close #
Why this PR
can use
train_extremely_large_corpus
for train tokenizer large corpusChanges
large_corpus
argument tooutput_path
for activatetrain_extremely_large_corpus
Related Issues
Close #
Checklist