Closed Lukeehoojet closed 6 months ago
In this Convert Vocabulary page, the shell script python vocab.py --input_file=tokenizer.model --output_file=vocab.mllm --type=Unigram should be python vocab.py --input_file=tokenizer.json --output_file=vocab.mllm --type=Unigram.
python vocab.py --input_file=tokenizer.model --output_file=vocab.mllm --type=Unigram
python vocab.py --input_file=tokenizer.json --output_file=vocab.mllm --type=Unigram
fixed
In this Convert Vocabulary page, the shell script
python vocab.py --input_file=tokenizer.model --output_file=vocab.mllm --type=Unigram
should bepython vocab.py --input_file=tokenizer.json --output_file=vocab.mllm --type=Unigram
.