search

pytorch-labs / gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.
BSD 3-Clause "New" or "Revised" License
5.56k stars 505 forks source link

tokenizer.model #186

Open hasakikiki opened 3 months ago

hasakikiki commented 3 months ago

I fine-tuned an llm based on the llama skeleton and used convert_hf_checkpoint and quantize to complete the quantification. However, when generating, the tokenizer.model file is missing. How can I operate or generate it?

yanboliang commented 2 weeks ago

Which model are you going to inference? If it's a Llama 3+ model, we already copied it to the right place in convert_hf_checkpoint. https://github.com/pytorch-labs/gpt-fast/blob/c9f683edd4f89d3e81ed8f52387e866a245e3226/scripts/convert_hf_checkpoint.py#L118-L126