search

aixcoder-plugin / aiXcoder-7B

official repository of aiXcoder-7B Code Large Language Model
Apache License 2.0
2.23k stars 351 forks source link

[New Feature] Add some instructions for Hugging Face based methods #29

Open YoctoHan opened 7 months ago

YoctoHan commented 7 months ago

While previous version didn't explain how the weights should be loaded, most runs used the default loading method (downloaded from the modelscope), this version some instructions will be added to help those who want to run the model based on the hugging face framework loading local weight.

Josephrp commented 7 months ago

i can help if you are reaching otu to outside collaborators for this :-)

HoratioJSY commented 7 months ago

Thank you for your kind assistance.

If you have downloaded model weights from HuggingFace or ModelScope through various methods, you can change the inference code to your local address for inference. For example, if I download the model to the local directory "D:\data\aix-7b-model", then the inference code can be modified to load the model from this location:


tokenizer = AutoTokenizer.from_pretrained("D:\data\aix-7b-model")
model = AutoModelForCausalLM.from_pretrained("D:\data\aix-7b-model", torch_dtype=torch.bfloat16)