Closed arbindpd96 closed 10 months ago
Hi, the problem is that the model architecture of llamaPeft is different from the original llama (lora and bias are added), but huggingface only contains the original llama implementation. If you want to convert such models to HF, you need to first reimplement the architecture using huggingface, and then write the conversion logic correspondingly.
Hey thanks for that response.
Can these checkpoints be used with llama.cpp or as a quantized single bin file. or even be converted to ggml format? or all these would require custom scripts ?
@ChrisLiu6 Hi Please suggest.
I'm sorry, but I'm really not familiar with llama.cpp😣. Unfortunately, no one on our team is working on that. The followings are all I can provide on this topic:
convert_weights_to_hf
script is designed exclusively for the original llama architecture. Therefore, you may fully finetune your model using Accessory and then convert it to Hugging Face. You can refer to this experiment.convert_weights_to_hf
script to deal with biases.
I am trying to convert a fine tuned checkpoint to hf so i can make the model work with llama.cpp Image attached for reference Any help would be appreciated.
Also in the latest commit