Allows to fine-tune Falcon 7b or any other model. Tested for Falcon 7b but not others.
If we fine-tune a model, the parameter lora_target_modules needs to be specified. This is a list of module names that the lora finetuning targets. These can be found on the model page on HugginFace in the class Attention of the model. For example for Falcon 7b the file is: https://huggingface.co/tiiuae/falcon-7b/blob/main/modelling_RW.py and the lora_target_modules = ["query_key_value"]
Allows to fine-tune Falcon 7b or any other model. Tested for Falcon 7b but not others.
If we fine-tune a model, the parameter
lora_target_modulesneeds to be specified. This is a list of module names that the lora finetuning targets. These can be found on the model page on HugginFace in theclass Attentionof the model. For example for Falcon 7b the file is: https://huggingface.co/tiiuae/falcon-7b/blob/main/modelling_RW.py and thelora_target_modules = ["query_key_value"]