johnsmith0031
commented
1 year ago The finetune script is outside of the src path so it would not be installed automatically by pip. Maybe I should move it inside the src folder, but in that case when you need to finetune you still need to know the absolute path of the finetune script. The script is here now: https://github.com/johnsmith0031/alpaca_lora_4bit/blob/winglian-setup_pip/finetune.py
I'll also update the readme for finetune.
Hola, sorry for the possibly naive question -
I tried switching to alpaca_lora_4bit@winglian-setup_pip but I'm not sure how to invoke the finetune.py script from that branch. I see
scripts/serverbut it appears that finetune.py is gone.I may be a tad confused here as Python is not my first language. After pip installing (in a venv or not, with appropriate conda env for cuda dependencies), it isn't clear to me how to run the finetune script.
Is it possible that the pip installable version dropped support for finetuning via command line?
The main branch suggests installing from pip and using that version, but The readme on the specified branch doesn't offer anything about running finetune, and The link in the readme in the specified branch, which describes install process, appears to reference the old/main branch, so also doesn't include updated information.
Would it be possible to provide some instructions, either here in the issue or in one of the README's? If I can get this working, with a little guidance, I am happy to submit a PR to update documentation so others like me can benefit in the future.
As a side-question, is alpaca_lora_4bit still sota for training llama / llama2 from GPTQ's? I started using alpaca_lora_4bit via finetune.py many moons ago, before the qlora paper, and as far as I know, this is still the best/fastest implementation by some margin. However, if there's a comparable method (in terms of training speed and resource usage), I'd be happy to switch to something else.