Closed ehartford closed 1 year ago
If it would be useful to anyone to have my Lora weights I can upload it somewhere.
I've been hanging out on the Alpaca Lora discord.
What params did you use? I run OOM even with mbatchsize of 1.
What params did you use? I run OOM even with mbatchsize of 1.
https://github.com/johnsmith0031/alpaca_lora_4bit/blob/main/arg_parser.py#L49
@ehartford I've been trying to fine-tune 65B (on either alpaca-cleaned or GPT4ALL or SHP ) for a couple days now with little luck... alternatively, there seem to be some 65B alpaca checkpoints on HF. I have the necessary compute and vram to load 65B so if you're willing to help, i'd gladly share the loras or merged model.
hit me up in discord, happy to help
I fine tuned 30b on a 4090 and success running inference on my GPU. It took 40 hours and made my office a sauna.
My instruct dataset: https://github.com/gururise/AlpacaDataCleaned
A few outputs:
Thanks to everyone for your contribution and help, this is as good as it gets on a consumer PC! Now I need to build a dual-3090 SLI system to get more VRAM so I can fine tune 65b!