Success

[ehartford]

I fine tuned 30b on a 4090 and success running inference on my GPU. It took 40 hours and made my office a sauna.

My instruct dataset: https://github.com/gururise/AlpacaDataCleaned

A few outputs:

Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
Rewrite the lyrics of "Walk Like an Egyptian", in the style of Shakespeare
### Response:

 Verily, thy feet shall stumble not 
In the manner of an Egyptian, O my love 
Thy arms will sway and thy hips will swivel 
As thou dost walk like an Egyptian, O my love 

Though thou be dressed in linen finery 
And thy hair be adorned with jewels galore 
Still thou shalt walk like an Egyptian, O my love 
For verily, 'tis the only way to show off thy wealth, O my love! 

Thou art so graceful as thou dost walk 
With thy hands held high and thy head bowed low 
Let all who see thee take note of this style 
For verily, it is the Egyptian way, O my love! 

Thanks to everyone for your contribution and help, this is as good as it gets on a consumer PC! Now I need to build a dual-3090 SLI system to get more VRAM so I can fine tune 65b!

[ehartford]

If it would be useful to anyone to have my Lora weights I can upload it somewhere.

I've been hanging out on the Alpaca Lora discord.

[brandonj60]

What params did you use? I run OOM even with mbatchsize of 1.

[ehartford]

What params did you use? I run OOM even with mbatchsize of 1.

https://github.com/johnsmith0031/alpaca_lora_4bit/blob/main/arg_parser.py#L49

[alxfoster]

@ehartford I've been trying to fine-tune 65B (on either alpaca-cleaned or GPT4ALL or SHP ) for a couple days now with little luck... alternatively, there seem to be some 65B alpaca checkpoints on HF. I have the necessary compute and vram to load 65B so if you're willing to help, i'd gladly share the loras or merged model.

[ehartford]

hit me up in discord, happy to help