Able to finetune `homebrewltd/llama3.1-s-instruct-v0.2` (Input=Text & Audio, Output=Text)

[asmith26]

Hi Unsloth!

I came across this interesting model on reddit: https://www.reddit.com/r/LocalLLaMA/comments/1ez8rmu/llama31_just_got_ears_early_experiments/

It allows Text and Audio as input, and outputs Text:

• HF: https://huggingface.co/homebrewltd/llama3.1-s-instruct-v0.2
• Code: https://github.com/homebrewltd/llama3-s
• Blog: https://homebrew.ltd/blog/llama3-just-got-ears
• Demo: https://demo.homebrew.ltd/

Just curious if you think Unsloth would be able to finetune such a model?

Thanks for any help, and this amazing lib!!

[danielhanchen]

Very interesting! Will check this out! We wanted to actually release some code for finetuning multimodal models, but it'll take a bit more time!

[asmith26]

Thanks for the info and help! I'd be happy to help in anyway that I can (I wouldn't want to slow you down though/I probably don't have the required technical knowledge to be of much use, but I'm always keen to learn if you can think of any resources that may help, or equally happy to do tasks like help run testing/experiments if helpful).

Thanks again! :)

[danielhanchen]

I'll definitely ask for help once i get to it!