user story: llama3-s can handle real life audio

[0xSage]

Goal

In real life, what are the challenges we anticipate? e.g.

• model struggles with detecting pauses in speech
• model struggles with imperfect natural speech, i.e. filler words "um"
• model struggles with noisier environments
• model struggles with multi speaker
• double compression losses from airpods, other headsets

What are user preferences on model responses?

• More concise responses
• Multiturn >> single turn, i.e. users want the model to ask follow up questions

What are the use cases?

• Once we provide a local/speech model, what are folks using it for?
• Function calling?
• Simple Q&A?
• A bit of an edge case: users input both audio & text in the same message instance

All of the above will likely inform our next data directions. @dan-homebrew

[tikikun]

I will provide a longer writeup soon but one thing is that there are no better ways to answer those questions you mentioned above than just launch it!

• We need more data to know the issues
• We need more data to fix those issues anyways
• We cannot have any data without launching it into an app or something (than our current simple demo)