search

bytedance / SALMONN

SALMONN: Speech Audio Language Music Open Neural Network
https://bytedance.github.io/SALMONN/
Apache License 2.0
978 stars 75 forks source link

The role of prompt_pattern parameter #8

Closed 980202006 closed 10 months ago

980202006 commented 10 months ago

I noticed that there is this parameter in your code: prompt_pattern. For music, do I need to modify it? Can you briefly talk about the process of training this model and the data set used?

TCL606 commented 10 months ago

You dont need to modify the prompt_pattern as this is the format used for all tasks during training.

For the training process, we first trained SALMONN using large amount of speech recognition and audio captioning data, then applied multi-task instruction finetuning on it. All the data for training is open source. We are planning to release the paper of SALMONN very soon, so stay tuned ~

980202006 commented 10 months ago

Thank you for your reply. This is a great work.