search

bytedance / SALMONN

SALMONN: Speech Audio Language Music Open Neural Network
https://bytedance.github.io/SALMONN/
Apache License 2.0
978 stars 75 forks source link

3 stage training #55

Closed signal1234 closed 1 month ago

signal1234 commented 1 month ago

README says, use python3 train.py --cfg-path configs/config.yaml to train the Salmonn model. How to specify for different stages ?
What does stage1 training means ? Just use salmonn_stage1_data.json for training ? Any other parameter to specify ? Also, do you have train/valid/split data for each stage ?

TCL606 commented 1 month ago

README says, use python3 train.py --cfg-path configs/config.yaml to train the Salmonn model. How to specify for different stages ? What does stage1 training means ? Just use salmonn_stage1_data.json for training ? Any other parameter to specify ? Also, do you have train/valid/split data for each stage ?

For different stages, it's just different data for training. Training data is the json files we released, and valid data is the combination of valid set of commonly used dataset such as LibriSpeech, AudioCaps and GigaSpeech.

signal1234 commented 1 month ago

Thank you. Will try.

signal1234 commented 1 month ago

@TCL606 May I ask you, is it possible to train the model using 4xA40 GPU , each has 46 GB memory ?

TCL606 commented 1 month ago

I have a feeling this might be a little difficult, but you can give it a try for the 7b model.