search

YuanGongND / ltu

Code, Dataset, and Pretrained Models for Audio and Speech Large Language Model "Listen, Think, and Understand".
389 stars 36 forks source link

question on cutoff_len #32

Open BenoitWang opened 7 months ago

BenoitWang commented 7 months ago

Hi, first thanks for this awesome work. I'm trying to rewrite the training code for ltu-as while I find that the cutoff_len for stage 1 and 2 is 108 which is rather small. There are still many samples that exceed this length and in this case the outputs are cut off. Is this because of a memory issue please?

YuanGongND commented 7 months ago

Yes, it is mainly due to computational efficiency consideration.

More specifically, we see a chance to fit a batch size, say 4, to our GPU, and then with this batch size, we find the max seq length, which is something around 108.

The model will see the same data with larger cutoff_len in later stages because we still use these data in later stages.

-Yuan