search

YatingMusic / remi

"Pop Music Transformer: Beat-based Modeling and Generation of Expressive Pop Piano Compositions", ACM Multimedia 2020
GNU General Public License v3.0
553 stars 85 forks source link

Questions about the training data #18

Closed theseventhflow closed 4 years ago

theseventhflow commented 4 years ago

Hello,

I have tried to train transformer-xl from scratch. However, the generating result is not as good as yours. The theme of one generated midi file is not consistent, which means the latest slice of sequence hears different from the prompt. Therefore,could you tell me if the dataset you provide is the whole dataset? or if you used other bigger dataset for pre-training and used the provided one only for finetuning?

My experiment setting is: n_layers: 12 x_len: 512 m_len:512 ff:2048

And could you tell me the testset cross-entropy loss you got in your experiment?

Thank you!

remyhuang commented 4 years ago

Yes, the dataset is the whole dataset I used to train the shared checkpoint. The testset loss is 0.25 in my experiment.

theseventhflow commented 4 years ago

Thanks for your reply! My test loss is 1.24 with the "chord" words. Did you get 0.25 without "chord"?

remyhuang commented 4 years ago

Both shared checkpoint losses are 0.25.

theseventhflow commented 4 years ago

Got it. Thank you!

dedededefo commented 1 year ago

Hello,

I have tried to train transformer-xl from scratch. However, the generating result is not as good as yours. The theme of one generated midi file is not consistent, which means the latest slice of sequence hears different from the prompt. Therefore,could you tell me if the dataset you provide is the whole dataset? or if you used other bigger dataset for pre-training and used the provided one only for finetuning?

My experiment setting is: n_layers: 12 x_len: 512 m_len:512 ff:2048

And could you tell me the testset cross-entropy loss you got in your experiment?

Thank you!

Hello, excuse me, I want to know how you started from scratch