Problem about the process of generate

jason9693 / MusicTransformer-tensorflow2.0

implementation of music transformer with tensorflow-2.0 (ICLR2019)

MIT License

352 stars 81 forks source link

Problem about the process of generate #2

Closed ujscjj closed 5 years ago

ujscjj commented 5 years ago

Hello. Thank you for your contributions about these code. After I finished training the model, the loss decreases normly. But when I generate, as your suggestion, the model tends to output a series of identical numbers. I am confused about this. Could please tell me something about this or upload a pretrained model. Thank you very much

jason9693 commented 5 years ago

@ujscjj Thank you for interest to my repository. Unfortunately, I'm also suffered from this issue. So now I've some experiments & debugging to fix this both model and pre-processing wise.

please wait for a second. Or,If you have some idea about this issue, I'm very thankful that you report to this repo's issue.

Tank you.

Jackaryyy commented 5 years ago

hello, Have you tried the Hyper Parameters you set on your code? Does it work? I tried to increase h but failed because out of memory. Then I reduced max sequence length but the model didn't work well.

jason9693 commented 5 years ago

@Jackaryyy I think you should both set small batch size to this model and GPU with at least V100. I trained on this env, and worked well. thank you.

mzliang-annie commented 4 years ago

I am still having the same problem. Is this problem fixed?