search

sander-wood / tunesformer

TunesFormer: Forming Irish Tunes with Control Codes by Bar Patching [HCMIR 2023]
MIT License
41 stars 6 forks source link

Question: batch size #2

Open stg1205 opened 11 months ago

stg1205 commented 11 months ago

Hi

Nice work! I would say it's the only model that can generate fluent melody from scratch so far. Got a question:

During training, why do you set the batch size to 1? I saw the squeeze(0) function in the forward process, is that due to some particular cases?

sander-wood commented 11 months ago

Thank you for your interest! The batch size is set to 1 due to complexities with the dual-decoder architecture that make larger batch sizes difficult to implement.

stg1205 commented 7 months ago

Is this problem related to the input dimension of gpt2? Can we just flatten the first two dimensions, batch_size, seq_len, of the input to the char decoder?