dataset separation

[balenko1992]

hi we have some problems about the training and the dataset. on each epochs we have to train all the songs (we select 100 songs). but the limit that we can achive with a gpu with 4Gb of memory is 40 songs.

the parameter which select the number of wav file trained by the network is NUM_WAVFILE in config.py in class TrainConfig.

do you use only one song on each epoch? how can we use a big dataset for training on each epoch?

[andabi]

@balenko1992 hi. I don't load all the dataset to gpu at a time. Instead, I sample a proper number of wavfiles for each step. one epoch is done by repeating several steps. this is so called stochastic gradient decent(SGD) method. NUM_WAVFILE in code means the number of wavs for each step. you can set NUM_WAVFILE=10 and repeat 10 steps to execute 1 epoch. The code is already done like what I say.

[balenko1992]

how can i know when one epoch is passed?

[andabi]

@balenko1992 because I consider the training as step-based, epoch is not much meaningful. but if the number of data is 100 and NUM_WAVFILE=10, one epoch would be 100/10 = 10 steps.