Closed benjamin-cates closed 1 year ago
Currently affected config variables:
sample_rate
n_mels
n_fft
chunk_length_s
max_offset
Changing these in a sweep when jobs!=0 will not actually vary these variables 😭
Locking when generating the singleton class may fix the issue
Through working with the chunking sweep, I have noticed the dataloaders have the wrong config arguments when jobs is not set to zero. This is because when it spawns a new job it regenerates the config variable and does not inherit variables from the wandb sweep. For example, changing
chunk_length_s
in the wandb sweep does not work because the dataloader workers use the variables fromconfig.yml
, not from the sweep.Proposed solution is pasted below:
The problem with this is that
wandb.init()
prints out the whole "logged in as..." statement and it will do this every time we start a new epoch.Any other suggestions, like somehow passing the config into the dataloaders (not sure if it's possible), would be appreciated.