Open charlesfrye opened 4 years ago
It also doesn't help that there are two copies of the data: one normalized and one not normalized.
Moving the normalization into the data-loading or into the model (as is standard Keras practice these days, to prevent GPUs from getting starved) would help.
On my reasonably-equipped home cpu-machine and on the (cpu) hub, the transfer-learning example causes OOM errors -- sometimes even before getting to the
model.fit
call.They're pretty scary-looking, if you're not expecting them, and depending on the exact system parameters, they can sometimes be triggered inside of the
wandb.init
call, making it look like our fault.Potential solutions: