Open ztosi opened 2 years ago
Hey sorry for getting back to you so late, I don't think I got a notification.
Good catch, I think the cleanest fix would probably be:
for batch_idx, (inputs, targets) in enumerate(test_loader, 1):
You can open a PR if you want :smile:
Hi there, nice project! I've actually been using it as a base for some of my own work on scaling CDN on HPC systems. In doing that I noticed a bug, though:
https://github.com/sash-a/CoDeepNEAT/blob/3476078a48986107ea1fc1a7ab1b42a55ac4f62f/src/phenotype/neural_network/evaluator/evaluator.py#L117
In test_nn where you calculate accuracy, you set the counter used as the denominator equal to the batch_idx, which undercounts the number of batches by one. In the limit, one can imagine a case where the entire dataset is a single batch and in that case "count" would be 0.
I was using a fairly large batch size and getting really inflated accuracy numbers.