Neuroglycerin / neukrill-net-tools

Tools coded as part of the NDSB competition.
MIT License
0 stars 0 forks source link

Online augmentation support for check_test and test #124

Closed gngdb closed 9 years ago

gngdb commented 9 years ago

Currently, all the models we've been training have, for some reason, been using the 4D representation (channels, width,height,examples) for minibatches, in the YAML file. When writing the online augmentation I "fixed" this so that it would be the other way because it's more memory efficient : (examples,width,height,channels).

Unfortunately, the check_test.pu and test.py scripts aren't very smart and can't deal with the change. We need to have a check to see which way it is (don't want to break old models) and then run the forward pass with or without the transpose. The model file that's loaded should have a representation stored internally we can just pull out and check.

Also, we need to decide how many augmentations to average predictions over; or to have an option to not apply augmentations. So there are three changes:

gngdb commented 9 years ago

Oh yeah, and of course they both assume you're wanting to use just the traditional DensePNGDataset. So also:

gngdb commented 9 years ago

Now dataset in check_test_score is loaded from the YAML with some clever parsing, should support both datasets. Tested with dense and it works fine.

gngdb commented 9 years ago

The above features are now implemented for check_test_score.py but not test.py, but it should be easy to copy most of the code across.

gngdb commented 9 years ago

There was a bug in check_test_score.py for a while there. Now fixed, but can no longer reproduce the score of 0.75 seen during model training.

gngdb commented 9 years ago

test.py now supports generating models that use ListDataset (online augmented) and models that use ParallelDataset. Both dataset files had to be updated for this to work.