Hello. Could I ask you whether the training and prediction of the "CREMI" data set work as expected or there is some problem with that? I used that data set for training using a very small learning rate value to be sure that the training went ok. My training was terminated automatically at the 158th epoch, however, the maximum value were set to be equal 1000. I didn't use either "early_abort_epoch" or "early_abort_loss" values while performing the training. Literally, I followed the example.
However, after the training I ran the prediction mode and the result of that was just a bunch of hdf5 files filled with zeros. All my predictions are just fully black images. Sometimes I can get the results with just 3 or 5 white points within the entire volume and that's all.
Could you please give me some piece of advice what can be wrong?
aschampion
commented
7 years ago
Hello. Could I ask you whether the training and prediction of the "CREMI" data set work as expected or there is some problem with that? I used that data set for training using a very small learning rate value to be sure that the training went ok. My training was terminated automatically at the 158th epoch, however, the maximum value were set to be equal 1000. I didn't use either "early_abort_epoch" or "early_abort_loss" values while performing the training. Literally, I followed the example.
However, after the training I ran the prediction mode and the result of that was just a bunch of hdf5 files filled with zeros. All my predictions are just fully black images. Sometimes I can get the results with just 3 or 5 white points within the entire volume and that's all.
Could you please give me some piece of advice what can be wrong?
Thanks in advance.