loss becomes 'nan' after few iterations

[gunjannaik]

In the BRATS 2013 dataset, the class label 0 patches are much more than other class patches. So, even after rotation operation, it is not working properly. But, right now after some iterations. I am getting the loss as 'nan' and final output is getting degraded. How to solve this issue?

[dudu114]

@furio I have been trying to run the code recently, and I have been able to get the training model, but there are a few problems I haven't been able to solve. 1、the training accuracy that i have gotten is very low. if you have run the code completely,and whether your training and test results are as high as the accuracy of the article. 2、The backend that i used of the keras is theano,i find that the n4_bias_correction.py,the brain_pipline.py and the patch_library.py can't run successfully after installing the theano.I also try to use tensorflow as backend,but i find it only can be used in puyhon3.5,but i am using python2.7,and I find the code is writed with python2.7.So what are you using as backend,and whether you have same problems with me and how did you solve it? 3、the testing data of BRATS2013 only have four files of .mha,it doesn't have 'more',so i can't convert the .mha to .png by brain_pipline.py,also the image_png_converter.py can't convert all .mha to .png.How can you get the complete test data ? I'm also trying to look for reasons,but i can't find it. Sincerely hope that you can help me solve these problems. Thanks

[Cesarec88]

@gunjannaik you can reduce the number of plain patches by changing the argwhere function at line 174 of patch_library.py by changing: '

if patch.shape != (4, h, w) or len(np.argwhere(patch == 0)) > (2 * h * w):
                    if class_num == 0 and patch.shape == (4, h, w):
                        pass
                else:
                    continue

' with: '

if patch.shape != (4, h, w) or len(np.argwhere(patch == 0)) > (2* h * w):
                    if class_num == 0 and patch.shape == (4, h, w) and len(np.argwhere(patch == 0)) < (3 * h * w):
                        pass
                else:
                    continue

'

Up to you to change how many plain patches to obtain in this way

[furio]

@dudu114 Here some answers from me and @Cesarec88

1. to have a more accurate result we where using an amount of at least 100'000 patches with respective augmentation
2. the code has been written in order to run for python 2.7 with Keras using Tensorflow be
3. the 'more'.mha file are nothing but the labels required for the segmentation. at the end of the process, if you have it in another name it's ok anyway is important only that is in the mha format. (In theory is not mandatory to the purpose of learning of neural network, you can try to remove it but then you have to fix all code in order to match al shape image type... you can occur in confusion, I highly discourage it)