I want to use ur model on BRATS dataset

[Ssica]

Hi, I've been working on implementing a FCN and use it on the BRATS data set, and I'd like to use your implementation of resnet and see how it works. I was hoping I could get some help for some questions to train. The data set of BRATS consists of:

• (155x240x240)x4. (155x240x240) is the image size and x4 is the modalities (instead of the normal 3 RGB channels).
• Groundtruth: segmented image with dimension: 155x240x240.

My question is how do I adapt your Resnet50 FCN implementation for my data?

I've read: https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf , and as I've understood I do only need to upsample the output, but I'm uncertain if this is correct.

Best Regards, Ssica

[aurora95]

I'm not familiar with this dataset, is it a 3D segmentation dataset? If so I think you cannot directly use current models. Or if you can do segmentation for each 240x240x4 image, maybe ResNet50 is ok (I'm not sure if it's good to use imagenet-pretrained weights), but you need to adapt many things. The data generater must be changed to deal with your 4 channel data, and you may need to change some parameters like class number, and if you don't need to ignore any label, you should change the loss function.

[Ssica]

I'm planning to run it on the axial plane, thus the input is 2D where I would upsample the X-space from 155 to 240. I'm planning to train the weights from scratch, thus I was thinking I could simply just change the input channel numbers to 4 in the model, and just run the training.

The current model does not seem to use labels when training, since it uses pre-trained weights, what would I need to change to add this to the architecture if any (mostly looking at the classification layer)? And am I overlooking some other things which I need to change before I can train (overlooking hyperparameters, loss functions etc)

EDIT: grammar

[Ssica]

Hi again,

I ran into an error when using your Bilinear 2D Upsampling layer, and was hoping if you could point me towards the right direction.

ValueError: Error when checking target: expected bilinear_up_sampling2d_1 to have shape (None, 8, 8, 5) but got array with shape (4800, 240, 240, 1)

Ive tried to go through the code, but since it seems that the shapes are mostly calculated dynamically I can't see what caused the error.

[Ssica]

After looking further into the problem, it seems that there is not a problem with your Bilinear 2D Upsampling layer implementation, though I still get the error and am not sure what is going wrong, it might just be something very trivial/basic.

Basically my network is expecting my last layer to receive (None, 8, 8, 5) but it gets (4800, 240, 240, 1) which is my amount of X training data. Any help would be gladly appreciated

[ahundt]

@Ssica do you have a fork with actual code you can link to?

[Ssica]

No unfortunately not but I can make one, I found out it was because I wanted to use another loss function which expected an incorrect output, so I've settled with using the same loss function as used in this repo/project. If you are interested in how it performs on the BRATS dataset I'll gladly make a github repo with it and post the results, though it seems that the training overfits and I'm thinking of using dropout

[ahundt]

@Ssica also be sure to read the journal paper on the same topic https://arxiv.org/abs/1605.06211, it contains a lot of additional useful information, and there is another useful paper on dilated convolutions https://arxiv.org/abs/1511.07122