Does VNet support to segment more than 2 classes?

[John1231983]

Hello, I am grateful to your code. I am working in brain segmentation which has 4 classes: WM, GM, CSF, and background. You are using DICE as loss function to maximize. It is very good idea but it may be only work in 2 classes. How can I apply your VNet in brain segmentation? Which layers do I need change the number of output? (May be will will not use Dice layer because it only support two classes). Thank you in advance

[gattia]

I haven't been using this library/code but have been working on creating a network based on v-net as well as u-net..

The U-net example doesnt use a softmax like v-net does. It instead uses a sigmoid activation and produces a single volume and dice is computed for that single volume (instead of the 2 volumes from the softmax in v-net).

I am currently playing with a binary segmentation (same as v-net does) and want to expand to multiple labels as you are asking about. I've been using the sigmoid activation and am assuming that it would be possible to output a labelmap for each region of interest (0 = background, 1=structure of interest).. well I know it would, you would just have to change the convolution at the end to have shape n,1,1,1 where n = number of labels of interest. I would assume you could calculate a dice coefficient for each individual labelmap and you could average those to get your error term.

[faustomilletari]

Hello,

I think I recall that at least 2d unet was using softmax and cross entropy loss.

Can you clarify?

I was also thinking that we could use the vnet code to run 3D unet as the authors of 3D unet didn't really released code that is ready to run.

Fausto Milletarì Sent from my iPhone

On 16 Feb 2017, at 15:23, Anthony notifications@github.com wrote:

I haven't been using this library/code but have been working on creating a network based on v-net as well as u-net..

The U-net example doesnt use a softmax like v-net does. It instead uses a sigmoid activation and produces a single volume and dice is computed for that single volume (instead of the 2 volumes from the softmax in v-net).

I am currently playing with a binary segmentation (same as v-net does) and want to expand to multiple labels as you are asking about. I've been using the sigmoid activation and am assuming that it would be possible to output a labelmap for each region of interest (0 = background, 1=structure of interest).. well I know it would, you would just have to change the convolution at the end to have shape n,1,1,1 where n = number of labels of interest. I would assume you could calculate a dice coefficient for each individual labelmap and you could average those to get your error term.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[gattia]

Hi,

Im not positive what the u-net original was using. I am basing this off of a re-construction of u-net by @jocicmarko and am using Keras to do so. You can check out my repo, vnet-implementation. It is somewhat a mess as it was not intended to be shared. This uses Keras to implement something that is close to v-net. I am using a sigmoid activation at the end, the same way that the 2d reproduction of u-net that I referenced does. This requires keras as well as keras_contrib because the deconvolution3d is not yet part of the main keras library, I beleive it can use either theano or tensorflow as the backend. I am using tensorflow.

[jocicmarko]

@gattia You can use Upsampling3D and then Convolution3D instead of Deconvolution3D. There are some indications that these work similarly.

[John1231983]

Hi all, Could we use Dice loss for 4 classes. For example, brain image: CSF, WM, GM and Background. I am looking for the way to use Dice loss of VNet for brain subject

[faustomilletari]

Hello,

recently another group has been working with dice for multi class segmentation problems with v-net and re-implemented in tensor flow.

this code can be found here: http://niftynet.io http://niftynet.io/

I didn’t check their code yet, but it should be good enough.

Fausto

On Jul 5, 2017, at 10:29 AM, John1231983 notifications@github.com wrote:

Hi all, Could we use Dice loss for 4 classes. For example, brain image: CSF, WM, GM and Background. I am looking for the way to use Dice loss of VNet for brain subject

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/faustomilletari/VNet/issues/15#issuecomment-313120117, or mute the thread https://github.com/notifications/unsubscribe-auth/AMtsvj9-M_7SnFB0ZhtQSmI-jX7plI3wks5sK520gaJpZM4MCvKH.

[John1231983]

Great reference. I just quickly looked at it. It may use automatically way to compute backward for Dice. I think it is great to point of tensorflow in comparison with caffe. Do you think the automatic calculation step size (from backward) in tensorflow for is correct for dice loss case?

[faustomilletari]

I have tried tensor flow with dice loss for two classes and i can confirm you that it works pretty well. indeed, there is no way it can be wrong :)

I think that implementation is very good. you can try that and also use the generalized wasserstein dice score for multi class.

Fausto

On Jul 5, 2017, at 11:21 AM, John1231983 notifications@github.com wrote:

Great reference. I just quickly looked at it. It may use automatically way to compute backward for Dice. I think it is great to point of tensorflow in comparison with caffe. Do you think the automatic calculation step size (from backward) in tensorflow for is correct for dice loss case?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/faustomilletari/VNet/issues/15#issuecomment-313135807, or mute the thread https://github.com/notifications/unsubscribe-auth/AMtsvmlYOXgeeIAva7_4TBfFLvTz6OQFks5sK6nzgaJpZM4MCvKH.

[John1231983]

Thanks for your suggestion. However, I am new in tensorflow and I would like to ask you one more question. It may not related to the thread. However, it related with your experiments. As you know, a layer includes forward and backward. For example, your Dice layer includes forward and backward and they implemented as a customer python layer. However, when I look at the tensorflow loss layer, they just define forward (dice formulation), I did not see any code for backward. Does tensorflow support automatic way to compute it?

This is his dice loss.

def dice(pred, labels):
    n_voxels = labels.get_shape()[0].value
    n_classes = pred.get_shape()[1].value
    pred = tf.nn.softmax(pred)
    # construct sparse matrix for labels to save space
    ids = tf.constant(np.arange(n_voxels), dtype=tf.int64)
    ids = tf.stack([ids, labels], axis=1)
    one_hot = tf.SparseTensor(indices=ids,
                              values=[1.0] * n_voxels,
                              dense_shape=[n_voxels, n_classes])
    # dice
    dice_numerator = 2.0 * tf.sparse_reduce_sum(one_hot * pred,
                                                reduction_axes=[0])
    dice_denominator = (tf.reduce_sum(tf.square(pred), reduction_indices=[0]) +
                        tf.sparse_reduce_sum(one_hot, reduction_axes=[0]))
    epsilon_denominator = 0.00001

    dice_score = dice_numerator / (dice_denominator + epsilon_denominator)
    dice_score.set_shape([n_classes])
    # minimising (1 - dice_coefficients)
    return 1.0 - tf.reduce_mean(dice_score)

[faustomilletari]

Yes, tensor flow has autodifferentiation. Dice is defined as a combination of different operators and variables for which derivative is already done by tensor flow team. When you execute that code everything gets represented as a graph of operations for which all the derivatives are defined when the data "flows" backward.

It's nice. :)

Fausto Milletarì Sent from my iPhone

On 5. Jul 2017, at 11:34, John1231983 notifications@github.com wrote:

Thanks for your suggestion. However, I am new in tensorflow and I would like to ask you one more question. It may not related to the thread. However, it related with your experiments. As you know, a layer includes forward and backward. For example, your Dice layer includes forward and backward and they implemented as a customer python layer. However, when I look at the tensorflow loss layer, they just define forward (dice formulation), I did not see any code for backward. Does tensorflow support automatic way to compute it?

This is his dice loss.

def dice(pred, labels): n_voxels = labels.get_shape()[0].value n_classes = pred.get_shape()[1].value pred = tf.nn.softmax(pred)

construct sparse matrix for labels to save space

ids = tf.constant(np.arange(n_voxels), dtype=tf.int64)
ids = tf.stack([ids, labels], axis=1)
one_hot = tf.SparseTensor(indices=ids,
                          values=[1.0] * n_voxels,
                          dense_shape=[n_voxels, n_classes])
# dice
dice_numerator = 2.0 * tf.sparse_reduce_sum(one_hot * pred,
                                            reduction_axes=[0])
dice_denominator = (tf.reduce_sum(tf.square(pred), reduction_indices=[0]) +
                    tf.sparse_reduce_sum(one_hot, reduction_axes=[0]))
epsilon_denominator = 0.00001

dice_score = dice_numerator / (dice_denominator + epsilon_denominator)
dice_score.set_shape([n_classes])
# minimising (1 - dice_coefficients)
return 1.0 - tf.reduce_mean(dice_score)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[PiaoLiangHXD]

@gattia I could not find your v-net keras implementation now. As keras2 has updated, have you tried to continue your implement job?