search

mlyg / unified-focal-loss

Apache License 2.0
149 stars 22 forks source link

not work #10

Closed htx-rest closed 2 years ago

htx-rest commented 2 years ago

Traceback (most recent call last): File "D:/retinal blood vessel segmentation/train_drive.py", line 81, in callbacks=[TensorBoard(log_dir='./autoencoder'), model_checkpoint]) File "D:\Anaconda3\envs\vessel\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "D:\Anaconda3\envs\vessel\lib\site-packages\tensorflow\python\framework\func_graph.py", line 1129, in autograph_handler raise e.ag_error_metadata.to_exception(e) ValueError: in user code:

File "D:\Anaconda3\envs\vessel\lib\site-packages\keras\engine\training.py", line 878, in train_function  *
    return step_function(self, iterator)
File "D:\retinal blood vessel segmentation\sym_unified_focal_loss_unetpp.py", line 102, in loss_function  *
    symmetric_ftl = symmetric_focal_tversky_loss(delta=delta, gamma=gamma)(y_true, y_pred)
File "C:\Users\DELL\AppData\Local\Temp\__autograph_generated_fileaj1cs6rw.py", line 25, in loss_function  **
    fore_dice = ((1 - ag__.ld(dice_class)[:, 1]) * ag__.converted_call(ag__.ld(K).pow, ((1 - ag__.ld(dice_class)[:, 1]), (- ag__.ld(gamma))), None, fscope_1))

ValueError: slice index 1 of dimension 1 out of bounds. for '{{node loss_function/strided_slice_2}} = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=1, ellipsis_mask=0, end_mask=1, new_axis_mask=0, shrink_axis_mask=2](loss_function/truediv, loss_function/strided_slice_2/stack, loss_function/strided_slice_2/stack_1, loss_function/strided_slice_2/stack_2)' with input shapes: [2,1], [2], [2], [2] and with computed input tensors: input[1] = <0 1>, input[2] = <0 2>, input[3] = <1 1>.

Process finished with exit code 1

mlyg commented 2 years ago

Thanks for showing the error code. It is not exactly clear to me what the issue is, but can you confirm that your model has:

  1. Final softmax output (rather than sigmoid)
  2. Final layer output of 2 (rather than 1)
htx-rest commented 2 years ago

from keras.layers import Conv2DTranspose, MaxPooling2D, concatenate from keras.layers import Conv2D from keras import * from tensorflow.keras.optimizers import Adam from tensorflow.keras import backend as K import tensorflow as tf

def identify_axis(shape):

Three dimensional

if len(shape) == 5:
    return [1, 2, 3]
# Two dimensional
elif len(shape) == 4:
    return [1, 2]
# Exception - Unknown
else:
    raise ValueError('Metric: Shape of tensor is neither 2D or 3D.')

def asymmetric_focal_loss(delta=0.7, gamma=2.): def loss_function(y_true, y_pred): axis = identify_axis(y_true.get_shape())

    epsilon = K.epsilon()
    y_pred = K.clip(y_pred, epsilon, 1. - epsilon)
    cross_entropy = -y_true * K.log(y_pred)

    # calculate losses separately for each class, only suppressing background class
    back_ce = K.pow(1 - y_pred[:, :, :, 0], gamma) * cross_entropy[:, :, :, 0]
    back_ce = (1 - delta) * back_ce

    fore_ce = cross_entropy[:, :, :, 1]
    fore_ce = delta * fore_ce

    loss = K.mean(K.sum(tf.stack([back_ce, fore_ce], axis=-1), axis=-1))

    return loss

return loss_function

#################################

Asymmetric Focal Tversky loss

################################# def asymmetric_focal_tversky_loss(delta=0.7, gamma=0.75): """This is the implementation for binary segmentation. Parameters

delta : float, optional
    controls weight given to false positive and false negatives, by default 0.7
gamma : float, optional
    focal parameter controls degree of down-weighting of easy examples, by default 0.75
"""

def loss_function(y_true, y_pred):
    # Clip values to prevent division by zero error
    epsilon = K.epsilon()
    y_pred = K.clip(y_pred, epsilon, 1. - epsilon)

    axis = identify_axis(y_true.get_shape())
    # Calculate true positives (tp), false negatives (fn) and false positives (fp)
    tp = K.sum(y_true * y_pred, axis=axis)
    fn = K.sum(y_true * (1 - y_pred), axis=axis)
    fp = K.sum((1 - y_true) * y_pred, axis=axis)
    dice_class = (tp + epsilon) / (tp + delta * fn + (1 - delta) * fp + epsilon)

    # calculate losses separately for each class, only enhancing foreground class
    back_dice = (1 - dice_class[:, 0])
    fore_dice = (1 - dice_class[:, 1]) * K.pow(1 - dice_class[:, 1], -gamma)

    # Average class scores
    loss = K.mean(tf.stack([back_dice, fore_dice], axis=-1))
    return loss

return loss_function

def asym_unified_focal_loss(weight=0.5, delta=0.6, gamma=0.5): """The Unified Focal loss is a new compound loss function that unifies Dice-based and cross entropy-based loss functions into a single framework. Parameters

weight : float, optional
    represents lambda parameter and controls weight given to asymmetric Focal Tversky loss and asymmetric Focal loss, by default 0.5
delta : float, optional
    controls weight given to each class, by default 0.6
gamma : float, optional
    focal parameter controls the degree of background suppression and foreground enhancement, by default 0.5
"""

def loss_function(y_true, y_pred):
    asymmetric_ftl = asymmetric_focal_tversky_loss(delta=delta, gamma=gamma)(y_true, y_pred)
    asymmetric_fl = asymmetric_focal_loss(delta=delta, gamma=gamma)(y_true, y_pred)
    if weight is not None:
        return (weight * asymmetric_ftl) + ((1 - weight) * asymmetric_fl)
    else:
        return asymmetric_ftl + asymmetric_fl

return loss_function

def sym_unified_focal_loss_unetpp(input_size=(512, 512, 3), base_filter_num=16): inputs = Input(input_size) conv0_0 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(inputs) conv0_0 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv0_0) pool1 = MaxPooling2D(pool_size=(2, 2))(conv0_0)

conv1_0 = Conv2D(base_filter_num * 2, 3, activation='relu', padding='same', kernel_initializer='he_normal')(pool1)
conv1_0 = Conv2D(base_filter_num * 2, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv1_0)
pool2 = MaxPooling2D(pool_size=(2, 2))(conv1_0)

up1_0 = Conv2DTranspose(base_filter_num, (2, 2), strides=(2, 2), padding='same')(conv1_0)
merge00_10 = concatenate([conv0_0, up1_0], axis=-1)
conv0_1 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(merge00_10)
conv0_1 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv0_1)

conv2_0 = Conv2D(base_filter_num * 4, 3, activation='relu', padding='same', kernel_initializer='he_normal')(pool2)
conv2_0 = Conv2D(base_filter_num * 4, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv2_0)
pool3 = MaxPooling2D(pool_size=(2, 2))(conv2_0)

up2_0 = Conv2DTranspose(base_filter_num * 2, (2, 2), strides=(2, 2), padding='same')(conv2_0)
merge10_20 = concatenate([conv1_0, up2_0], axis=-1)
conv1_1 = Conv2D(base_filter_num * 2, 3, activation='relu', padding='same', kernel_initializer='he_normal')(
    merge10_20)
conv1_1 = Conv2D(base_filter_num * 2, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv1_1)

up1_1 = Conv2DTranspose(base_filter_num, (2, 2), strides=(2, 2), padding='same')(conv1_1)
merge01_11 = concatenate([conv0_0, conv0_1, up1_1], axis=-1)
conv0_2 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(merge01_11)
conv0_2 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv0_2)

conv3_0 = Conv2D(base_filter_num * 8, 3, activation='relu', padding='same', kernel_initializer='he_normal')(pool3)
conv3_0 = Conv2D(base_filter_num * 8, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv3_0)
pool4 = MaxPooling2D(pool_size=(2, 2))(conv3_0)

up3_0 = Conv2DTranspose(base_filter_num * 4, (2, 2), strides=(2, 2), padding='same')(conv3_0)
merge20_30 = concatenate([conv2_0, up3_0], axis=-1)
conv2_1 = Conv2D(base_filter_num * 4, 3, activation='relu', padding='same', kernel_initializer='he_normal')(
    merge20_30)
conv2_1 = Conv2D(base_filter_num * 4, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv2_1)

up2_1 = Conv2DTranspose(base_filter_num * 2, (2, 2), strides=(2, 2), padding='same')(conv2_1)
merge11_21 = concatenate([conv1_0, conv1_1, up2_1], axis=-1)
conv1_2 = Conv2D(base_filter_num * 2, 3, activation='relu', padding='same', kernel_initializer='he_normal')(
    merge11_21)
conv1_2 = Conv2D(base_filter_num * 2, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv1_2)

up1_2 = Conv2DTranspose(base_filter_num, (2, 2), strides=(2, 2), padding='same')(conv1_2)
merge02_12 = concatenate([conv0_0, conv0_1, conv0_2, up1_2], axis=-1)
conv0_3 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(merge02_12)
conv0_3 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv0_3)

conv4_0 = Conv2D(base_filter_num * 16, 3, activation='relu', padding='same', kernel_initializer='he_normal')(pool4)
conv4_0 = Conv2D(base_filter_num * 16, 3, activation='relu', padding='same', kernel_initializer='he_normal')(
    conv4_0)

up4_0 = Conv2DTranspose(base_filter_num * 8, (2, 2), strides=(2, 2), padding='same')(conv4_0)
merge30_40 = concatenate([conv3_0, up4_0], axis=-1)
conv3_1 = Conv2D(base_filter_num * 8, 3, activation='relu', padding='same', kernel_initializer='he_normal')(
    merge30_40)
conv3_1 = Conv2D(base_filter_num * 8, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv3_1)

up3_1 = Conv2DTranspose(base_filter_num * 4, (2, 2), strides=(2, 2), padding='same')(conv3_1)
merge21_31 = concatenate([conv2_0, conv2_1, up3_1], axis=-1)
conv2_2 = Conv2D(base_filter_num * 4, 3, activation='relu', padding='same', kernel_initializer='he_normal')(
    merge21_31)
conv2_2 = Conv2D(base_filter_num * 4, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv2_2)

up2_2 = Conv2DTranspose(base_filter_num * 2, (2, 2), strides=(2, 2), padding='same')(conv2_2)
merge12_22 = concatenate([conv1_0, conv1_1, conv1_2, up2_2], axis=-1)
conv1_3 = Conv2D(base_filter_num * 2, 3, activation='relu', padding='same', kernel_initializer='he_normal')(
    merge12_22)
conv1_3 = Conv2D(base_filter_num * 2, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv1_3)

up1_3 = Conv2DTranspose(base_filter_num, (2, 2), strides=(2, 2), padding='same')(conv1_3)
merge03_13 = concatenate([conv0_0, conv0_1, conv0_2, conv0_3, up1_3], axis=-1)
conv0_4 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(merge03_13)
conv0_4 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv0_4)
# 二分类任务
conv0_4 = Conv2D(1, 1, activation='softmax')(conv0_4)

model = Model(inputs=inputs, outputs=conv0_4)
model.compile(optimizer=Adam(lr=1e-3), loss=asym_unified_focal_loss(weight=0.5, delta=0.6, gamma=0.5),
              metrics=['acc'])
return model

------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2022年4月14日(星期四) 下午5:39 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [mlyg/unified-focal-loss] not work (Issue #10)

感谢显示错误代码。我不太清楚问题是什么，但你能证实你的模型有：

最终 soft max 输出（而不是 sigmoid ）

最终层输出为 2 （而不是 1 ）

— 直接回复此邮件，在 GitHub 上查看或取消订阅. 您收到这个消息是因为您编写了这个线程。留言 ID @.***与>

htx-rest commented 2 years ago

this is my code.When I did experiments on DRIVE, it didn't work

mlyg commented 2 years ago

Thanks, I think the issue is with your implementation of the U-Net++ architecture.

For example:

conv0_4 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(merge03_13)
conv0_4 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv0_4)

The input layer to the last conv0_4 is 3, but you have written base_filter_num which is 16.

Once you have fixed the architecture, the final layer to be compatible with our loss function will need to be:

conv0_4 = Conv2D("previous output channels", 2, activation='softmax')(conv0_4)

htx-rest commented 2 years ago

thanks for your help

------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2022年4月14日(星期四) 晚上6:02 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [mlyg/unified-focal-loss] not work (Issue #10)

Thanks, I think the issue is with your implementation of the U-Net++ architecture.

For example: conv0_4 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(merge03_13) conv0_4 = Conv2D(base_filter_num, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv0_4)
The input layer to the last conv0_4 is 3, but you have written base_filter_num which is 16.

Once you have fixed the architecture, the final layer to be compatible with our loss function will need to be:

conv0_4 = Conv2D("previous output channels", 2, activation='softmax')(conv0_4)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>