questions of the training details

[xinario]

First of all, very impressive work.

When I was trying to finetune the resnet50 work to reproduce your results, I found that the network tends to overfit the training data very easily and I got very low AP on the test data.

So I'm wondering are there any tricks in the fintuning. e.g.

1. what's the augmented size of your training set?
2. Did you employ any form of early stropping or just trained directly to 10000 iterations.

Really appreciate your help, thanks.

[yulequan]

Hi,

1. We use heavily data augmentation in the classification work. After augmentation, we have about 20K images in total.

2. We do not use early stop. Since the heavily augmented images, it is not very prob to overfitting. By the way, combine the SVM result and softmax result can improve the performance a little.

[xinario]

Thanks so much for the reply.

And the final training set is balanced. 10k for melanoma and 10k for the other?

I did augmentation too, mainly rotation (30 degrees step) and translation. The size I got (16k train and 600 val) is slightly smaller than yours. However, during the training, the training loss keeps decreasing to almost zero but the validation loss just decrease in the first few epochs and then increase after that. In the test, I could barely get half of the melanomas detected. Justing wondering which part might go wrong.

[yulequan]

Yes, I have balanced the melanoma and benign cases. Beside rotation, I have also done scale augmentation, randomly cropping at the training, and randomly add noise.

[xinario]

Ok, I see. You did scale augmentation.

When I read your paper, I thought the way you cope with the scale problem is by scaling all the lesions to 250x250.

I’ll give that a try. Thanks.

On Nov 1, 2017, at 11:17 PM, Lequan Yu notifications@github.com<mailto:notifications@github.com> wrote:

Yes, I have balanced the melanoma and benign cases. Beside rotation, I have also done scale augmentation, randomly cropping at the training, and randomly add noise.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/yulequan/melanoma-recognition/issues/3#issuecomment-341320279, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEucECx6wB0pk9RbiPrkdgDd2Ag9WI5Dks5syVB9gaJpZM4QKAMC.

[xinario]

I augmented the dataset to 20k+20k (640 from 900, I reserved the rest for validation ) and used exactly your code for the training. After 10000 iterations, the AP calculated from sklearn is like 0.34.

I reimplemented in pytorch and with early stopping, I only managed to get the AP to 0.53, which is still far from the one you reported in the paper (0.609 by just using softmax).

Not matter how I change the augmentation (rotation, translation, cropping, noise, scale), or freeze certain layers of resnet50, the network overfits the training data in the first few epochs.

I'm wondering if you could kindly share your augmented dataset so that I could find out the problem.