Typo in dropout line?

[jessetrana]

https://github.com/GantMan/nsfw_model/blob/80a60e3103f325bf0c42d5b3aa4b63104e7f8e3f/training/mobilenetv2_transfer/train_initialization.py#L46

I think there is accidentally a "-" instead of a "=", which could be affecting the intended regularization.

[GantMan]

Good catch, would you like to submit a PR?

[jessetrana]

What was the original intent? I'm assuming the "-" is effectively evaluating rather than assigning and causing the line of dropout to basically disappear. "Reenabling" this line seems like it might cause more regularization than you intended. If regularization is intended in both locations, it seems like a SpatialDropout2D might be a better fit while perhaps reducing the rate in both places. My limited experience has led me to something like this for use in my own finetuning: x = model_mn.output #basically MobileNetV2 without the top x = SpatialDropout2D(0.1)(x) x = GlobalAveragePooling2D()(x) x = Dense(128, name='fc_final', activation='relu')(x) x = Dropout(0.25, name='dropout')(x) x = Dense(NUM_CLASSES, name='classifier', activation='softmax')(x)

I see slightly better accuracy with this approach than my prior approach of no spatial dropout and setting just the one dropout to 0.45. But of course, everybody's use case is a little different and in particular with your dataset being as large as it is, I'm not sure what the best route is. (My current dataset is about 100K with four classes but some oversampling due to class imbalance, as a reference point.) What variant do you think will work best for your situation? (Also - I'm quite hesitant to submit a PR without running it myself, and do not currently have a great system to test this large of a dataset on.)

[GantMan]

You're right. I tried training with the dropout returning, and it destroyed training performance.

[jessetrana]

29 Here's a PR for nixing the bad dropout line.

[jessetrana]

This got merged a long time ago, closing!