Can you recomment which layer to freeze?

[sanghyun0927]

Hi, plemeri

I'm trying transfer learing from ckpt_base(Plus_Ultra) with my custom dataset

My dataset include about 100 samples until now, planned to be 10,000

1. To avoid overfitting, Can you recommend whith layer to be frozen?
2. It would be nice to have some additional advice on hyper-parameters, but think it's better to do it after learning

[plemeri]

You can freeze backbone network if there are only 100 samples, while you don't need to freeze any other layers if you have more than 10000 samples. Freezing layers is especially unnecessary if they are domain specific, such as human only samples.

Also, try using lower learning rate, different settings for scheduler might be useful.

Thanks.

[sanghyun0927]

Got it. Thanks.