search

qubvel / segmentation_models

Segmentation models with pretrained backbones. Keras and TensorFlow Keras.
MIT License
4.7k stars 1.03k forks source link

hyper parameter tuning #320

Open sadransh opened 4 years ago

sadransh commented 4 years ago

Hi, I'm totally new to these kinds of networks.

Can anybody help me with hyperparameter tuning for a specific architecture(e.g. unet) ? for example changing activation functions, num of filters padding or ...? Where Can I start doing so? Am I thinking about it correctly? Since I have very low mIoU and I think the problem is that the hyperparameters should tune.

maybe performing Bayesian optimization on hyperparameter?

JordanMakesMaps commented 4 years ago

What kind of networks are you familiar with?

I would start with using a generic U-Net architecture with any encoder (I find that the EfficientNet do quite well), with pre-trained ImageNet weights for the encoder.

The activation function really depends on if you're doing a binary (sigmoid) or multi-class (softmax) classification problem.

Don't worry about the number of filters or padding so much, those are really specific parameters that you should look into changing after you've investigated other hyperparameters. For example, I find that freezing the encoder does wonders, and turning batch_normalization to False is also useful.

If you do end up doing Bayesian optimization, others that see this post (and myself personally) might like to know how you did so please share your results :)

sadransh commented 4 years ago

@JordanMakesMaps Thanks for suggestions, I have tested my dataset with Keras-segmentation so far. The mIoU is around 0.30 on testset with different structures and backbones and 5 epochs of training. I am going to start with your suggestions and then try to see what happens with scikit-opt. I am not sure whether is it possible to use that Bayesian optimization on this implementation.