CNN 모델 개선

[choidam]

1. Weight Initialization Xavier, He Initialization 사용해서 가중치 초기화하기

2. Batch Normalization

3. Dropout

[choidam]

Weight Initialization

he

• 공식 문서 https://www.tensorflow.org/api_docs/python/tf/keras/initializers

tf.keras.initializers.he_normal(
    seed=None
)

• model 에 추가하기 https://buomsoo-kim.github.io/keras/2018/04/22/Easy-deep-learning-with-Keras-4.md/

def mlp_model():
    model = Sequential()

    model.add(Dense(50, input_shape = (784, ), kernel_initializer='he_normal'))     # use he_normal initializer
    model.add(Activation('sigmoid'))    
    model.add(Dense(50, kernel_initializer='he_normal'))                            # use he_normal initializer
    model.add(Activation('sigmoid'))    
    model.add(Dense(50, kernel_initializer='he_normal'))                            # use he_normal initializer
    model.add(Activation('sigmoid'))    
    model.add(Dense(50, kernel_initializer='he_normal'))                            # use he_normal initializer
    model.add(Activation('sigmoid'))    
    model.add(Dense(10, kernel_initializer='he_normal'))                            # use he_normal initializer
    model.add(Activation('softmax'))

    sgd = optimizers.SGD(lr = 0.001)
    model.compile(optimizer = sgd, loss = 'categorical_crossentropy', metrics = ['accuracy'])

    return model

위의 예시는 일반 Dense Layer 에만 적용되는듯 .. CNN 에 적용은 좀 더 찾아보자.

[choidam]

🌟정독하기🌟

CNN 모델 개선하기

최고다 최고 .. 👍

https://buomsoo-kim.github.io/keras/2018/05/05/Easy-deep-learning-with-Keras-11.md/

https://github.com/buomsoo-kim/Easy-deep-learning-with-Keras/blob/master/2.%20CNN/2-Advanced-CNN/2-advanced-cnn.ipynb

[choidam]

코드엔 추가 해보았다.

동일한 라이브러리에서 제대로 추가하지 않으면 오류 발생. 유의하자 ..

from tensorflow.keras.layers import Conv2D, MaxPooling2D, ReLU, BatchNormalization, Dropout
from tensorflow.keras import optimizers
from keras.optimizers import adam

model2 = Sequential()

model2.add(Conv2D(50, (3, 3),
                 padding='same',
                 input_shape=X_train.shape[1:],
                 kernel_initializer='he_normal'))
model2.add(BatchNormalization())
model2.add(Activation('relu'))
model2.add(MaxPooling2D(pool_size = (2,2)))

model2.add(Conv2D(filters = 50, 
                  kernel_size = (3,3), 
                  strides = (1,1), 
                  padding = 'same', 
                  kernel_initializer='he_normal'))
model2.add(BatchNormalization())
model2.add(Activation('relu'))
model2.add(MaxPooling2D(pool_size = (2,2)))

model2.add(Flatten())

model2.add(Dense(50, activation = 'relu', kernel_initializer='he_normal'))
model2.add(Dropout(0.5))
model2.add(Dense(10, activation = 'softmax', kernel_initializer='he_normal'))

adam = optimizers.Adam(lr = 0.001)
model2.compile(loss = 'categorical_crossentropy', 
               optimizer = adam, 
               metrics = ['accuracy'])

BatchNormalization, he, dropout, relu, adam 모두 적용해보았으나 성능이 얼마나 좋아졌는지는 알 수가 없다 .. 파라미터 개수가 엄청 많아져서 오히려 떨어졌을 것 같기도 .. 일단 빨리 실행해야 알 듯

[choidam]

https://github.com/GRuuuuu/GRuuuuu.github.io/blob/master/assets/resources/machine-learning/CNN-CIFAR10/cifar10%20dropout%20test%20notebook-s.ipynb

[choidam]

https://becominghuman.ai/understanding-the-basics-of-cnn-with-image-classification-7f3a9ddea8f9

[choidam]

bayesian optimization

https://medium.com/@crawftv/parameter-hyperparameter-tuning-with-bayesian-optimization-7acf42d348e1

https://www.dlology.com/blog/how-to-do-hyperparameter-search-with-baysian-optimization-for-keras-model/