search

mindspore-ai / mindspore

MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.
https://gitee.com/mindspore/mindspore
Apache License 2.0
4.32k stars 713 forks source link

An example of transfer learning #51

Open stereomatchingkiss opened 4 years ago

stereomatchingkiss commented 4 years ago

Trying to perform transfer learning by MobileNet, but cannot find an example to show us how to freeze/set the learning rate of each layer. Any example about this? Thanks

DouSong commented 4 years ago

A simple demo as follows：

net = resnet()
conv1 = list(filter(lambda x: 'conv1' in x.name, net.trainable_params()))
conv2 = list(filter(lambda x: 'conv2' in x.name, net.trainable_params()))
group_params = [{'params': conv1, 'lr': 0.1},{'params': conv2, 'lr': 0.2}] # setting different learning rate for each layer
optimizer = Momentum(params=group_params, learning_rate=0.9, momentum=0.9)

Other examle: API: https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html?highlight=adam#mindspore.nn.Adam UT: https://gitee.com/mindspore/mindspore/blob/master/tests/ut/python/optimizer/test_optimizer_with_parameter_groups.py