Closed Drea1989 closed 6 years ago
I use the following "pseudo" code to convert old weights to new weights.
layers = [....] # Conversion routine would require the same layers used to create the old model
model = Model(layers=layers)
model.load_params(old_weight_file, load_state=False/True) load_state=False if no states information are in the old_weight_file, True otherwise
model.save_params(new_weight_file)
Take VGG for example:
[omitting unrelated code ...] Full files can be found at https://github.com/NervanaSystems/ModelZoo/blob/VGG_neon2.3/ImageClassification/ILSVRC2012/VGG/vgg_neon_train.py
54 # Set up the model layers 55 layers = [] 56 57 # set up 3x3 conv stacks with different feature map sizes 58 for nofm in [64, 128, 256, 512, 512]: 59 layers.append(Conv((3, 3, nofm), conv_params)) 60 layers.append(Conv((3, 3, nofm), conv_params)) 61 if nofm > 128: 62 layers.append(Conv((3, 3, nofm), conv_params)) 63 if args.vgg_version == 'E': 64 layers.append(Conv((3, 3, nofm), conv_params)) 65 layers.append(Pooling(2, strides=2)) 66 67 layers.append(Affine(nout=4096, init=initfc, bias=Constant(0), activation=relu)) 68 layers.append(Dropout(keep=0.5)) 69 layers.append(Affine(nout=4096, init=initfc, bias=Constant(0), activation=relu)) 70 layers.append(Dropout(keep=0.5)) 71 layers.append(Affine(nout=1000, init=initfc, bias=Constant(0), activation=Softmax())) 72 73 cost = GeneralizedCost(costfunc=CrossEntropyMulti()) 74 75 model = Model(layers=layers)
Suppose you have old weights named VGG_D.p, assuming the VGG_D.p model contains the same number of layers as layers variable above, you can print out the VGG_D.p to double check. If VGG_D.p does not contain all the above layers (e.g. it does not contain line 66-71 layers), then please delete line 66071 from the layers.
step 1: load it model.load_params(VGG_D.p) Run the program, if VGG_D.p does not contain states information, it will error out and ask you to change the above to: model.load_params(VGG_D.p, load_state=False)
step 2: save it (in new weight format)
model.save_params(VGG_D_fused_conv_bias.p)
Done.
Please let me know if you are able to convert. If you prefer, you can send me the weight file and the script used to generate the weight file, I would be able to convert it for you. Thank you!
Oh, I see. It is DS2 pre-trained model https://s3-us-west-1.amazonaws.com/nervana-modelzoo/Deep_Speech/Librispeech/librispeech_16_epochs.prm Will work on converting that.
hello, thanks for the help, i was able to load and save the new model by printing out the model architecture and verifying that the structure of the layers where equal. all good now i was able to train and infer.
OK, thanks for the update and I am glad that you have got the new model weights.
For me I had to do the following extra in neon/layers/recurrent.py to convert ds2 model:
if getattr(self, key) is not None:
serial_dict['params'][key] = getattr(self, key).get()
else:
serial_dict['params'][key] = None
Closing this issue. Feel free to open new issues.
hello, i have tried to load my model trained with neon 2.0 as well as the pretrained model available in the modelzoo returning the same error, only if i train a new model in neon 2.3.0 i am also able to load it. any conversion tool planned?