search

Hughes-Genome-Group / deepHaem

Implementation of a deep convolutional neural network for predicting chromatin features from DNA sequence
GNU General Public License v3.0
17 stars 6 forks source link

How to Import pre-trained weights from trained model? #1

Closed callsobing closed 4 years ago

callsobing commented 5 years ago

Hi rschwess, Say if I want to load the existing model parameters from the saved models (Ex: The trained model from DeepSEA data.), What's the best way for me to do it? Did I need to load the model and then save the weights in the model as a separated file? Thanks!

rschwess commented 4 years ago

Hi callsobing.

Please use run_save_weights.py for doing this. I've updated it with some comments. You can run it in an interactive session and link to the model and graph files as described in the script. Or comment out the FLAGS definition to run it from the command line. I prefer the first variant as it lets me check the saved arrays and dimensions.

For the erythroid model, which is the model used for transfer learning in deepC. I have also included the saved weights in the .tar.gz archive for download. just redownload from the link in the models README.

Let me know if that works for you.

callsobing commented 4 years ago

Wow, that was a real quick update, Thanks!! I have already pull out the weights from the model by myself and re-build it and it works smoothly! Thanks for updating the run_save_weights script, I will try it out and see if it works!

callsobing commented 4 years ago

Sorry, but I have another issues here. I tried to used the saved weights for the run_training_deephaemwindow.py script and it looks like I need to set the "hidden_units_scheme" as "300,600,900,900,900" 'kernel_width_scheme' as "8,8,8,4,4" 'max_pool_scheme' as "4,5,5,5,2"

rschwess commented 4 years ago

Yes thats the architecture for the erythroid deepHaem model. The default values are different but if you want to seed with saved weights you need to adjust the scheme and so as you described, also the seed_scheme flag needs adjusting to "1,1,1,1,1" in your case. Out of curiosity why do you want to seed the run training script? You can also reload the model using the reload flag and linking to the model file using the model flag.

callsobing commented 4 years ago

I am not sure if I am correct, but I want to build another model based on these pre-trained weights. What I need to do is the same convolution layers and weights but with different size of input and outputs. The reload_model flag seems to reload the whole model and couldn't fit the different size of inputs.

rschwess commented 4 years ago

Ah, yes in that case going the route you took was correct. Reload will load the entire thing including the fully connected layer (and dimensions). Using the seeding will require you to specify the convolution dimensions as least as big as the weights to load and fail otherwise. If you specify a bigger architecture (e.g. more filters), it will populate what it can with the provided filter weights and sample the remainder randomly from the "same" distribution. Let me know if you run into more problems.

callsobing commented 4 years ago

Thanks rschwess, that was really helpful! Also thank you for the great work for both pre-trained model and deepC!