Transfert learning with deepsignal2 human mod

[pterzian]

Hi Peng,

I was wondering if it would be possible to load your human CpG model and push the training further my own data (other species). The idea would be to generalize your model to other species without retraining it on human data. Do you think it is possible ? If you do, maybe you can give me a hint on where to start ? The only bias I would think of is that my datasets were sequenced using a promethION and not a gridION like your training dataset and there should be some signal differences between both tech.

Best, Paul

[PengNi]

Hi Paul,

I think it is possible, however, without tests, I cannot now guarantee the performance: whether loading the params of an pre-trained model before training a new model is a better choice.

Actually I did also thought about this idea (deepsignal-plant issue #1), but I am totally not familiar with transfer learning and fine-tuning now, and haven't got time to test it.

To test this idea, the python files need to be modified, I think, are train.py and maybe models.py:

• In train.py, maybe an --initial_model option should be added first.
• If you use pre-training + fine-tuning to train a model, maybe both train.py and models.py should be modified, cause only params of a few layers need to be tuned.

I didn't test whether datasets from different sequencers can raise bias, but in my opinion, I don't think we need to pay much attention about it. however, maybe different flowcells (R9.4/R10.3) need different model training.

As I said, I don't know much about transfer learning. However, I will add the --initial_model option in a few days, hope it can help.

Best wishes, Peng

[pterzian]

Hi Peng,

Thank you for this fast answer,

Honestly I don't think I would have the skill to implement it from DeepSignal code, also because I am a still a DNN newbie. And right now I would feel more confortable using Keras on R. I was thinking of extracting only the last layer of the model and maybe add one or two as a test. I will have a lot to dig that is for sure but it is interesting!

Anyway an --initial_model an option to choose wether creating a model from scratch or not would be useful, if it is what you were thinking about. I guess the best way to do it would be to load the model into deepsignal and continue to feed it with other data but I understand it may not be that simple. I will see what I can try.

Best, Paul