search

compomics / DeepLC

DeepLC: Retention time prediction for (modified) peptides using Deep Learning.
https://iomics.ugent.be/deeplc
Apache License 2.0
52 stars 18 forks source link

Pygam calibration is the only option in the latest version #59

Closed markmipt closed 1 year ago

markmipt commented 1 year ago

Hi everyone,

I've noticed that Pygam calibration is forced in the DeepLC 2+ versions. Any reason for that? According to my experience with DeepLC (v. 1.1.2), your default linear calibration usually works better (~20-30% reduce in standard deviation between predicted and experimental RTs) compared to pygam calibration. Of course, I could do "old" calibration by myself, but I trying to make sure that I'm not missing something.

Regards, Mark

RobbinBouwmeester commented 1 year ago

Hi Mark,

I will have a look, to support the old calibration again. My experience is that pygam provides better calibration, but it is good to know this is not always the case.

FYI, if you really want to get everything out of your predictions maybe check the "deeplc_retrain" parameter when initializing the DeepLC object and set it to true. This flag will apply transfer learning, which is usually much more performant. With this transfer learned model the calibration should also be a non-issue.

Kind regards,

Robbin

markmipt commented 1 year ago

Hi Robbin,

Thanks for advise (I've never noticed that transfer learning could be easily done just by using deeplc_retrain option)! I'm currently playing with transfer learning using your instructions from the proteomicsml.org project. So far, the transfer learning also works worse compared to simple calibration. That is due to specifics of my calibration data - I have some fraction of false peptides (~20-30%) and I expect that sophisticated calibration methods (like pygam) and transfer learning are inaccurate due to "overfitting" to false peptides.

Regards, Mark

RobbinBouwmeester commented 1 year ago

I see! That could indeed be the case, pygam and transfer learning are both heavily dependent on the quality of the calibration peptides.

I am currently looking into getting the old calibration method working again.

RobbinBouwmeester commented 1 year ago

You should be able to perform piecewise linear calibration again (>= v2.2.6, 8b250ac).

markmipt commented 1 year ago

Thanks! Calibration results looks better now. Preliminary numbers say that v. 1.1.2 is still slightly better, but cannot do full testing until the batch_number issue will be resolved. Anyway, thanks for adding linear calibration back!

Mark