jbloomlab / polyclonal

Model mutational escape from polyclonal antibodies.
Other
3 stars 0 forks source link

default fitting parameters? #149

Closed jbloom closed 1 year ago

jbloom commented 1 year ago

I am updating polyclonal to the next major version, and I would like input on whether the default values for the regularization (spatial, uniqueness, etc) seem to mostly make sense or if people have found it consistently necessary to adjust those weights.

@fwelsh @caelanradford @Bernadetadad @timcyu, can you reply on this thread as soon as you get a chance with details if any of the default regularization weights are consistently different from what you are finding works best?

(Obviously they will remain adjustable, but I'd like to have defaults as reasonable as possible.)

Bernadetadad commented 1 year ago

In my case defaults worked well to start with (caveat being that I wasn't able to fit more than 2 epitopes with any settings I tried). I do always have more than one epitope so I would maybe change max_epitopes to 2, but that might be spike specific .

caelanradford commented 1 year ago

These are the parameters that I am currently using for HIV sera: max_epitopes: 2 n_bootstrap_samples: 50 min_epitope_activity_to_include: .25 fit_kwargs: reg_escape_weight: .1 reg_uniqueness_weight: 0 reg_uniqueness2_weight: 2 reg_spatial_weight: 0.0 reg_spatial2_weight: 0.001

jbloom commented 1 year ago

@caelanradford, I think you can drop n_bootstrap_samples, right? The current code does not do any bootstrapping.

Otherwise looks like you have just increased the uniqueness2 and escape regularization weights. Did doing these turn out to be important?

fwelsh commented 1 year ago

Default parameters I'm currently using -

max_epitopes=2, reg_escape_weight=0.1, reg_uniqueness_weight=0, reg_uniqueness2_weight=1, reg_spatial_weight=0.0, reg_spatial2_weight=0.001,

I would prefer not raising the uniqueness2 weight higher than 1 as a default. Weights above 1 tend to be too high for any models I fit, and overweighting significantly drops the sitewise escape scores.

caelanradford commented 1 year ago

Increasing the escape regularization weight seems to generally reduce the noise in the maps.

The uniqueness2 increase for HIV was to get the models to stop putting residues in the same glycosylation motifs into separate epitopes, because that does not make sense.

jbloom commented 1 year ago

Addressed in #151. The reg_escape_weight was bumped from 0.02 to 0.05 since most people were using > 0.02.