Model is predicting low numbers for nearly all lineages

[ArtPoon]

• Nearly all lineages are being coloured as having low numbers of unsampled infections.
• We should switch from using this model as being the default colouration for trees, back to Divergence (residual from clock prediction)
• Need to go back and check fit and outputs for the number of infections model.

[ArtPoon]

Looks like the molecular clock model residuals are off as well.

[ArtPoon]

Default display has been switched back to divergence by @GopiGugan

[ArtPoon]

• Run pipeline to capture intermediate outputs if necessary
• See if regression model needs to be updated, or if there is something that has changed for the inputs that is causing the model to go awry

[ArtPoon]

• Downloaded recoded.json and lineage-specific Newick files from Paphlagon to my local machine for some testing
• the make_beadplots function in batch_utils.py badly needs some refactoring, will split it into multiple functions at least.

[ArtPoon]

new branch iss497 56a0786d30e18aeda59d2ba4ba1d6da16e89b0a6

[ArtPoon]

I'll try to wrap up some of this refactoring and then I may need to hand this off to someone else

[ArtPoon]

@GopiGugan can you please retrieve by_lineages from the database and run it through the make_beadplots function, and send me a CSV of the summary stats and predicted number of infections for each lineage?

[ArtPoon]

Obtained CSV from @GopiGugan, will analyze

[ArtPoon]

Summary stats seem reasonable:

Scatterplot of predicted number of infections (from HUNePi model) against sample size (number of sequences):

[ArtPoon]

The issue seems to be that there is a small number of outlier lineages with very high predicted numbers of infections. For the diagnostic summary_stats.csv data that @GopiGugan sent me, the maximum predicted number is about 4.4 million, but most of the predicted numbers fall within the range of 10 to 100,000. On a linear scale, this causes most lineages to be coloured purple/blue:

[ArtPoon]

A log-transform on mapping predicted numbers of infections to the colour scale should resolve this on the front end

[ArtPoon]

Ok this is fixed, tree colours look much better: