Missing a lot of sequences

[ArtPoon]

Currently we are only displaying 3 samples categorized as BA.2.86:

However the GISAID database currently holds 776 records with this lineage label.

[ArtPoon]

[covizu@Paphlagon data]$ unxz -c provision.2024-04-13T00\:00\:06.json.xz | grep -c "\"covv_lineage\": \"BA.2.86\""
947

[GopiGugan]

=# select count(accession) from sequences where lineage='BA.2.86';
 count
-------
   916
(1 row)

It looks like we have 916 records in the database. Checking to see if these sequences are being filtered

[GopiGugan]

Of the 918 BA.2.86 records that made it to the filter_problematic function:

https://github.com/PoonLab/covizu/blob/ca3379dc823bda5b6e849820a861d60047699d84/covizu/utils/gisaid_utils.py#L196-L267

850 sequences were filtered out as being outliers, 65 were filtered out for having a lot of missing sites

[ArtPoon]

Ok I think we have to turn off molecular clock filtering for now. Let's do the following:

• pass cutoff=0 to filter_problematic
• set qp = None if cutoff==0 around line:

  qp = QPois(quantile=1-cutoff, rate=rate, maxtime=maxtime, origin=origin)

• modify test if qp.is_outlier(coldate, ndiffs): to if qp and qp.is_outlier(coldate, ndiffs): to skip test if qp is None

[ArtPoon]

Alternatively it might be easier to modify QPois to return False for every call to is_outlier when it is initialized with cutoff=0

[ArtPoon]

Reprocessing everything in the database to update variants with this change