COVID19Tracking / issues

Repo for issues and administration
45 stars 6 forks source link

Cases where states failed to report negative tests #151

Closed philgoetz closed 4 years ago

philgoetz commented 4 years ago

I've been graphing the fraction of new test results that are positive by state. Some states have a very low fraction of tests come back positive, except for days where no negatives were reported. I'm reporting all the cases I saw which seem statistically unbelievable.

AL had 100% + 3/28-29 AZ had 137 of 137 + 3/28. CA has many days of reporting 100% of 100-1000 tests positive. DC reported no positives on 4/1. DE reported 100% + on 3/16-3/29, under 2% + after that. GA reported 100% + on 3/30. HI reported 100% + 3/26-28. KS usually has around 10% positive, but from 3/19-3/23 reported 66 positives and no negatives. MD reported mostly 10-20% positive, but from 3/12-3/27 reported 762 positives and no negatives. ME reported 100% positive on 3/21, 3/22, 3/27-3/30, 4/1-4/2 (225 positives), but under 2% positive all other days. MI reported 100% positive (or more) on 3/21-3/25, 3/30-4/02 (7851 positives, 0 negatives) and some other days. MN had 100% positive on 3/23 (over 60 positives), and under 1% positive all other days. MO had 100% positive 3/21-3/27 (622 positives), under 2% all other days. MS has days of suspicious 100% positive on 3/28-29. NC had 100% positive on 3/24 (>100 positives) and under 2% all other days. NM had 100% positive 3/21, under 20% all other days. OH had 100% positive 3/17-3/24, 3/28, 3/29, 4/1; under 1% all other recent days. SC has suspicious high positives 3/23, 3/26. TN has 100% + 3/19, 3/22-23 TX had 100% + of 500 cases on 3/29, under 3% + all other days. UT had 100% positive through 3/18, and under 1% positive all later days. VI failed to report negatives through 3/24. VT had 100% of 33 + on 3/23; usually 1% positive. WA had 100% + 3/24-25, 4/01, under 1% all other recent days.

philgoetz commented 4 years ago

Could this happen because states delay reporting the negative cases, as they're not time-critical? E.g., NY on 3/6 reported cumulative totals of 92-, 236 pending. Then NY reported no new negatives until 3/13, reporting 2687 - on that day. 2779 appears to be the true cumulative total negatives at that time, based on the usual +/- ratio, so that 2687 new negatives means all the negatives for tests done 3/7-3/13 were reported on 3/13.

julia326 commented 4 years ago

@philgoetz thanks for your attention here! Forwarding to our data folks

Jmuccigr commented 4 years ago

Note that the data page already says not to report positive rate for states with a data grade less than A. The grades aren't given diachronically, so it's difficult to tell whether a state got better and used to be poor, but many of the states you cite have B ratings, so you can't get a reliable positive rate.

CA for sure is a mess and so is MI, sadly in both cases because one is a large state with an early hotspot (Bay Area) and the other is another hotspot.

PS Not affiliated with the project, just a happy user. Note that no tracking project can get around problems with the states, but at least this one (looking at you JHU) tries to do test counts.

una-smith commented 4 years ago

For over a week now I have been graphing cumulative positives vs negatives, watching as some states get on top of reporting their negatives and others don't.

The states are now being told to report all test results but in some states commercial labs are struggling to keep up with the reporting. Oklahoma says on their dashboard that they are under-reporting negative test results due to delays in the commercial labs. It looks like Michigan, New Jersey, and New York may also be under-reporting negatives. Possibly other states "above the line".

Another reason for a large pos:neg ratio is that the state's epidemic is growing faster than the state's capacity to test, so the state is testing only persons who are symptomatic and their close contacts, and may not be able to do much surveillance testing. This appears to be what is happening in Guam and Puerto Rico, and it may also be happening in Michigan. Read the vertical distance from the green line and keep in mind that the graph is logarithmic. Michigan has by far the highest ratio of positive to negative results.

posneg.04.05.pdf

Jmuccigr commented 4 years ago

Oh, it's been clear for a while that few, if any, states are testing asymptomatic people. Most European countries aren't either, though Germany seems an exception (and some small-scale tests in Italy), but the European countries are mostly testing a lot more people a day per capita than most states.

Genuine question: how are states (like CA yesterday) underreporting only negative cases? What does it matter whether the case is pos or neg when they get them back from commercial labs? They can report the numbers easily. I find this very odd.

una-smith commented 4 years ago

How states are under reporting only negative cases: The commercial labs are reporting positive tests promptly but report negative tests weekly and in surge situations the weekly reports are getting delayed. The reports are not just counts; they are individual test results with sample codes that need to be reconciled against other records for various purposes, including to track all the tests by county and sample date. That involves a lot of manpower.

New Mexico has been testing asymptomatic persons returning from travel to certain hotspots, and has been doing COVID-19 surveillance testing via its influenza surveillance system. Possibly some other states with low pos:neg ratios and relatively large numbers of tests are also testing many asymptomatic persons. See Alaska, North Dakota, Hawaii, Minnesota.

Jmuccigr commented 4 years ago

@una-smith Thanks. That was helpful.

djsandel638 commented 4 years ago

WA reports, "4/14: We are now able to report both positive and negative testing results, and data have been updated back through April 4.​" However, the dataset does not currently reflect these corrections. https://www.doh.wa.gov/Emergencies/Coronavirus

una-smith commented 4 years ago

the dataset

Which dataset?

djsandel638 commented 4 years ago

"Historical data for Washington" https://covidtracking.com/data/state/washington#historical I noticed the lack of negatives when viewing https://data.world/liz-friedman/covid-tracking-project-data/workspace/file?filename=States+-+historical

femto113 commented 4 years ago

Recent data from Tennessee just looks weird to me. There's a note stating "Tennessee now provides positive and negative test results from all laboratories" but the positivity rate jumps around like crazy. Here's a chart comparing the daily positivity rate (positiveIncrease / totalTestResultsIncrease) vs positiveIncrease. There are days when the rate jumps but cases don't, days when the cases jump when the rates don't, and days when they jump together, and an oscillation pattern in new cases. My immediate instinct is that different labs are reporting differently and not on the same cadence.

Screen Shot 2020-04-20 at 7 08 06 PM
stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

stale[bot] commented 4 years ago

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."