Vaccination Rings - Counter intuitive results

[missyschoenbaum]

Here is some documentation. TX QUADS simulations further analysis.docx

[missyschoenbaum]

To do analysis, I need a dataset that shows every farm, every day, every disease state (as they change), indicator when exposure happened, indicator when infection happened, indicator when vaccination happened.

[missyschoenbaum]

@ndh2 I am going to have to ask for help on this. Can you go through the steps you did (long ago) to determine if we are having the same action?

[ndh2]

You want the pie charts showing how often vaccination had an effect like in that old doc?

[missyschoenbaum]

Can I have counts also? I think I was having a hard time figure out what is getting counted as total vaccinations, since vaccination may happen more than once. I can figure out the protective ones.

[ndh2]

Can you upload the file to Google Drive?

[missyschoenbaum]

Done, this is the VX Rings side only. Need to match in Production. In this issue number.

[ndh2]

Just as background, what is counterintuitive about the outputs?

[missyschoenbaum]

Changed files, my 7 file wasn't opening. Counter intuitive was the terminology you used originally. As I recall, vaccination was happening, but was only minimally protective. The vaccinated units didn't get challenged as much as expected. My notes from an email say:

When I say "in the iterations where vaccine had an effect", I mean iterations in which there was at least one occurrence of an adequate exposure going to a vaccine immune unit. In our current model implementation, that is the only decision point at which vaccination can change the course of the outbreak.

In most of the iterations (85-95% of them) that never happened. And in the iterations where it did happen, only a tiny fraction of the units that were vaccinated actually ended up receiving an adequate exposure.

I'll see if I can find that! As I recall we looked at:

1. how many farms that were vaccinated
2. how many of those went vaccine immune (that is, they didn't get infected during the delay before immunity happens)
3. how many of those received an adequate contact

3 is the only place where vaccination actually gets to influence the progress of a simulation (a farm that would have become infected, didn't because it was vaccine immune). And I think we found in the particular simulations we were looking at that #3 only occurred very rarely.

[ndh2]

From a first go at counting what happened. Does this look roughly like what you were counting @missyschoenbaum?

840RingTestVXOnly8.sqlite3 Simulations with no vaccine effect: 1 (1.0%) In remaining 99 simulations, Average # of units vaccinated = 839 Vaccinated, but did not go immune: 75.5% Vaccinated, went immune, but never blocked an adequate exposure: 0.3% Vaccinated, went immune, and blocked at least one adequate exposure: 24.2%

[missyschoenbaum]

@ndh2 Yes, this is what I was trying to figure out. From the old notes, I recall that 85% - 92% were having no protection at all in the rings branch. When we compare this to ProdTestVXOnly8, are we in some ballpark of 12% protection (compared to Vaccinated, went immune, and blocked at least one adequate exposure: 11.8%) ?

[ndh2]

Here's what I get from ProdTestVXOnly8:

Total simulations: 100 Simulations with no vaccine effect: 0 (0.0%) In remaining 100 simulations, Average # of units vaccinated = 367 Vaccinated, but did not go immune: 67.7% Vaccinated, went immune, but never blocked an adequate exposure: 0.4% Vaccinated, went immune, and blocked at least one adequate exposure: 31.8%

[missyschoenbaum]

OK, let me confirm that we are running matched comparisons as much as possible. Then I may ask again. I will take it back for now.

[ndh2]

These numbers seem odd though. Why so many vaccinated don't go immune? The first explanation that comes to mind is that most of the vaccination is happening just before the simulation exits (so the simulation stops before the immunity develops).

And "never blocked an adequate exposure" seems really low. Why would almost every vaccine-immune unit be getting hit with an adequate exposure? Are they in a really small area with strong airborne spread parameters? Or is the direct/indirect contact really high? Or did I miscount them?

[missyschoenbaum]

Direct/indirect contact is very high.

[missyschoenbaum]

@ndh2 I turned contact down some and made 2 new files. They have the number 9 in title, and are on google drive. I also turned off zones.

I have been looking at the "Why so many vaccinated don't go immune?" questions while I was attempting to figure this out. In a combined dataset, I can see that those that don't go immune were already in one of the disease states (L,B,C).

[ndh2]

I thought a Sankey plot might be better for visualizing this than a pie chart. Here's what I get for the 2 new files. The outcome of vaccination is about half-and-half between "has a protective effect" and "was already infected when vaccination happened". Top chart is production, bottom chart is new branch. Counts are based on 100 runs.

[missyschoenbaum]

Excellent, we are going to look at it this afternoon.

[missyschoenbaum]

Tim and I reviewed notes. We think that the situation described from QUADs is not happening. We are going to call this good, recognizing that there may be ways that zones interact with vaccination that we have to figure out.

[ConradSelig]

As decided upon in today's Development meeting (7/23), we decided that while there may still be something off regarding vaccination rings, it does not need to be backed out.

[missyschoenbaum]

Item resolved

[missyschoenbaum]

additional notes ExtraNotesfromassortedEmails.docx