Investigating Sweden divergences/High Rhat/Low ESS

[karimn]

SE, FR, and IT are having problems. Need to investigate this.

Investigate job 64349420

[wwiecek]

Please share stanfit summaries or mobiltiy reports if you think that would be helpful. Especially for Sweden, which we worked with before.

[karimn]

@wwiecek do you still have problems accessing Dropbox? I'm wondering if for temporary files like fit and results files investigating issues like these, we can use Dropbox. These files don't need to be version controlled on github and would end up taking up LFS storage space.

[karimn]

Related to #28

[wwiecek]

@wwiecek do you still have problems accessing Dropbox? I'm wondering if for temporary files like fit and results files investigating issues like these, we can use Dropbox. These files don't need to be version controlled on github and would end up taking up LFS storage space.

Ah, yes, I can probably get on it. But maybe PDF (mobility_report) is enough anyway?

[wwiecek]

I meant PDF for Sweden, mostly. For France I replied elsewhere.

[karimn]

Here's the SE report mobility_report.pdf

Notice the sub-region Norrbotten

[wwiecek]

We saw it give problems before. It's very similar to Vasterbotten, a remote region with 200k population. I think setting imputed_cases lower depending on population size (as we discussed before) will help. In this run the imputed cases for this one province were way too high.

I guess that even re-running it or just setting low initial values for imputed cases might fix that. If not, lower prior on imputed cases. A more comprehensive solution is to have this totally new parameterisation for imputed cases, but we can save that for the next version

BTW, I see a bit of a funnel shape. Not related to the above and I'm not sure if this is a problem, but good to keep an eye on?

[karimn]

Yes, we are yet to implement a population dependent prior for imputed cases. Do you have a sensible prior for that in mind? By the way this is issue #16

Yes, I'm seeing the funnel as well but these divergences don't appear to be caused by that, shape as they are not in the narrow part. Am I right?

On Thu, Jul 16, 2020 at 10:24 AM Witold Wiecek notifications@github.com wrote:

We saw it give problems before. It's very similar to Vasterbotten, a remote region with 200k population. I think setting imputed_cases lower depending on population size (as we discussed before) will help. In this run the imputed cases for this one province were way too high.

I guess that even re-running it or just setting low initial values for imputed cases might fix that. If not, lower prior on imputed cases. A more comprehensive solution is to have this totally new parameterisation for imputed cases, but we can save that for the next version

BTW, I see a bit of a funnel shape. Not related to the above and I'm not sure if this is a problem, but good to keep an eye on?

[image: image] https://user-images.githubusercontent.com/20645798/87682693-0e693400-c778-11ea-9782-3e385f524f3e.png

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/karimn/covid-19-transmission/issues/26#issuecomment-659446520, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABB55ZEGDKWZ2TMQKI3QDLR34EQ3ANCNFSM4OZ76BSA .

[wwiecek]

I think so, I'm just wary of funnels!

[karimn]

I know, all Stan users have been indoctrinated to fear funnels :-)

On Thu, Jul 16, 2020 at 11:03 AM Witold Wiecek notifications@github.com wrote:

I think so, I'm just wary of funnels!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/karimn/covid-19-transmission/issues/26#issuecomment-659471524, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABB557FUNONB26IMOWZG63R34JDJANCNFSM4OZ76BSA .

[karimn]

Fixing IFR doesn't help Sweden. Also I'm not actually seeing the population of Vasterbotten to be very different than other regions in Sweden.

Updated results with population plot mobility_report.pdf

[wwiecek]

Well, different run and different provinces break. That alone is proof of something. Previously the conclusion was that a region like Vasterbotten will not converge when imputed cases are high. This seems to be the conclusion here too. What's the prior on tau imputed cases parameter? 0.18 (instead of old 0.03)?

[karimn]

Yes, 0.18

[karimn]

One more run, no divergences by the still having problems with Rhat and ESS. the posterior dist is showing some bimodality (again IFR is fixed here). mobility_report.pdf

[wwiecek]

I think we can let this slide for now? To me the concluson is that some regions are still very sensitive to starting conditions or heavy-tailed priors, esp. to N cases, but this may (should) not be the case anymore when we make more fixes to imputed cases, which we are planning to do anyway. What do you think?

[karimn]

Alright, I'll hold off work on this until imputed cases are fixed #16

[karimn]

I can't repro this anymore after several runs.