Kevin-Haigis-Lab / speclet

A Bayesian hierarchical model to discover tissue-specific cancer driver genes and synthetic lethal interactions from CRISPR/Cas9 LoF screens.
GNU General Public License v3.0
0 stars 0 forks source link

Add uniformity check to SBC pipeline #102

Closed jhrcook closed 3 years ago

jhrcook commented 3 years ago

Add the uniformity check spelled out in the original simulation based calibration paper (cited below). Keep the current diagnostics and checks in the report, as they provide additional useful information, though care should be taken when analyzing the “accuracy” of the models on simulated data.

Talts, Sean, Michael Betancourt, Daniel Simpson, Aki Vehtari, and Andrew Gelman. 2018. “Validating Bayesian Inference Algorithms with Simulation-Based Calibration.” arXiv [stat.ME]. arXiv. http://arxiv.org/abs/1804.06788.

jhrcook commented 3 years ago

While I am at it, it would be useful to collate the MCMC diagnositcs that I recently added to the reports and show plots relating the overall status of the MCMC process. For example, what is the distribution of the number of divergences per chain and variability in the number of divergences per simulation.

jhrcook commented 3 years ago

Sources:

jhrcook commented 3 years ago

I have added the uniformity check to the simulation_based_calibration_helpers module. 🥳

A few more changes: