search

flatironinstitute / q2-matchmaker

A qiime2 plugin for case-control differential abundance analysis
BSD 3-Clause "New" or "Revised" License
6 stars 2 forks source link

Out-of-distribution statistics #21

Open mortonjt opened 3 years ago

mortonjt commented 3 years ago

Until CmdStanMCMC have checkpoints, we will need to have some methodology in place to handle out-of-distribution statistics.

Basically, this function should take as input an arviz InferenceData object, and calculation the distribution of log-likelihoods for another observed dataset.

gibsramen commented 3 years ago

Pinging @gwarmstrong as I believe this is something he was interested in as well.

gibsramen commented 3 years ago

I know the PR was closed but would something like this be what you're looking for? I think there should be a way to do this (possible with fixed_param) in CmdStanPy.

Side note but if this were to work I think it would be a great general purpose script/package that we could expand to arbitrary Stan modeling.

mortonjt commented 3 years ago

ahhh. Yes I think that may be the hack we need to get this to work. Thanks! And yes, if we get this to work, it would be a great way to build out-of-distribution statistics.

On Tue, May 4, 2021 at 11:19 AM Gibs @.***> wrote:

I know the PR was closed but would something like this https://mc-stan.org/docs/2_24/stan-users-guide/stand-alone-generated-quantities-and-ongoing-prediction.html be what you're looking for? I think there should be a way to do this (possible with fixed_param) in CmdStanPy.

Side note but if this were to work I think it would be a great general purpose script/package that we could expand to arbitrary Stan modeling.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mortonjt/q2-differential/issues/21#issuecomment-832107886, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA75VXICI3ETZCQMVDG4SRTTMAUB7ANCNFSM432VQTHQ .