search

stan-dev / posterior

The posterior R package
https://mc-stan.org/posterior/
Other
165 stars 23 forks source link

Add functions for PIT and LOO-PIT #338

Open TeemuSailynoja opened 6 months ago

TeemuSailynoja commented 6 months ago

loo_pit is currently in rstantools but needs implementation for discrete variables, and should probably be moved to posterior. pit isn't currently available, but is used, for example, by bayesplot, where I have currently implemented the computation.

My questions are: 1 . PIT would require observations in addition to the draws object, which to my understanding isn't the case for other functions. Is this a problem?

  1. Should there be a new subtitle on the functions page for PIT and LOO-PIT to fall under?
  2. Should they be in the same function as regular PIT is just LOO-PIT with even weights?
avehtari commented 6 months ago
  1. I think it fits here. It's still generic so that there is no dependency on where the draws come from, and if pit function would support draws objects it would be natural here. Yesterday @paul-buerkner also agreed initially
  2. Makes sense
  3. Makes sense
paul-buerkner commented 6 months ago

I think that PIT in posterior does make sense. Will we want to use it for other stuff than plotting though? If yes, than posterior is I think good. If we only ever want to use it for bayesplot, then perhaps also bayesplot could be a reasonable target(?)

paul-buerkner commented 6 months ago

Just to clarify, are we talking about the methods or the generic (or both)?

avehtari commented 6 months ago
paul-buerkner commented 6 months ago

sounds like good arguments for posterior indeed.

Aki Vehtari @.***> schrieb am Fr., 19. Jan. 2024, 18:55:

  • PIT values could be summarised with scalar diagnostic value, but the preferred way is graphics, but we may want to use other plots than what is available in bayesplot (which is what I was doing)
    1. methods, 2. it would be nice if the method function could be called with any posterior draws object type

— Reply to this email directly, view it on GitHub https://github.com/stan-dev/posterior/issues/338#issuecomment-1900761992, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADCW2ABIJ65MBMUIWRKAWHDYPKQP5AVCNFSM6AAAAABCB7SLJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBQG43DCOJZGI . You are receiving this because you were mentioned.Message ID: @.***>

jgabry commented 6 months ago

Sounds good to me too