search

leeper / prediction

Tidy, Type-Safe 'prediction()' Methods
https://cran.r-project.org/package=prediction
Other
89 stars 14 forks source link

Questions: missing data behavior, general advice #28

Open alexpghayes opened 5 years ago

alexpghayes commented 5 years ago

Missing data behavior

Does prediction provide any guarantees about predictions on missing data? I've been playing around some and it seems like most methods behave like predict() with na.action = na.pass. Is this tested/guaranteed?

I'm currently reworking the augment() methods for broom and am considering moving to prediction as a backend, but need to guarantee that data doesn't get silently dropped.

General advice for predictions

Do you have any general advice for developers implementing predict() methods?

I'm gathering suggestions for R model package development at https://github.com/alexpghayes/principles (currently just rough notes), and if you'd be willing to share your thoughts on what makes a predict() method easy to work with, I'd love to write that up.

leeper commented 5 years ago

prediction() doesn't per se do anything. The na.action is actually handled by find_data() (in the default case) here: https://github.com/leeper/prediction/blob/master/R/find_data.R#L48-L51 I really haven't tested that much.

On the second point, I think we need standardization of model objects so that predict()'s default method just works. Lacking that, type safety seems the most critical (like the main example in the README here).

alexpghayes commented 5 years ago

Gotcha.

Do you have any particular pain points in terms of standardization? We're brainstorming some at topepo/parsnip#41.