Meta-modeling function

[evaenns]

metam()

Input: PSA results from a model, name(s) of the outcome(s) of interest to be predicted by meta-model, meta-model type (linear, higher-order polynomial, splines, "find the best")

Output: meta-model

Check out meta-modeling code embedded in OneWaySA.R, TwoWaySA.R in feralaes/dampack as start.

Need to provide guidance on the number of PSA samples needed. Does this vary by the number of parameters varied? And/or number of outcomes to be predicted? Need clear documentation about best-practices on using a meta-model in different situations / different types of models.

[caleb-easterly]

@feralaes , @evaenns I started working on the metamodeling function and I'm having trouble understanding the outcome part of it. A few questions:

1) is the outcome of interest usually cost, effectiveness, or both? 2) would we always want to model outcomes for all strategies?

Thanks!

[caleb-easterly]

just pinging you again, @feralaes, @evaenns.

For example, I'm using the standard example (Chemo vs. Radio vs. Surgery), and I'm wondering what the metamodel call should be.

For testing, I'm doing

mm <- metamod(y = psa_big$effectiveness, psa = psa_big, parm = "pFailChemo")

So, y is the data frame with the outcome(s) of interest. In this case, it's the effectiveness of all 3 strategies. I guess I was wondering if we would want to pass, instead, something like:

mm <- metamod(outcome = c("eff", "cost", "both"), psa = psa_big, parm = "pFailChemo")

Or, we could do

mm <- metamod(outcome = c("eff", "cost", "nhb", "nmb"),
    psa = psa_big, parm = "pFailChemo", wtp = NULL)

and allow for NHB and NMB as well.

And I was also wanting if we'd want to restrict the strategies as well:

# strategies = NULL means all strategies are modeled
mm <- metamod(outcome = c("eff", "cost", "both"), strategies = NULL,
              psa = psa_big, parm = "pFailChemo")

The most general version, I suppose, is just letting people pass whatever they want to y (as it is currently). But I didn't know if I was misunderstanding something.

[caleb-easterly]

progress in branch metam, #24

[feralaes]

just pinging you again, @feralaes, @evaenns.

For example, I'm using the standard example (Chemo vs. Radio vs. Surgery), and I'm wondering what the metamodel call should be.

For testing, I'm doing

mm <- metamod(y = psa_big$effectiveness, psa = psa_big, parm = "pFailChemo")

So, y is the data frame with the outcome(s) of interest. In this case, it's the effectiveness of all 3 strategies. I guess I was wondering if we would want to pass, instead, something like:

mm <- metamod(outcome = c("eff", "cost", "both"), psa = psa_big, parm = "pFailChemo")

Or, we could do

mm <- metamod(outcome = c("eff", "cost", "nhb", "nmb"),
    psa = psa_big, parm = "pFailChemo", wtp = NULL)

and allow for NHB and NMB as well.

And I was also wanting if we'd want to restrict the strategies as well:

# strategies = NULL means all strategies are modeled
mm <- metamod(outcome = c("eff", "cost", "both"), strategies = NULL,
              psa = psa_big, parm = "pFailChemo")

The most general version, I suppose, is just letting people pass whatever they want to y (as it is currently). But I didn't know if I was misunderstanding something.

Should we discuss this in our call this week?

[caleb-easterly]

Sure! Sounds like we might not be able to meet until next week, though.

In the meantime, this is the function call I wrote:

metamod(psa, parm, strategies = NULL, outcome = c("eff", "cost", "nhb", "nmb"),
        wtp = NULL, poly.order = 2)

[feralaes]

Cool! Is there a way to call it and produce a new "outcome" based on the effectiveness and costs? This is, to compute later on the VOI metrics. Specifically, we will need to compute the losses and then do the metamodel on the losses. You can find more details here: https://github.com/feralaes/VOI-Gaussian-Approximation

[caleb-easterly]

Yes, I think we could either implement that directly (outcome = c("eff", "cost", "nhb", "nmb", "loss")). That's for EVSI? I think we could keep metamod as it is for now, and then discuss how it needs to be expanded for EVSI, etc.

[feralaes]

Not only for EVSI but also for EVPPI. But yes, let's discuss this on Monday. You can get an idea of how we are going to use it based on the code of the old dampack "evppi_lrmm"