search

florianhartig / DHARMa

Diagnostics for HierArchical Regession Models
http://florianhartig.github.io/DHARMa/
203 stars 22 forks source link

Integrating a new (bivariate) model into DHARMa #241

Open florianhartig opened 3 years ago

florianhartig commented 3 years ago

Question from a user:

I am wondering about your preference for me using DHARMa in my own package. I have read the following links

https://cran.r-project.org/web/packages/DHARMa/vignettes/DHARMa.html#other-packages

and

https://github.com/florianhartig/DHARMa/wiki/Adding-new-R-packages-to-DHARMA

however I wanted to clarify with you before making any decisions.

Specifically, I have a bivariate model, and I would like to use DHARMa on each margin. However, createDHARMa applies to only one set of data, meaning users would have to type out the createDHARMa parameters twice, or wrangle their data to be compatible with a loop. I have written a wrapper function around createDHARMa for my package that

  1. Takes in model object
  2. Simulates for each margin
  3. Runs createDHARMa on each set of simulations, fitted values, etc
  4. Returns list of DHARMa objects, one for each margin

I am wondering if this is acceptable by you? I have tried explicitly extending createDHARMa, eg, createDHARMa.class(), but this does not seem to be working, so I resorted to the wrapper function. If permitted, the wrapper documentation will make it clear that this is simply calling up your function, but I wanted to clear it with you before moving on.

florianhartig commented 3 years ago

Hi,

OK, there are two aspects here, 1) how to use DHARMa for multivariate responses in general, and 2) how to technically integrate the model.

About 1: I have had various discussions about this in the past, and there are certainly several options. One of the easiest, however, would be simply to "flatten" your multivariate response (i.e. make it a vector), and run it through DHARMa. You can then still subset residuals (e.g. response 1) for particular plots if you want. If this makes sense for your particular problem I can't say without knowing more about what you are doing.

I would also be open to implement additional checks for multivariate responses. I guess the general question is how to best test these residuals. I'm not convinced that testing each margin separately is extracting the maximum info in every case. Residual problems could, e.g., be in the correlation of that margins

About 2: It depends on what you mean by "acceptable". If you want to do this for your package, everything is fine of course. If you want me to consider integrating what you do into DHARMa, then you have to follow the advice in https://github.com/florianhartig/DHARMa/wiki/Adding-new-R-packages-to-DHARMA , which I have just updated a bit, because of recent changes for the 3.4 version. The bottomline of the updates is: the entire communication / documentation with other packages is now handled via the S3 functions in https://github.com/florianhartig/DHARMa/blob/master/DHARMa/R/compatibility.R, so if you can supply this (either in DHARMa or in your package), DHARMa will work with your model classes.

Best, Florian

florianhartig commented 3 years ago

p.s.: just realized this is related to https://github.com/florianhartig/DHARMa/issues/189