search

florianhartig / DHARMa

Diagnostics for HierArchical Regession Models
http://florianhartig.github.io/DHARMa/
203 stars 22 forks source link

Multiple models DHARMa #204

Open florianhartig opened 3 years ago

florianhartig commented 3 years ago

From @seanhardison1

Hi Florian,

I wanted to reach out and let you know that I've updated the simulateResiduals function to accept a list of models as input (forked repo at https://github.com/seanhardison1/DHARMa). If there's more than one model used as input, then the function will return a nested list containing the DHARMa residuals objects and the input models (to help keep track of them).

Should I next adjust downstream functions depending on simulateResiduals(), or do you think users should rely on their own loops to deal with the list of DHARMa residual objects? I could at least see updating the plot method to display formulas for keeping track of model inputs.

What do you think the best way to proceed would be? Happy to follow your lead

Thanks,

Sean

florianhartig commented 3 years ago

Hi Sean,

thanks for that. Two immediate comments / thoughts:

  1. The models are contained in the DHARMa objects, so this is redundant.
  2. No, I would definitely not implement support from lists of DHARMa objects.

The latter is simply that (apart from the significant work do do so), experience shows that if you implement a an innocent additional feature at some point in a larger software project, you are carrying this around for years, and it often restricts what you can do later. For that reason, I would add additional interface features only if they are strictly necessary, and I don't see why this is the case here, because the users could simply run the functions on the single DHARMa objects.

For the same reason (I'm sorry, I realize that you put quite a bit of work in this), I wonder about the necessity to overload simulateResiduals. Why can't the user simply create a range of DHARMa objects, as in 

m1 = lm()
r1 = simulateResiduals(m1)
plot(r1)

m2 = lm()
r2 = simulateResiduals(m2)
plot(r2)

and then the only thing to do would be to add a summary function

summary(r1,r2) that implements the desired summary statistics (and possibly some messages that give advice about particular problems)

seanhardison1 commented 3 years ago

Thanks, Florian. I agree with your arguments that updating simulateResiduals is not necessary and will overcomplicate things. I'll focus my efforts instead on a summary function as you described it. Also, don't worry, I just added a for loop to simulateResiduals so I haven't sunk a lot of time into this (yet) ;)

florianhartig commented 3 years ago

OK, great!

florianhartig commented 3 years ago

Hi Sean, I wanted do ask - are you still actively working on this?

seanhardison1 commented 3 years ago

Hi Florian,

I apologize for not being responsive about this. I'm not actively working on this, but I should be able to pick it up again in the next couple of months. That said, I understand if you would like to implement this on your own schedule.

Sean

On Tue, Jan 26, 2021 at 8:48 AM Florian Hartig notifications@github.com wrote:

Hi Sean, I wanted do ask - are you still actively working on this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/florianhartig/DHARMa/issues/204#issuecomment-767634111, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHSPB4YDUSURP3DQ4A55HI3S33P3XANCNFSM4RQJVTEA .

florianhartig commented 3 years ago

Hi Sean,

no worries, I am just scheduling a new release and wanted to know if we should include it in there, just moved this issue to future development for the moment.

Cheers, Florian

seanhardison1 commented 3 years ago

Great! thank you.