philchalmers / SimDesign

Structure for organizing Monte Carlo simulations in R
http://philchalmers.github.io/SimDesign/
61 stars 18 forks source link

Validate function outputs #30

Closed mronkko closed 7 months ago

mronkko commented 7 months ago

The package should provide an informative error message when the generate or analyse functions do not return valid outputs.

Here is a test case that produces a non-informative error

library(lavaan)
library(tidyverse)
library(SimDesign)

model.str <- "
F1 =~ x1 + x2 + x3
F2 =~ x4 + x5 + x6
M =~ x1 + x2 + x3 + x4 + x5 + x6
"

model <- lavaanify(model.str, model.type = "cfa")

popModel <- mutate(model,
                    free = 0,
                    ustart = 1)

# approximate population covariance matrix

cov(simulateData(popModel, sample.nobs = 100000))

Design <- createDesign(N = 10^(2:5))

#-------------------------------------------------------------------

Generate <- function(condition, fixed_objects = NULL) {
  dat <- simulateData(popModel, sample.nobs = condition$N)
  dat
}

Analyse <- function(condition, dat, fixed_objects = NULL) {
  ret <- cfa(model.str, dat)
  ret
}

Summarise <- function(condition, results, fixed_objects = NULL) {
  ret <- c(converged = lavInspect(results, "converged"),
           admissible = lavInspect(results,"post.check"))
  ret
}

#-------------------------------------------------------------------

res <- runSimulation(design=Design, replications=1000, generate=Generate, 
                     analyse=Analyse, summarise=Summarise)
philchalmers commented 7 months ago

Thanks. The example above now gives the following:

res <- runSimulation(design=Design, replications=1000, generate=Generate, 
                     analyse=Analyse, summarise=Summarise)

Design: 1/4;   RAM Used: 259.9 Mb;   Replications: 1000;   Total Time: 0.00s 
 Conditions: N=100
  |                                                  | 0 % ~calculating  
Error: Invalid object returned from Analyse()
mronkko commented 7 months ago

Can you adjust the error a bit: Instead of saying "invalid object", can you modify it to tell what kind of object is expected and what was returned instead?

philchalmers commented 7 months ago

Sure, though I don't see any reason to return the class of the original object returned. Here's what is returned now:

> res <- runSimulation(design=Design, replications=1000, generate=Generate, 
+                      analyse=Analyse, summarise=Summarise)

Design: 1/4;   RAM Used: 267.9 Mb;   Replications: 1000;   Total Time: 0.00s 
 Conditions: N=100
  |                                                  | 0 % ~calculating  
Error: Analyse() must return a numeric vector, list, or data.frame
mronkko commented 7 months ago

Thanks. This helps a lot. I was thinking that returning the class name would give even more information that can help in understanding the error in how the analyse function is defined.