search

ropensci / drake

An R-focused pipeline toolkit for reproducibility and high-performance computing
https://docs.ropensci.org/drake
GNU General Public License v3.0
1.34k stars 128 forks source link

drake does not deal well with nlmixr's domain specific language #910

Closed mattfidler closed 5 years ago

mattfidler commented 5 years ago

Prework

From devtools::session_info()

drake       * 7.4.0.9000 2019-06-17 [1] Github (ropensci/drake@da7d2ab)

Description

drake does not parse nlmixr model functions correctly. nlmixr has its own domain specific language to specify nonlinear mixed effects models, which breaks drake's DSL (as mentioned as a possibility by #717)

Reproducible example

library(drake)
## library(nlmixr) # Not necessary to reproduce issue.

### NOTE nlmixr dsl also parses comments;  However, this requires R to run with keep.source=TRUE

p <- drake_plan(model=function() {
    ini({
        tka <- 0.45 # Log Ka
        tcl <- 1 # Log Cl
        tv <- 3.45    # Log V
        eta.ka ~ 0.6
        eta.cl ~ 0.3
        eta.v ~ 0.1
        add.err <- 0.7
    })
    model({
        ka <- exp(tka + eta.ka)
        cl <- exp(tcl + eta.cl)
        v <- exp(tv + eta.v)
        d/dt(depot) = -ka * depot
        depot(0) = 3
        d/dt(center) = ka * depot - cl / v * center
        cp = center / v
        cp ~ add(add.err)
    })
})

make(p)
#> Error: not a symbol: 'depot(0) = 3'

Created on 2019-06-17 by the reprex package (v0.3.0)

Session info ``` r devtools::session_info() #> - Session info ---------------------------------------------------------- #> setting value #> version R version 3.6.0 (2019-04-26) #> os Windows 10 x64 #> system x86_64, mingw32 #> ui RTerm #> language (EN) #> collate English_United States.1252 #> ctype English_United States.1252 #> tz America/Chicago #> date 2019-06-17 #> #> - Packages -------------------------------------------------------------- #> package * version date lib source #> assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.0) #> backports 1.1.4 2019-04-10 [1] CRAN (R 3.6.0) #> base64url 1.4 2018-05-14 [1] CRAN (R 3.6.0) #> callr 3.2.0 2019-03-15 [1] CRAN (R 3.6.0) #> cli 1.1.0 2019-03-19 [1] CRAN (R 3.6.0) #> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.0) #> desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.0) #> devtools 2.0.2 2019-04-08 [1] CRAN (R 3.6.0) #> digest 0.6.19 2019-05-20 [1] CRAN (R 3.6.0) #> drake * 7.4.0.9000 2019-06-17 [1] Github (ropensci/drake@da7d2ab) #> evaluate 0.14 2019-05-28 [1] CRAN (R 3.6.0) #> fs 1.3.1 2019-05-06 [1] CRAN (R 3.6.0) #> glue 1.3.1 2019-03-12 [1] CRAN (R 3.6.0) #> highr 0.8 2019-03-20 [1] CRAN (R 3.6.0) #> htmltools 0.3.6 2017-04-28 [1] CRAN (R 3.6.0) #> igraph 1.2.4.1 2019-04-22 [1] CRAN (R 3.6.0) #> knitr 1.23 2019-05-18 [1] CRAN (R 3.6.0) #> magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.0) #> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.6.0) #> pillar 1.4.1 2019-05-28 [1] CRAN (R 3.6.0) #> pkgbuild 1.0.3 2019-03-20 [1] CRAN (R 3.6.0) #> pkgconfig 2.0.2 2018-08-16 [1] CRAN (R 3.6.0) #> pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.6.0) #> prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.6.0) #> processx 3.3.1 2019-05-08 [1] CRAN (R 3.6.0) #> ps 1.3.0 2018-12-21 [1] CRAN (R 3.6.0) #> R6 2.4.0 2019-02-14 [1] CRAN (R 3.6.0) #> Rcpp 1.0.1 2019-03-17 [1] CRAN (R 3.6.0) #> remotes 2.0.4 2019-04-10 [1] CRAN (R 3.6.0) #> rlang 0.3.4 2019-04-07 [1] CRAN (R 3.6.0) #> rmarkdown 1.13 2019-05-22 [1] CRAN (R 3.6.0) #> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.0) #> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.6.0) #> storr 1.2.1 2018-10-18 [1] CRAN (R 3.6.0) #> stringi 1.4.3 2019-03-12 [1] CRAN (R 3.6.0) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 3.6.0) #> testthat 2.1.1 2019-04-23 [1] CRAN (R 3.6.0) #> tibble 2.1.3 2019-06-06 [1] CRAN (R 3.6.0) #> usethis 1.5.0 2019-04-07 [1] CRAN (R 3.6.0) #> withr 2.1.2 2018-03-15 [1] CRAN (R 3.6.0) #> xfun 0.7 2019-05-14 [1] CRAN (R 3.6.0) #> yaml 2.2.0 2018-07-25 [1] CRAN (R 3.6.0) #> #> [1] c:/R/nlmixr_1.1.0-9/R/library ```

Provide a minimal reproducible example with code and output that demonstrates the bug. The reprex package is extremely helpful for this.

Session info

End the reproducible example with a call to sessionInfo() in the same session (e.g. reprex(si = TRUE)) and include the output.

Expected output

What output would the correct behavior have produced?

Just the complete plan

mattfidler commented 5 years ago

In general nlmixr functions can have more arbitrary lhs experssions than typical R.

One other question, do you run R with keep.source=TRUE or is this even possible?

wlandau commented 5 years ago

And drake assumes all your commands and functions are typical R code. A temporary workaround is to wrap the offending bits in ignore(). The downside is that drake does not react to changes inside ignore() blocks. I will add a special DSL function to tell drake to track changes in the code but not look for dependencies, something like no_deps().

library(drake)
# library(nlmixr)

# It is best to define functions outside the plan.
# Targets are for long-ish computations that produce data.
model_fn <- function() {
  ini({
    tka <- 0.45 # Log Ka
    tcl <- 1 # Log Cl
    tv <- 3.45    # Log V
    eta.ka ~ 0.6
    eta.cl ~ 0.3
    eta.v ~ 0.1
    add.err <- 0.7
  })
  ignore(
    model({
      ka <- exp(tka + eta.ka)
      cl <- exp(tcl + eta.cl)
      v <- exp(tv + eta.v)
      d/dt(depot) = -ka * depot
      depot(0) = 3
      d/dt(center) = ka * depot - cl / v * center
      cp = center / v
      cp ~ add(add.err)
    })
  )
}

# Take shortcuts for the sake of demonstration:
run_nlmixr <- function(...) {NULL}
your_data <- NULL

plan <- drake_plan(analysis = run_nlmixr(model_fn, your_data))

make(plan)
#> target analysis

# Change the contents of the `ignore()` block:

model_fn <- function() {
  ini({
    tka <- 0.45 # Log Ka
    tcl <- 1 # Log Cl
    tv <- 3.45    # Log V
    eta.ka ~ 0.6
    eta.cl ~ 0.3
    eta.v ~ 0.1
    add.err <- 0.7
  })
  ignore(
    model({
      ka <- exp(tka + eta.ka)
      cl <- exp(tcl + eta.cl)
    })
  )
}

# make() does not update the analysis.
make(plan)
#> All targets are already up to date.

Created on 2019-06-17 by the reprex package (v0.3.0)

mattfidler commented 5 years ago

That would be great. I can confirm the first code works, though.

library(nlmixr)
#> Registered S3 methods overwritten by 'ggplot2':
#>   method         from 
#>   [.quosures     rlang
#>   c.quosures     rlang
#>   print.quosures rlang
library(drake)

model <- function() {
    ini({
        tka <- 0.45 # Log Ka
        tcl <- 1 # Log Cl
        tv <- 3.45    # Log V
        eta.ka ~ 0.6
        eta.cl ~ 0.3
        eta.v ~ 0.1
        add.err <- 0.7
    })
    ignore(model({
        ka <- exp(tka + eta.ka)
        cl <- exp(tcl + eta.cl)
        v <- exp(tv + eta.v)
        d/dt(depot) = -ka * depot
        depot(0) = 3
        d/dt(center) = ka * depot - cl / v * center
        cp = center / v
        cp ~ add(add.err)
    }))
}

m <- nlmixr(model)
print(m)
#> __ RxODE-based ODE model __________________________________________________ 
#> -- Initialization: -------------------------------------------------------- 
#> Fixed Effects ($theta): 
#>  tka  tcl   tv 
#> 0.45 1.00 3.45 
#> 
#> Omega ($omega): 
#>        eta.ka eta.cl eta.v
#> eta.ka    0.6    0.0   0.0
#> eta.cl    0.0    0.3   0.0
#> eta.v     0.0    0.0   0.1
#> -- mu-referencing ($muRefTable): ------------------------------------------ 
#> +-------------------+
#> ¦ theta   ¦ eta     ¦
#> +---------+---------¦
#> ¦ tka     ¦ eta.ka  ¦
#> +---------+---------¦
#> ¦ tcl     ¦ eta.cl  ¦
#> +---------+---------¦
#> ¦ tv      ¦ eta.v   ¦
#> +-------------------+
#> -- Model: ----------------------------------------------------------------- 
#>         ka <- exp(tka + eta.ka)
#>         cl <- exp(tcl + eta.cl)
#>         v <- exp(tv + eta.v)
#>         d/dt(depot) = -ka * depot
#>         depot(0) = 3
#>         d/dt(center) = ka * depot - cl / v * center
#>         cp = center / v
#>         cp ~ add(add.err) 
#> ___________________________________________________________________________

Created on 2019-06-17 by the reprex package (v0.3.0)

Session info ``` r devtools::session_info() #> - Session info ---------------------------------------------------------- #> setting value #> version R version 3.6.0 (2019-04-26) #> os Windows 10 x64 #> system x86_64, mingw32 #> ui RTerm #> language (EN) #> collate English_United States.1252 #> ctype English_United States.1252 #> tz America/Chicago #> date 2019-06-17 #> #> - Packages -------------------------------------------------------------- #> package * version date lib #> assertthat 0.2.1 2019-03-21 [1] #> backports 1.1.4 2019-04-10 [1] #> base64url 1.4 2018-05-14 [1] #> brew 1.0-6 2011-04-13 [1] #> callr 3.2.0 2019-03-15 [1] #> cli 1.1.0 2019-03-19 [1] #> colorspace 1.4-1 2019-03-18 [1] #> crayon 1.3.4 2017-09-16 [1] #> desc 1.2.0 2018-05-01 [1] #> devtools 2.0.2 2019-04-08 [1] #> digest 0.6.19 2019-05-20 [1] #> dparser 0.1.8 2017-11-13 [1] #> dplyr 0.8.1 2019-05-14 [1] #> drake * 7.4.0.9000 2019-06-17 [1] #> evaluate 0.14 2019-05-28 [1] #> farver 1.1.0 2018-11-20 [1] #> fs 1.3.1 2019-05-06 [1] #> generics 0.0.2 2018-11-29 [1] #> ggforce 0.2.2 2019-04-23 [1] #> ggplot2 3.1.1 2019-04-07 [1] #> glue 1.3.1 2019-03-12 [1] #> gtable 0.3.0 2019-03-25 [1] #> highr 0.8 2019-03-20 [1] #> htmltools 0.3.6 2017-04-28 [1] #> huxtable 4.5.0 2019-03-19 [1] #> igraph 1.2.4.1 2019-04-22 [1] #> knitr 1.23 2019-05-18 [1] #> lattice 0.20-38 2018-11-04 [1] #> lazyeval 0.2.2 2019-03-15 [1] #> lbfgs 1.2.1 2014-08-31 [1] #> lotri 0.1.1 2019-05-14 [1] #> magrittr 1.5 2014-11-22 [1] #> MASS 7.3-51.4 2019-03-31 [1] #> memoise 1.1.0 2017-04-21 [1] #> munsell 0.5.0 2018-06-12 [1] #> mvnfast 0.2.5 2018-01-31 [1] #> n1qn1 6.0.1-3 2018-09-17 [1] #> nlme 3.1-139 2019-04-09 [1] #> nlmixr * 1.1.1-0 2019-06-13 [1] #> pillar 1.4.1 2019-05-28 [1] #> pkgbuild 1.0.3 2019-03-20 [1] #> pkgconfig 2.0.2 2018-08-16 [1] #> pkgload 1.0.2 2018-10-29 [1] #> plyr 1.8.4 2016-06-08 [1] #> polyclip 1.10-0 2019-03-14 [1] #> PreciseSums 0.3 2018-04-12 [1] #> prettyunits 1.0.2 2015-07-13 [1] #> processx 3.3.1 2019-05-08 [1] #> ps 1.3.0 2018-12-21 [1] #> purrr 0.3.2 2019-03-15 [1] #> R6 2.4.0 2019-02-14 [1] #> Rcpp 1.0.1 2019-03-17 [1] #> RcppArmadillo 0.9.500.2.0 2019-06-12 [1] #> remotes 2.0.4 2019-04-10 [1] #> rex 1.1.2 2017-10-19 [1] #> rlang 0.3.4 2019-04-07 [1] #> rmarkdown 1.13 2019-05-22 [1] #> rprojroot 1.3-2 2018-01-03 [1] #> RxODE 0.9.0-7 2019-06-13 [1] #> scales 1.0.0 2018-08-09 [1] #> sessioninfo 1.1.1 2018-11-05 [1] #> storr 1.2.1 2018-10-18 [1] #> stringi 1.4.3 2019-03-12 [1] #> stringr 1.4.0 2019-02-10 [1] #> testthat 2.1.1 2019-04-23 [1] #> tibble 2.1.3 2019-06-06 [1] #> tidyselect 0.2.5 2018-10-11 [1] #> tweenr 1.0.1 2018-12-14 [1] #> units 0.6-3 2019-05-03 [1] #> usethis 1.5.0 2019-04-07 [1] #> vpc 1.1.0 2018-08-27 [1] #> withr 2.1.2 2018-03-15 [1] #> xfun 0.7 2019-05-14 [1] #> yaml 2.2.0 2018-07-25 [1] #> source #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> Github (ropensci/drake@da7d2ab) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> local #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> local #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> CRAN (R 3.6.0) #> #> [1] c:/R/nlmixr_1.1.0-9/R/library ```

I haven't been able to successfully run a simple model with drake+nlmixr yet, though.

mattfidler commented 5 years ago

Is there a way to inject the property in the parsing so users of drake + nlmixr will not have to add extra code?

Also does drake track package versions as well as data files? I couldn't immediately tell from the documentation.

Thank you for you work.

wlandau commented 5 years ago

Added no_deps() in #911.

Is there a way to inject the property in the parsing so users of drake + nlmixr will not have to add extra code?

Sorry, nlmixr is too much of a niche use case to build into drake directly.

Also does drake track package versions as well as data files? I couldn't immediately tell from the documentation.

It does not. I recommend renv or packrat for this.

I can confirm the first code works, though.

ignore(model({...})) prevents the error, but it prevents drake from seeing changes to the model, so the target will not rebuild in make() if the model changes. On the other hand, if you write no_deps(model({...})), make() detects changes to the model and automatically rebuilds downstream targets as necessary.