barrettk
commented
5 months ago FYI I finalized a working prototype. Can open up a scratch branch/draft PR, though I imagine there might be some discussion around A) supporting windows, and B) whether or not we need to copy the model to a temporary location like I have. It's worth noting I took quite a bit from NMProject (here & here). The main differences are A) how we determine the NMTRAN executable path (using bbi.yaml) and B) using bbr and nmrec for getting model paths and modifying/fetching data paths.
- Note: I know we initially discussed using
processx, butsystemseems to work pretty well, and gave me no issues when trying to generate the necessary system call. See comments below, but I had issues generating the system call viaprocessx.
run_nmtran & helper functions
```r run_nmtran <- function( .mod, .config_path = NULL, nmtran_exe = NULL, delete_on_exit = TRUE, ... ){ nmtran_exe <- locate_nmtran(.mod, .config_path, nmtran_exe) nm_ver <- attr(nmtran_exe, "nonmem_version") mod_path <- get_model_path(.mod) data_path <- get_data_path_from_ctl(.mod) # make temporary directory in current directory mod_name <- fs::path_ext_remove(basename(mod_path)) tempdir0 <- paste0("nmtran_", mod_name, "_", basename(tempdir())) dir.create(tempdir0) if(isTRUE(delete_on_exit)){ on.exit(unlink(tempdir0, recursive = TRUE, force = TRUE)) } # copy model and dataset file.copy(mod_path, tempdir0) file.copy(data_path, tempdir0) # overwrite $DATA of new model modify_data_path_ctl( mod_path = file.path(tempdir0, basename(mod_path)), data_path = basename(data_path) ) # Get command & append the control file name cmd <- stringr::str_glue(nmtran_exe, .envir = list(ctl_name = basename(mod_path)), .na = NULL) cmd <- paste(cmd, "<", basename(mod_path)) # Run NMTRAN if(!is.null(nm_ver)){ message(glue("Running NMTRAN with NONMEM version `{nm_ver}`")) } system_nm(cmd, dir = tempdir0, wait = TRUE, ...) if(isFALSE(delete_on_exit)){ return(tempdir0) } } locate_nmtran <- function(.mod, .config_path = NULL, nmtran_exe = NULL){ if(is.null(nmtran_exe)){ model_dir <- get_model_working_directory(.mod) config_path <- .config_path %||% file.path(model_dir, "bbi.yaml") if(!file_exists(config_path)){ stop(paste("No bbi configuration was found in the execution directory.", "Please run `bbi_init()` with the appropriate directory to continue.")) } if (!is.null(.config_path)) { config_path <- normalizePath(.config_path) } bbi_config <- yaml::read_yaml(config_path) nm_config <- bbi_config$nonmem # look for default nonmem installation default_nm <- purrr::keep(nm_config, function(nm_ver){ !is.null(nm_ver$default) }) # Set nonmem path if(length(default_nm) > 0){ default_nm <- default_nm[[1]] }else{ # If no default, use the last one (likely higher version) default_nm <- nm_config[[length(nm_config)]] } # Set NMTRAN executable path nm_path <- default_nm$home nmtran_exe <- file.path(nm_path, "tr", "NMTRAN.exe") # If executable found via bbi.yaml, append NONMEM version as attribute attr(nmtran_exe, "nonmem_version") <- basename(default_nm$home) } if(!file_exists(nmtran_exe)){ stop(glue("Could not find an NMTRAN executable at `{nmtran_exe}`")) } return(nmtran_exe) } system_nm_default <- function(cmd, ...) { if (.Platform$OS.type == "windows") { local_env_vars <- Sys.getenv() stdout_unit_vars <- local_env_vars[grepl("STDOUT_UNIT|STDERR_UNIT", names(local_env_vars))] for (i in seq_along(stdout_unit_vars)) { Sys.unsetenv(names(stdout_unit_vars)[i]) } on.exit({ if (length(stdout_unit_vars) > 0) { do.call(Sys.setenv, as.list(stdout_unit_vars)) } }) args <- list(...) if (!"wait" %in% names(args)) wait <- FALSE else wait <- args$wait if (wait == FALSE) { shell(paste("START CMD /C", cmd), ...) } else { shell(cmd, ...) } } else { system(cmd, ...) } } system_nm <- function(cmd, dir = NULL, ...) { if (is.null(dir) || !file.exists(dir)) dir <- "." if (file.exists(dir)) { currentwd <- getwd() setwd(dir) on.exit(setwd(currentwd)) } else { stop(paste0("Directory \"", dir, "\" doesn't exist.")) } system_nm_default(cmd, ...) } get_data_path_from_ctl <- function(.mod){ mod_path <- get_model_path(.mod) ctl <- nmrec::read_ctl(mod_path) data_rec <- nmrec::select_records(ctl, "data")[[1]] data_path <- nmrec::get_record_option(data_rec, "filename")$value data_path_norm <- fs::path_norm(file.path(mod_path, data_path)) if(!fs::file_exists(data_path_norm)){ stop(glue("Could not find data at {data_path_norm}")) } return(data_path_norm) } modify_data_path_ctl <- function(mod_path, data_path){ # Get data record ctl <- nmrec::read_ctl(mod_path) data_rec <- nmrec::select_records(ctl, "data")[[1]] data_rec$parse() # Overwrite 'filename' option # TODO: confirm this works with .mod extensions data_rec$values <- purrr::map(data_rec$values, function(data_opt){ if(inherits(data_opt, "nmrec_option_pos") && data_opt$name == "filename"){ data_opt$value <- data_path } data_opt }) # Write out modified ctl nmrec::write_ctl(ctl, mod_path) } ```Example output
> run_nmtran(MOD1)
Running NMTRAN with NONMEM version `nm75`
WARNINGS AND ERRORS (IF ANY) FOR PROBLEM 1
(WARNING 2) NM-TRAN INFERS THAT THE DATA ARE POPULATION.
Note: Analytical 2nd Derivatives are constructed in FSUBS but are never used.
You may insert $ABBR DERIV2=NO after the first $PROB to save FSUBS construction and compilation timeOne unfortunate thing is that NMTRAN does require a valid data path to properly execute. While this makes sense, and is there for good reason, this does prevent us from testing this out on a few of our 'complex' models (within bbr), as we do not have the necessary dataset as part of this package.
Something to think about/look into:
- Are there options/args we can pass to
NMTRAN? If so, should we allow those commands?NMProjectallows users to specify a full command via nm_tran_command().
kyleam
This idea has been brought up numerous times, but the core idea is to provide a method for validating a
bbrmodel object viaNMTRANbefore submitting the model. There are other applications for this as well, such as validating a recently tweaked model, or just ensuringNONMEMsyntax is correct in general.Limitations
NMTRAN,bbrneeds to know where theNONMEMexecutable is. We can determine this via abbi.yaml, as well as via a user-provided.config_path(see?submit_modelarguments), thought this doesn't necessarily match up with the version of NONMEM used when actually submitting the model. This becausebbidetermines the path a little differently.bbi_init()be called before validating the model. This requirement may limit what functions we can nest arun_nmtran()call into (such as the new (in-flight)tweak_initial_estimatesfunction).Below is a prototype scaffold of how we can determine the
NMTRANexecutable path:Prototype functions
```r locate_nmtran <- function(.mod, .config_path = NULL, nmtran_exe = NULL){ if(is.null(nmtran_exe)){ model_dir <- get_model_working_directory(.mod) config_path <- .config_path %||% file.path(model_dir, "bbi.yaml") if(!file_exists(config_path)){ stop(paste("No bbi configuration was found in the execution directory.", "Please run `bbi_init()` with the appropriate directory to continue.")) } if (!is.null(.config_path)) { config_path <- normalizePath(.config_path) } bbi_config <- yaml::read_yaml(config_path) nm_config <- bbi_config$nonmem # look for default nonmem installation default_nm <- purrr::keep(nm_config, function(nm_ver){ !is.null(nm_ver$default) }) # Set nonmem path if(length(default_nm) > 0){ default_nm <- default_nm[[1]] }else{ # If no default, use the last one (likely higher version) default_nm <- nm_config[[length(nm_config)]] } # Set NMTRAN executable path nm_path <- default_nm$home nmtran_exe <- file.path(nm_path, "tr", "NMTRAN.exe") } if(!file_exists(nmtran_exe)){ stop(glue("Could not find an NMTRAN executable at `{nmtran_exe}`")) } return(nmtran_exe) } run_nmtran <- function(.mod, .config_path = NULL, nmtran_exe = NULL){ nmtran_exe <- locate_nmtran(.mod, .config_path, nmtran_exe) } ```