Closed NeutralKaon closed 6 years ago
mattfidler
commented
6 years ago It seems that RxODE models are working OK; The failure you see in the RxODE tests tells me that the .so is not unloaded so it cannot delete the directory. This is not an issue that should affect your ability to fit data.
You can also see that the RxODE model compiles correctly by the following line:
Compiling RxODE differential equations...done.
The nlmixr saem piece seems to be failing.
mattfidler
commented
6 years ago Does the so saem14b432577bbc7.so exist?
NeutralKaon
commented
6 years ago Yes, it does:
-rwxr-xr-x 1 jackmiller staff 467792 1 May 15:47 saem14b432577bbc7.so
-rw-r--r-- 1 jackmiller staff 3738 1 May 15:47 saem14b432577bbc7.cppFitting with nlme (est="nlme") works fine. It's just saemix that has these problems.
Thanks hugely for your help!
mattfidler
commented
6 years ago You can try:
dyn.load("saem14b432577bbc7.so")To see if it loads now. If not you could try adding executable permissions
chmod +x saem14b432577bbc7.soAnd then try seeing if it loads in R with the command:
dyn.load("saem14b432577bbc7.so")It could need some delay before loading the so object...?
NeutralKaon
commented
6 years ago I think the error is stating that it's referencing another (random) library that isn't found:
dyn.load("saem14b432577bbc7.so")
Error in dyn.load("saem14b432577bbc7.so") :
unable to load shared object '/Users/jackmiller/RCrud/saem14b432577bbc7.so':
dlopen(/Users/jackmiller/RCrud/saem14b432577bbc7.so, 6): Library not loaded: rx_09391714b559fd25241d2dc1be0808ca_.so
Referenced from: /Users/jackmiller/RCrud/saem14b432577bbc7.so
Reason: image not foundThe library itself is executable (although I don't think you need this?)
Perhaps this more a problem with saemix?
mattfidler
commented
6 years ago saemix isn't used in nlmixr. But, yes it is a problem with the dll loading; It is salying that the library rx_093.... isn't loaded
mattfidler
commented
6 years ago The rx_ is the RxODE library.
mattfidler
commented
6 years ago As you stated in your example, it could be some permissions issue; I recently changed RxODE so that dlls are in the temporary directory. It could be that OSX doesn't allow you to load so files from the temporary directory (as a security precaution).
To test this, try the following:
options(RxODE.tempfiles=FALSE)
library(nlmixr)
one.compartment.saem <- function() {
ini({
tka <- .5
tcl <- -3.2
tv <- -1
eta.ka ~ 1
eta.cl ~ 2
eta.v ~ 1
add.err <- 0.1
})
model({
ka <- exp(tka + eta.ka)
cl <- exp(tcl + eta.cl)
v <- exp(tv + eta.v)
d/dt(depot) = -ka * depot
d/dt(center) = ka * depot - cl / v * center
cp = center / v
cp ~ add(add.err)
})
}
fit <- nlmixr(one.compartment.model, dat, est="saem")
kylebaron
commented
6 years ago It could be that OSX doesn't allow you to load so files from the temporary directory (as a security precaution).
@mattfidler This shouldn't be an issue; it is how mrgsolve works by default on any OS.
FWIW ...
This works for me with out of the box install from github
fit <- nlmixr(uif, theo_sd, est="saem", calc.resid = FALSE)This doesn't work for me:
one.compartment.saem <- function() {
ini({
tka <- .5
tcl <- -3.2
tv <- -1
eta.ka ~ 1
eta.cl ~ 2
eta.v ~ 1
add.err <- 0.1
})
model({
ka <- exp(tka + eta.ka)
cl <- exp(tcl + eta.cl)
v <- exp(tv + eta.v)
d/dt(depot) = -ka * depot
d/dt(center) = ka * depot - cl / v * center
cp = center / v
cp ~ add(add.err)
})
}fit <- nlmixr(one.compartment.saem, theo_sd, est="saem", calc.resid=FALSE)Same message as @NeutralKaon
Error in dyn.load(saem.dll) :
unable to load shared object '/Users/kyleb/demo/saemae1685f1355.so':
dlopen(/Users/kyleb/demo/saemae1685f1355.so, 6): Library not loaded: rx_d8b23d3408b5efd8ed0c261182ae3318_.so
Referenced from: /Users/kyleb/demo/saemae1685f1355.so
Reason: image not foundSession Details below
mattfidler
commented
6 years ago Researching this a bit, it could be due to the version of Xcode you are using; The linking mechanism may have changed. I know that some people have been running nlmixr fine on different versions of OSX/Xcode.
I have changed the linking between SAEM and RxODE to use R's methods (R_GetCCallable). My guess is if you update nlmixr this should be fixed.
mattfidler
commented
6 years ago Thanks @kylebmetrum it was helpful to see that it wasn't about loading dlls in the temporary directories.
NeutralKaon
commented
6 years ago Thanks @kylebmetrum and @mattfidler for your help Unfortunately, I'm still having this issue using your kindly provided bugfix -- see below.
```R > session_info() Session info -------------------------------------------------------------------------- setting value version R version 3.5.0 (2018-04-23) system x86_64, darwin15.6.0 ui RStudio (1.1.447) language (EN) collate en_GB.UTF-8 tz Europe/London date 2018-05-02 Packages ------------------------------------------------------------------------------ package * version date source base * 3.5.0 2018-04-24 local brew 1.0-6 2011-04-13 CRAN (R 3.5.0) colorspace 1.3-2 2016-12-14 CRAN (R 3.5.0) compiler 3.5.0 2018-04-24 local curl 3.2 2018-03-28 CRAN (R 3.5.0) datasets * 3.5.0 2018-04-24 local devtools * 1.13.5 2018-02-18 CRAN (R 3.5.0) digest 0.6.15 2018-01-28 CRAN (R 3.5.0) dparser 0.1.8 2017-11-13 CRAN (R 3.5.0) ggplot2 2.2.1 2016-12-30 CRAN (R 3.5.0) git2r 0.21.0 2018-01-04 CRAN (R 3.5.0) graphics * 3.5.0 2018-04-24 local grDevices * 3.5.0 2018-04-24 local grid 3.5.0 2018-04-24 local gtable 0.2.0 2016-02-26 CRAN (R 3.5.0) httr 1.3.1 2017-08-20 CRAN (R 3.5.0) knitr 1.20 2018-02-20 CRAN (R 3.5.0) lattice 0.20-35 2017-03-25 CRAN (R 3.5.0) lazyeval 0.2.1 2017-10-29 CRAN (R 3.5.0) lbfgs 1.2.1 2014-08-31 cran (@1.2.1) magrittr 1.5 2014-11-22 CRAN (R 3.5.0) memoise 1.1.0 2017-04-21 CRAN (R 3.5.0) methods * 3.5.0 2018-04-24 local munsell 0.4.3 2016-02-13 CRAN (R 3.5.0) mvnfast 0.2.5 2018-01-31 cran (@0.2.5) n1qn1 6.0.0-1 2017-10-30 cran (@6.0.0-1) nlme * 3.1-137 2018-04-07 CRAN (R 3.5.0) nlmixr * 0.9.1-0 2018-05-02 Github (nlmixrdevelopment/nlmixr@d7951b7) parallel 3.5.0 2018-04-24 local pillar 1.2.2 2018-04-26 CRAN (R 3.5.0) plyr 1.8.4 2016-06-08 CRAN (R 3.5.0) PreciseSums 0.3 2018-04-12 cran (@0.3) R6 2.2.2 2017-06-17 CRAN (R 3.5.0) Rcpp 0.12.16 2018-03-13 CRAN (R 3.5.0) RcppArmadillo 0.8.400.0.0 2018-03-01 CRAN (R 3.5.0) rex 1.1.2 2017-10-19 cran (@1.1.2) rlang 0.2.0 2018-02-20 CRAN (R 3.5.0) rstudioapi 0.7 2017-09-07 CRAN (R 3.5.0) RxODE * 0.7.1-0 2018-05-02 Github (nlmixrdevelopment/RxODE@b817487) scales 0.5.0 2017-08-24 CRAN (R 3.5.0) stats * 3.5.0 2018-04-24 local tibble 1.4.2 2018-01-22 CRAN (R 3.5.0) tools 3.5.0 2018-04-24 local utils * 3.5.0 2018-04-24 local vpc 1.0.1 2018-01-23 cran (@1.0.1) withr 2.1.2 2018-03-15 CRAN (R 3.5.0) ``` I don't think there's anything obviously wrong here.
The using temporary files flag alas doesn't fix the issue either.
```R > fit <- nlmixr(one.compartment.saem, theo_sd, est="saem", calc.resid=FALSE) Compiling RxODE differential equations...done. PKG_CXXFLAGS=-I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/nlmixr//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/StanHeaders//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppEigen//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/BH//include PKG_LIBS=/private/var/folders/5m/f1dkvq590bd4jx45ycptf0180000gn/T/RtmpS4sfNS/rx_ad375ebde6dadfe67515b54380eb4817_.so $(BLAS_LIBS) $(LAPACK_LIBS) $(FLIBS) /usr/local/clang4/bin/clang++ -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/usr/local/include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/nlmixr//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/StanHeaders//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppEigen//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/BH//include -fPIC -Wall -g -O2 -c saem1758a5d20f4c1.cpp -o saem1758a5d20f4c1.o /usr/local/clang4/bin/clang++ -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang6/lib -o saem1758a5d20f4c1.so saem1758a5d20f4c1.o /private/var/folders/5m/f1dkvq590bd4jx45ycptf0180000gn/T/RtmpS4sfNS/rx_ad375ebde6dadfe67515b54380eb4817_.so -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation Error in dyn.load(saem.dll) : unable to load shared object '/Users/jackmiller/RCrud/saem1758a5d20f4c1.so': dlopen(/Users/jackmiller/RCrud/saem1758a5d20f4c1.so, 6): Library not loaded: rx_ad375ebde6dadfe67515b54380eb4817_.so Referenced from: /Users/jackmiller/RCrud/saem1758a5d20f4c1.so Reason: image not found > options(RxODE.tempfiles=FALSE) > fit <- nlmixr(one.compartment.saem, theo_sd, est="saem", calc.resid=FALSE) Compiling RxODE differential equations...done. PKG_CXXFLAGS=-I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/nlmixr//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/StanHeaders//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppEigen//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/BH//include PKG_LIBS=/private/var/folders/5m/f1dkvq590bd4jx45ycptf0180000gn/T/RtmpS4sfNS/rx_ad375ebde6dadfe67515b54380eb4817_.so $(BLAS_LIBS) $(LAPACK_LIBS) $(FLIBS) /usr/local/clang4/bin/clang++ -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/usr/local/include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/nlmixr//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/StanHeaders//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppEigen//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/BH//include -fPIC -Wall -g -O2 -c saem1758a1cc8a88.cpp -o saem1758a1cc8a88.o /usr/local/clang4/bin/clang++ -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang6/lib -o saem1758a1cc8a88.so saem1758a1cc8a88.o /private/var/folders/5m/f1dkvq590bd4jx45ycptf0180000gn/T/RtmpS4sfNS/rx_ad375ebde6dadfe67515b54380eb4817_.so -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation Error in dyn.load(saem.dll) : unable to load shared object '/Users/jackmiller/RCrud/saem1758a1cc8a88.so': dlopen(/Users/jackmiller/RCrud/saem1758a1cc8a88.so, 6): Library not loaded: rx_ad375ebde6dadfe67515b54380eb4817_.so Referenced from: /Users/jackmiller/RCrud/saem1758a1cc8a88.so Reason: image not found ```
I'm using Xcode 9.3, but with clang4 for R following the advice of others for problems openmp:
```R jackmiller$ xcodebuild -version Xcode 9.3 Build version 9E145 jackmiller$ cat ~/.R/Makevars LDFLAGS=-L/usr/local/clang4/lib #-L/usr/local/lib CC=/usr/local/clang4/bin/clang CXX=/usr/local/clang4/bin/clang++ CXX1X=/usr/local/clang4/bin/clang++ CXX98=/usr/local/clang4/bin/clang++ CXX11=/usr/local/clang4/bin/clang++ CXX14=/usr/local/clang4/bin/clang++ CXX17=/usr/local/clang4/bin/clang++ ```
mattfidler
commented
6 years ago Could be an openmp issue?
RxODE has explicit support for openmp; RcppArmadillo also has support for openmp. This version of saem uses RcppArmadillo. Could you see what your global Makevars is; If you followed the advice here:
https://github.com/Rdatatable/data.table/wiki/Installation#openmp-enabled-compiler-for-mac
Then you may not have turned off global openmp support.
mattfidler
commented
6 years ago I'm sorry I didn't see your Makevars
Could you try to add the openmp flag (-fopenmp) to the Makevars; Perhaps because RxODE uses openmp, you need openmp enabled? Its a shot in the dark.
mattfidler
commented
6 years ago It could be the clang 4 you are using...?
From https://github.com/ContinuumIO/anaconda-issues/issues/6312
You could try the anaconda clang or another version of clang?
There they said:
BTW, if you add packages that need compilation you'd be best off using our compilers. They are very modern (clang 4.0.1) but also very compatible (macOS 10.9): conda install clang_osx-64 and/or conda install clangxx_osx-64
mattfidler
commented
6 years ago Another possibility is to turn off Integrety protection, as suggested by
NeutralKaon
commented
6 years ago Very oddly, upgrading to clang6 and using the CRAN version of RxODE works with -- but even after trying lots of different things such as installing Rccp, RcppEigen, RcppArmadillo etc from source, the github version still doesn't!
I've got SIP disabled and have tried a few different version of clang (and one attempt at g++). How odd. I might just end up using a docker version.
For the record, here are my cflags and the environment that "works" with the CRAN RxODE only
mattfidler
commented
6 years ago Thank you @NeutralKaon. The information that the CRAN RxODE works indeed is informative.
However, with the CRAN version of RxODE you will likely run into problems with the cran version of nlmixr. The new features in RxODE are now used to:
Thank you for your dedication. Hopefully you can help me debug this a bit further?
I have setup backward compatability tests with RxODE, with your new environment can you make sure they work:
library(RxODE)
rxTest("backward")If they indeed work (which they did above), then perhaps the latest nlmixr will work with your system, since I load and link the library before running saem.
NeutralKaon
commented
6 years ago Thanks again for your interest in this @mattfidler -- I know that these issues are hardly reproducible!
The version of RxODE on cran doesn't seem to have completely complete test support, at least on my system:
> library(RxODE)
> rxTest("backward")
Error in setwd(path) : cannot change working directoryThis is due to the path not existing; rxTest calls
Sys.setenv(RxODE_VALIDATION_FULL = "false", NOT_CRAN = "true")
path <- file.path(system.file("tests", package = "RxODE"),
"testthat")which returns null for me:
> system.file("tests", package = "RxODE")
[1] ""This is consistent with looking at the package location itself:
system.file("", package = "RxODE")
[1] "/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RxODE/"
list.files(system.file("", package = "RxODE"))
[1] "Changes.txt" "COPYRIGHTS" "demo" "DESCRIPTION"
[5] "doc" "help" "html" "include"
[9] "INDEX" "Install-windows.Rmd" "libs" "Meta"
[13] "NAMESPACE" "ode.c" "Porting" "R"
[17] "THANKS" "TODO.txt" "tran.g" This seems at odds with the git CRAN mirror that includes the test directory in the same version (0.6-1) of RxODE. I don't really understand this!
Here's what happens if I install the latest version and run the tests (they pass!)
Alas, we still can't get the linker to be happy:
> library(nlmixr)
Loading required package: nlme
> fit <- nlmixr(one.compartment.saem, theo_sd, est="saem", calc.resid=FALSE)
Compiling RxODE differential equations...done.
/usr/local/clang6/bin/clang++ -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/usr/local/include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/nlmixr//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/StanHeaders//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppEigen//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/BH//include -fPIC -Wall -g -O2 -c saem59972b5cfe24.cpp -o saem59972b5cfe24.o
/usr/local/clang6/bin/clang++ -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang6/lib -o saem59972b5cfe24.so saem59972b5cfe24.o /private/var/folders/5m/f1dkvq590bd4jx45ycptf0180000gn/T/Rtmp9tPifc/rx_d89150c945f1881d18e96e5c1e7c2a2b_.so -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
Error in dyn.load(saem.dll) :
unable to load shared object '/Users/jackmiller/RCrud/saem59972b5cfe24.so':
dlopen(/Users/jackmiller/RCrud/saem59972b5cfe24.so, 6): Library not loaded: rx_d89150c945f1881d18e96e5c1e7c2a2b_.so
Referenced from: /Users/jackmiller/RCrud/saem59972b5cfe24.so
Reason: image not foundReverting back to CRAN RxODE restores the expected behaviour:
> remove.packages("RxODE")
Removing package from ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library’
(as ‘lib’ is unspecified)
> install.packages("RxODE")
Error in install.packages : Updating loaded packages
Restarting R session...
> install.packages("RxODE")
trying URL 'https://mirrors.ebi.ac.uk/CRAN/bin/macosx/el-capitan/contrib/3.5/RxODE_0.6-1.tgz'
Content type 'application/x-gzip' length 882476 bytes (861 KB)
==================================================
downloaded 861 KB
The downloaded binary packages are in
/var/folders/5m/f1dkvq590bd4jx45ycptf0180000gn/T//Rtmp37twej/downloaded_packages
> library(RxODE)
> library(nlmixr)
Loading required package: nlme
> fit <- nlmixr(one.compartment.saem, theo_sd, est="saem", calc.resid=FALSE)
Compiling RxODE differential equations...done.
/usr/local/clang6/bin/clang++ -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/usr/local/include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/nlmixr//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/StanHeaders//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppEigen//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/BH//include -fPIC -Wall -g -O2 -c saem5b14302f5c9b.cpp -o saem5b14302f5c9b.o
/usr/local/clang6/bin/clang++ -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang6/lib -o saem5b14302f5c9b.so saem5b14302f5c9b.o /Users/jackmiller/RCrud/rx_2ffd24efce43ce34a0b5dbb3ef56fe36_.so -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
done.
1: 0.1503 -3.3314 -0.6447 0.9500 1.9000 0.9500 9.6167
2: 0.5210 -3.5986 -0.5154 0.9025 1.8050 0.9025 4.2880
3: 0.7840 -3.5040 -0.6070 0.8574 1.7147 0.8574 2.1259
<Snip>The version of RxODE on cran doesn't seem to have completely complete test support, at least on my system
Thanks for pointing this out; My guess is that CRAN takes out all the tests from their package repository.
You can get the tests and the full package from any github by specifying the revision:
devtools::install_github("nlmixrdevelopment/RxODE",ref="v0.6-1")I will look some more into this issue soon.
kylebaron
commented
6 years ago @mattfidler @NeutralKaon
To get the tests:
install.packages("RxODE", INSTALL_opts="--install-tests", type = "source")
They are in the source package (from CRAN) but you have to ask for them to be installed.
mattfidler
commented
6 years ago Thanks @kylebmetrum. That is very informative.
mattfidler
commented
6 years ago @NeutralKaon I tried one more thing; Update RxODE and let me know if it works for you.
If it doesn't, perhaps you can help me debug by running R with a debugger attached. I will give instructions.
NeutralKaon
commented
6 years ago Thank you again @kylebmetrum and @mattfidler -- I'm sorry for consuming so much of your time!
The tests from CRAN all pass. Alas, the newest version still fails with exactly the same linking problem, namely this one
> fit <- nlmixr(one.compartment.saem, theo_sd, est="saem", calc.resid=FALSE)
Compiling RxODE differential equations...done.
/usr/local/clang6/bin/clang++ -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/usr/local/include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/nlmixr//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/StanHeaders//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppEigen//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/BH//include -fPIC -Wall -g -O2 -c saem401f2e3bd75e.cpp -o saem401f2e3bd75e.o
/usr/local/clang6/bin/clang++ -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang6/lib -o saem401f2e3bd75e.so saem401f2e3bd75e.o /private/var/folders/5m/f1dkvq590bd4jx45ycptf0180000gn/T/Rtmpw5C5NV/rx_09a73ad424acd9687ff75c419f1e6d23_.so -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
Error in dyn.load(saem.dll) :
unable to load shared object '/Users/jackmiller/RCrud/saem401f2e3bd75e.so':
dlopen(/Users/jackmiller/RCrud/saem401f2e3bd75e.so, 6): Library not loaded: rx_09a73ad424acd9687ff75c419f1e6d23_.so
Referenced from: /Users/jackmiller/RCrud/saem401f2e3bd75e.so
Reason: image not foundwith this session info
> session_info()
Session info ----------------------------------------------------------------------------------------
setting value
version R version 3.5.0 (2018-04-23)
system x86_64, darwin15.6.0
ui RStudio (1.1.447)
language (EN)
collate en_GB.UTF-8
tz Europe/London
date 2018-05-07
Packages --------------------------------------------------------------------------------------------
package * version date source
base * 3.5.0 2018-04-24 local
brew 1.0-6 2011-04-13 CRAN (R 3.5.0)
colorspace 1.3-2 2016-12-14 CRAN (R 3.5.0)
compiler 3.5.0 2018-04-24 local
datasets * 3.5.0 2018-04-24 local
devtools * 1.13.5 2018-02-18 CRAN (R 3.5.0)
digest 0.6.15 2018-01-28 CRAN (R 3.5.0)
dparser 0.1.8 2017-11-13 CRAN (R 3.5.0)
ggplot2 2.2.1 2016-12-30 CRAN (R 3.5.0)
graphics * 3.5.0 2018-04-24 local
grDevices * 3.5.0 2018-04-24 local
grid 3.5.0 2018-04-24 local
gtable 0.2.0 2016-02-26 CRAN (R 3.5.0)
lattice 0.20-35 2017-03-25 CRAN (R 3.5.0)
lazyeval 0.2.1 2017-10-29 CRAN (R 3.5.0)
lbfgs 1.2.1 2014-08-31 cran (@1.2.1)
magrittr 1.5 2014-11-22 CRAN (R 3.5.0)
memoise 1.1.0 2017-04-21 CRAN (R 3.5.0)
methods * 3.5.0 2018-04-24 local
munsell 0.4.3 2016-02-13 CRAN (R 3.5.0)
mvnfast 0.2.5 2018-01-31 cran (@0.2.5)
n1qn1 6.0.0-1 2017-10-30 cran (@6.0.0-1)
nlme * 3.1-137 2018-04-07 CRAN (R 3.5.0)
nlmixr * 0.9.0-3 2018-02-07 CRAN (R 3.5.0)
parallel 3.5.0 2018-04-24 local
pillar 1.2.2 2018-04-26 CRAN (R 3.5.0)
plyr 1.8.4 2016-06-08 CRAN (R 3.5.0)
PreciseSums 0.3 2018-04-12 cran (@0.3)
Rcpp 0.12.16.3 2018-05-07 Github (RcppCore/Rcpp@adfaf2c)
rex 1.1.2 2017-10-19 cran (@1.1.2)
rlang 0.2.0 2018-02-20 CRAN (R 3.5.0)
rstudioapi 0.7 2017-09-07 CRAN (R 3.5.0)
RxODE * 0.7.1-0 2018-05-07 Github (nlmixrdevelopment/rxode@1568cb0)
scales 0.5.0 2017-08-24 CRAN (R 3.5.0)
stats * 3.5.0 2018-04-24 local
tibble 1.4.2 2018-01-22 CRAN (R 3.5.0)
tools 3.5.0 2018-04-24 local
utils * 3.5.0 2018-04-24 local
withr 2.1.2 2018-03-15 CRAN (R 3.5.0)
What is your LD_LIBRARY_PATH and DYLD_LIBRARY_PATH in R?
mattfidler
commented
6 years ago Thank you for taking the time to debug this with me.
Lets go slowly through the error to try to pinpoint why your particular setup isn't linking correctly.
First, output the RxODE compilation process to the console; This is done by setting the following option before loading RxODE:
options(RxODE.echo.compile=TRUE)When running see if there is any clang warnings that we should investigate.
Second after running nlmixr, and getting the error could you tell me the output of the command:
getLoadedDLLs()In theory it should have the DLL that we are trying to link to...
mattfidler
commented
6 years ago Currently the github version of RxODE now passes the maverick Mac builds, so perhaps it will work for you @NeutralKaon ?
The checks are located:
mattfidler
commented
6 years ago Also loaded symbol table into global scope; See http://www.stat.ucdavis.edu/~duncan/R/dynload/
mattfidler
commented
6 years ago To test you will have to update both RxODE and nlmixr.
NeutralKaon
commented
6 years ago Thanks again for your help.
First off, at the moment there's a missing ) at the end of line 30 of nlmixr/R/simulate.R. Alas, even so with that I still get the same blasted error! The compiled library is indeed listed in GetLoadedDLLs, and there are no clang errors:
> options(RxODE.echo.compile=TRUE)
> fit<-nlmixr(one.compartment.saem, IVGTTData, est="saem")
Compiling RxODE differential equations...done.
PKG_CXXFLAGS=-I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/nlmixr//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/StanHeaders//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppEigen//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/BH//include
PKG_LIBS=/var/folders/5m/f1dkvq590bd4jx45ycptf0180000gn/T//RtmpsLys7T/rx_65951daa2e04964c1ecb5d77c176de53_.so $(BLAS_LIBS) $(LAPACK_LIBS) $(FLIBS)
/usr/local/clang6/bin/clang++ -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/usr/local/include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/nlmixr//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/StanHeaders//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppEigen//include -I/Library/Frameworks/R.framework/Versions/3.5/Resources/library/BH//include -fPIC -Wall -g -O2 -c saem15f071d2e298.cpp -o saem15f071d2e298.o
/usr/local/clang6/bin/clang++ -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang6/lib -o saem15f071d2e298.so saem15f071d2e298.o /var/folders/5m/f1dkvq590bd4jx45ycptf0180000gn/T//RtmpsLys7T/rx_65951daa2e04964c1ecb5d77c176de53_.so -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
Error in dyn.load(saem.dll) :
unable to load shared object '/Users/jackmiller/RCrud/saem15f071d2e298.so':
dlopen(/Users/jackmiller/RCrud/saem15f071d2e298.so, 6): Library not loaded: rx_65951daa2e04964c1ecb5d77c176de53_.so
Referenced from: /Users/jackmiller/RCrud/saem15f071d2e298.so
Reason: image not found
> getLoadedDLLs()
Filename
base base
methods /Library/Frameworks/R.framework/Versions/3.5/Resources/library/methods/libs/methods.so
utils /Library/Frameworks/R.framework/Versions/3.5/Resources/library/utils/libs/utils.so
grDevices /Library/Frameworks/R.framework/Versions/3.5/Resources/library/grDevices/libs/grDevices.so
graphics /Library/Frameworks/R.framework/Versions/3.5/Resources/library/graphics/libs/graphics.so
stats /Library/Frameworks/R.framework/Versions/3.5/Resources/library/stats/libs/stats.so
tools /Library/Frameworks/R.framework/Versions/3.5/Resources/library/tools/libs/tools.so
internet /Library/Frameworks/R.framework/Resources/modules//internet.so
(embedding) (embedding)
parallel /Library/Frameworks/R.framework/Versions/3.5/Resources/library/parallel/libs/parallel.so
Biobase /Library/Frameworks/R.framework/Versions/3.5/Resources/library/Biobase/libs/Biobase.so
S4Vectors /Library/Frameworks/R.framework/Versions/3.5/Resources/library/S4Vectors/libs/S4Vectors.so
IRanges /Library/Frameworks/R.framework/Versions/3.5/Resources/library/IRanges/libs/IRanges.so
bitops /Library/Frameworks/R.framework/Versions/3.5/Resources/library/bitops/libs/bitops.so
RCurl /Library/Frameworks/R.framework/Versions/3.5/Resources/library/RCurl/libs/RCurl.so
XVector /Library/Frameworks/R.framework/Versions/3.5/Resources/library/XVector/libs/XVector.so
GenomicRanges /Library/Frameworks/R.framework/Versions/3.5/Resources/library/GenomicRanges/libs/GenomicRanges.so
matrixStats /Library/Frameworks/R.framework/Versions/3.5/Resources/library/matrixStats/libs/matrixStats.so
BiocParallel /Library/Frameworks/R.framework/Versions/3.5/Resources/library/BiocParallel/libs/BiocParallel.so
grid /Library/Frameworks/R.framework/Versions/3.5/Resources/library/grid/libs/grid.so
lattice /Library/Frameworks/R.framework/Versions/3.5/Resources/library/lattice/libs/lattice.so
Matrix /Library/Frameworks/R.framework/Versions/3.5/Resources/library/Matrix/libs/Matrix.so
rlang /Library/Frameworks/R.framework/Versions/3.5/Resources/library/rlang/libs/rlang.so
Rcpp /Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp/libs/Rcpp.so
glue /Library/Frameworks/R.framework/Versions/3.5/Resources/library/glue/libs/glue.so
purrr /Library/Frameworks/R.framework/Versions/3.5/Resources/library/purrr/libs/purrr.so
tibble /Library/Frameworks/R.framework/Versions/3.5/Resources/library/tibble/libs/tibble.so
tidyr /Library/Frameworks/R.framework/Versions/3.5/Resources/library/tidyr/libs/tidyr.so
deSolve /Library/Frameworks/R.framework/Versions/3.5/Resources/library/deSolve/libs/deSolve.so
digest /Library/Frameworks/R.framework/Versions/3.5/Resources/library/digest/libs/digest.so
htmltools /Library/Frameworks/R.framework/Versions/3.5/Resources/library/htmltools/libs/htmltools.so
later /Library/Frameworks/R.framework/Versions/3.5/Resources/library/later/libs/later.so
promises /Library/Frameworks/R.framework/Versions/3.5/Resources/library/promises/libs/promises.so
httpuv /Library/Frameworks/R.framework/Versions/3.5/Resources/library/httpuv/libs/httpuv.so
mime /Library/Frameworks/R.framework/Versions/3.5/Resources/library/mime/libs/mime.so
colorspace /Library/Frameworks/R.framework/Versions/3.5/Resources/library/colorspace/libs/colorspace.so
plyr /Library/Frameworks/R.framework/Versions/3.5/Resources/library/plyr/libs/plyr.so
scales /Library/Frameworks/R.framework/Versions/3.5/Resources/library/scales/libs/scales.so
lazyeval /Library/Frameworks/R.framework/Versions/3.5/Resources/library/lazyeval/libs/lazyeval.so
bindrcpp /Library/Frameworks/R.framework/Versions/3.5/Resources/library/bindrcpp/libs/bindrcpp.so
dplyr /Library/Frameworks/R.framework/Versions/3.5/Resources/library/dplyr/libs/dplyr.so
datatable /Library/Frameworks/R.framework/Versions/3.5/Resources/library/data.table/libs/datatable.so
jsonlite /Library/Frameworks/R.framework/Versions/3.5/Resources/library/jsonlite/libs/jsonlite.so
RcppArmadillo /Library/Frameworks/R.framework/Versions/3.5/Resources/library/RcppArmadillo/libs/RcppArmadillo.so
PreciseSums /Library/Frameworks/R.framework/Versions/3.5/Resources/library/PreciseSums/libs/PreciseSums.so
mvnfast /Library/Frameworks/R.framework/Versions/3.5/Resources/library/mvnfast/libs/mvnfast.so
RxODE /Library/Frameworks/R.framework/Versions/3.5/Resources/library/RxODE/libs/RxODE.so
lbfgs /Library/Frameworks/R.framework/Versions/3.5/Resources/library/lbfgs/libs/lbfgs.so
n1qn1 /Library/Frameworks/R.framework/Versions/3.5/Resources/library/n1qn1/libs/n1qn1.so
nlme /Library/Frameworks/R.framework/Versions/3.5/Resources/library/nlme/libs/nlme.so
nlmixr /Library/Frameworks/R.framework/Versions/3.5/Resources/library/nlmixr/libs/nlmixr.so
lapack /Library/Frameworks/R.framework/Resources/modules//lapack.so
dparser /Library/Frameworks/R.framework/Versions/3.5/Resources/library/dparser/libs/dparser.so
rx_65951daa2e04964c1ecb5d77c176de53_ /var/folders/5m/f1dkvq590bd4jx45ycptf0180000gn/T//RtmpsLys7T/rx_65951daa2e04964c1ecb5d77c176de53_.so
Dynamic.Lookup
base FALSE
methods FALSE
utils FALSE
grDevices FALSE
graphics FALSE
stats FALSE
tools FALSE
internet TRUE
(embedding) FALSE
parallel FALSE
Biobase TRUE
S4Vectors TRUE
IRanges TRUE
bitops TRUE
RCurl TRUE
XVector TRUE
GenomicRanges TRUE
matrixStats FALSE
BiocParallel TRUE
grid FALSE
lattice FALSE
Matrix FALSE
rlang FALSE
Rcpp TRUE
glue FALSE
purrr FALSE
tibble FALSE
tidyr FALSE
deSolve FALSE
digest TRUE
htmltools FALSE
later FALSE
promises FALSE
httpuv FALSE
mime TRUE
colorspace TRUE
plyr TRUE
scales FALSE
lazyeval FALSE
bindrcpp FALSE
dplyr FALSE
datatable FALSE
jsonlite TRUE
RcppArmadillo FALSE
PreciseSums FALSE
mvnfast FALSE
RxODE FALSE
lbfgs TRUE
n1qn1 TRUE
nlme FALSE
nlmixr TRUE
lapack TRUE
dparser FALSE
rx_65951daa2e04964c1ecb5d77c176de53_ FALSEFor a colleague of mine running a Mac, commit 90188b4062a80d7b7c0d79972847c7740e2c4276 is the first bad commit. However, I don't really see anything here that should affect loading/unloading with the exception of the temporary directory. Kyle has already stated that the temporary directory wasn't an issue with mrgsolve.
However the nlmixr saem dlls in the current directory are linked to RxODE dlls in the temporary directory. mrgsolve only creates a dll in a temporary directory so it isn't exactly the same situation here.
So, going back to the temporary directory possibility, did you try:
options(RxODE.tempfiles=FALSE)
library(nlmixr)
one.compartment.saem <- function() {
ini({
tka <- .5
tcl <- -3.2
tv <- -1
eta.ka ~ 1
eta.cl ~ 2
eta.v ~ 1
add.err <- 0.1
})
model({
ka <- exp(tka + eta.ka)
cl <- exp(tcl + eta.cl)
v <- exp(tv + eta.v)
d/dt(depot) = -ka * depot
d/dt(center) = ka * depot - cl / v * center
cp = center / v
cp ~ add(add.err)
})
}
fit <- nlmixr(one.compartment.model, dat, est="saem")If this works, it means:
mattfidler
commented
6 years ago I have a 6 year old mac I just got temporary access to (it has a few problems). I am upgrading the mac to High Sierra and seeing if I can reproduce your problems (or even upgrade the system).
mattfidler
commented
6 years ago Still installing xcode. However I was informed that the temporary directory seemed to have no impact on this issue.
mattfidler
commented
6 years ago 54bd513f1e4d1ea271d688ca7840fe3787fc882f is the first bad commit according to my bisection.
mattfidler
commented
6 years ago The bisection was a bit misleading for me.
I found that the default instructions include the most recent gfortran, but I didn't have the gfortran library installed in the same location that R used.
Apparently this is a common OSX issue:
The strange thing is RxODE 0.6 worked and compiling RxODE (which has fortran in it) also works
You may have already tried this (I see you link to both /usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 and /usr/local/gfortran in your compile script.
My guess is /usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 does not actually exist in your system, just like it doesn't exist in my old mac.
To fix the linking issues, I took out C linking to the RxODE dll as well as linking to the fortran libraries; It seems to work for me on R 3.5 on Mac and windows. I will test ubuntu shortly.
You will have to update nlmixr and RxODE to test.
The corresponding commits in nlmixr are:
mattfidler
commented
6 years ago Ubuntu also works for me.
NeutralKaon
commented
6 years ago @mattfidler Thank you SO SO much for going to such a lot of effort to squash this bug -- your changes to linking have indeed worked and it compiles code happily. It is now squashed!
I had indeed investigated gfortran as part of the above, and I do have an actual extant version of it installed:
$ ls -l $(which gfortran)
lrwxr-xr-x 1 root staff 32 15 Aug 2017 /usr/local/bin/gfortran -> /usr/local/gfortran/bin/gfortran
$ ls /usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0/
crt3.o crtprec32.o crtprec80.o crttms.o i386 include-fixed install-tools libcaf_single.la libgcc_eh.a plugin
crtfastmath.o crtprec64.o crttme.o finclude include include-gnu-runtime libcaf_single.a libgcc.a libgcov.aHowever, I additionally have macports's installation at /opt/local/bin/gfortran-mp-6 and /opt/local/lib/. Probably the wrong one was getting linked against.
I really am incredibly grateful for this -- just to check, at present the citaiton() commands for both nlmixr and rxode don't correspond to a journal article -- is that still correct or is there anything in the works? I will be using it to fit simple ODE models in the context of hyperpolarised NMR / MRI and will definitely acknowledge you in the resulting article (which will hopefully exist!).
Thanks again!
mattfidler
commented
6 years ago RxODE has a journal article, possibly with more to come (we would be talking about some of the new features such as threaded solving).
https://dx.doi.org/10.1002%2Fpsp4.12052
nlmixr will have a journal article, though we haven't finalized or submitted an article yet
Thanks for confirming this for me!
I've installed RxODE and nlmixr on Mac OS (10.13.4) with R 3.5 as per the instructions here. I'm using the current (github) versions of both RxODE and nlmixr. Thanks again for your work on this great tool.
Some things work but for reasons I don't entirely understand quite a lot of saemix's compile code generates unknown linking issues. I believe this is probably a glorified permissions or path error in disguise.
Here's a snippet of the full rxTest() output:
If I try and run the example fitting (to the theophylline data) with
saem, I get similar woes:Am I doing something stupidly wrong?