MacOS: No way to use PKG_LIBS_STATIC defined in configure script

[lentils]

Hello,

I'm trying to build a statically linked version of jqr for MacOS (like what's built on CRAN) and I'm running into an issue with configure and PKG_LIBS used in compilation.

First I brew install jq, which installs oniguruma as a dependency. Then I run:

R --vanilla CMD install jqr_1.0.0.tar.gz -l /Users/jenkins/test-r-pkg-build --build --preclean

For dynamic linking (ie using .dylib), I have no problems and can compile fine. It's only when I try static linking (ie using .a files) where I run into issue.

For static linking, I remove the .dylib files for both libjq and libonig with the following:

rm -f /usr/local/opt/jq/lib/*.dylib
rm -f /usr/local/opt/oniguruma/lib/*.dylib

Then, when I build, I get the following:

* installing *source* package 'jqr' ...
** package 'jqr' successfully unpacked and MD5 sums checked
Using PKG_CFLAGS=-I/usr/local/opt/jq/include
Using PKG_LIBS=-L/usr/local/lib -ljq
** libs
rm -f jqr.so jqr.o register.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/jq/include  -I/usr/local/include   -fPIC  -Wall -g -O2  -c jqr.c -o jqr.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/jq/include  -I/usr/local/include   -fPIC  -Wall -g -O2  -c register.c -o register.o
/usr/local/clang4/bin/clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -Xlinker -v -L/usr/local/clang4/lib -L/usr/local/gfortran/lib -L/opt/X11/lib -o jqr.so jqr.o register.o -L/usr/local/lib -ljq -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
@(#)PROGRAM:ld  PROJECT:ld64-278.4
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em (tvOS)
Library search paths:
    /Library/Frameworks/R.framework/Resources/lib
    /usr/local/clang4/lib
    /usr/local/gfortran/lib
    /opt/X11/lib
    /usr/local/lib
    /usr/lib
    /usr/local/lib
Framework search paths:
    /Library/Frameworks
    /Library/Frameworks/
    /System/Library/Frameworks/
installing to /Users/jenkins/test-r-pkg-build/jqr/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
Error: package or namespace load failed for 'jqr' in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/jenkins/test-r-pkg-build/jqr/libs/jqr.so':
  dlopen(/Users/jenkins/test-r-pkg-build/jqr/libs/jqr.so, 6): Symbol not found: _OnigEncodingUTF8
  Referenced from: /Users/jenkins/test-r-pkg-build/jqr/libs/jqr.so
  Expected in: flat namespace
 in /Users/jenkins/test-r-pkg-build/jqr/libs/jqr.so
Error: loading failed
Execution halted
ERROR: loading failed

The issue seems to be the fact that jqr uses PKG_LIBS=-L/usr/local/lib -ljq instead of PKG_LIBS=-L/usr/local/lib -ljq -lonig. The former is fine when you have dynamic linking because libjq.dylib is dynamically linked to libonig.dylib, so the dependency is resolved at runtime. With static linking, the compiler must know to look for and include libonig at compile time.

I was able to build a statically linked version of jqr successfully by overwriting PKG_LIBS in Makevars, but that isn't a very good solution when you're building lots of packages at once. R seems to recommend using --configure-vars as proper way to customize package builds.

Looking at https://github.com/ropensci/jqr/blob/master/configure#L15 I see you do have the flag I need in your configure script, but I don't see any reference to that variable inside the script. This seems like a bug to me.

So, my questions are: 1) Is this the right approach statically compile jqr? 2) Is this a bug in configure?

I'm happy to submit a PR addressing this, but I haven't written too many R packages before. I would expect a fix to include a flag like --enable-static that configure would consume to choose the right value for PKG_LIBS. Let me know what you think about this approach.

Thanks for taking the time to read my question. Appreciate any feedback you have here!

Here's the ouput of sessionInfo as well:

> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_3.4.1

[jeroen]

What is it you are trying to do? Why not just use the binary package from CRAN?

[lentils]

Hi jeroen,

Good question. The goal is to build R packages that are as portable as possible, hence the statically compiled package. At my organization, we host everything internally to avoid unexpected outages by CRAN mirrors. We also write our own R packages for internal use only, so they must be built and hosted outside of CRAN. We use the same process for building both CRAN packages and our own internal-only packages.

For this particular case, we could certainly use CRAN's version, but ideally we can use the same mechanism to build and deploy any package. I suppose part of my question in this issue is whether or not I'm approaching statically compiled packages correctly, in the first place.

Hope the context helps. Thanks again.

[sckott]

@lentils is this still an issue for you?

[lentils]

Hi @sckott,

Thanks for checking in. As far as I can tell, the original problem of not being able to build with -ljq -lonig remains. Like I said, it's defined in configure as PKG_LIBS_STATIC but I can't figure out how to get R to build jqr using that variable without modifying the source directly. Maybe I'm misunderstanding how to configure R's build process properly.

That said, this not as important of an issue for us as it was a few months ago. At this point, I'm mostly looking to satisfy my curiosity. I can't tell if this is a bug or user error on my part! Clearly the package is able to statically linked, since it's that way on CRAN, but I can't reproduce that same build artifact myself.

[sckott]

thanks @lentils - @jeroen can you chime in here. i'm not quite sure how to help on this

[jeroen]

Maybe you can try passing them via LIB_DIR?

install.packages('jqr', type = 'source', configure.vars = 'LIB_DIR="/usr/local/lib -ljq -lonig"')

[sckott]

ping @lentils a suggestion from jeroen above