search

astamm / nloptr

nloptr provides an R interface to NLopt, a free/open-source library for nonlinear optimization providing a common interface to a number of different optimization routines which can handle nonlinear constraints and lower and upper bounds for the controls.
https://astamm.github.io/nloptr/
Other
106 stars 34 forks source link

Error when installing: `cannot find -lnlopt` #112

Closed MiguelRodo closed 2 years ago

MiguelRodo commented 2 years ago

Hi

Thank you for all the effort maintaining this package.

So, I'm attempting to install nloptr on a CentOS 7 server to which I do not have root access. Simply running install.packages("nloptr") did not work as cmake's version is too low.

However, it seemed like from some comments on here and on Stackoverflow that it's a good idea to install nlopt "manually", and then try install nloptr in R.

So, I asked the system administrator to install nlopt, which was done (version 2.7.1). Since this wasn't installed in a standard path, I set PKG_CONFIG_PATH=/opt/exp_soft/nlopt-2.7.1/lib64/pkgconfig. However, running install.packages("nloptr") still throws an error. The primary error message seems to be this:

/usr/bin/ld: cannot find -lnlopt

I presume that this is some library. However, I'm not sure how to specify this. I tried using the --with... arguments to config.vars as specified in issue 33, but had no success and was told that these arguments are not recognised.

Here is the entire printed output from running install.packages("nloptr"):

* installing *source* package ‘nloptr’ ...
** package ‘nloptr’ successfully unpacked and MD5 sums checked
** using non-staged installation
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ -std=gnu++14 accepts -g... yes
checking how to run the C++ preprocessor... g++ -std=gnu++14 -E
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ -std=gnu++14 accepts -g... (cached) yes
checking for pkg-config... /opt/exp_soft/miniconda3-py39-usr/bin/pkg-config
checking if pkg-config knows NLopt... yes
checking for pkg-config checking NLopt version... >= 2.7.0
configure: creating ./config.status
config.status: creating src/Makevars
** libs
gcc -I"/opt/exp_soft/R-4.1.1/lib64/R/include" -DNDEBUG -I../inst/include -I/opt/exp_soft/nlopt-2.7.1/include -I'/home/rdxmig002/programs/r/lib/4.1/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c init_nloptr.c -o init_nloptr.o
gcc -I"/opt/exp_soft/R-4.1.1/lib64/R/include" -DNDEBUG -I../inst/include -I/opt/exp_soft/nlopt-2.7.1/include -I'/home/rdxmig002/programs/r/lib/4.1/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c nloptr.c -o nloptr.o
g++ -std=gnu++11 -I"/opt/exp_soft/R-4.1.1/lib64/R/include" -DNDEBUG -I../inst/include -I/opt/exp_soft/nlopt-2.7.1/include -I'/home/rdxmig002/programs/r/lib/4.1/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c test-C-API.cpp -o test-C-API.o
g++ -std=gnu++11 -I"/opt/exp_soft/R-4.1.1/lib64/R/include" -DNDEBUG -I../inst/include -I/opt/exp_soft/nlopt-2.7.1/include -I'/home/rdxmig002/programs/r/lib/4.1/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c test-runner.cpp -o test-runner.o
g++ -std=gnu++11 -shared -L/opt/exp_soft/R-4.1.1/lib64/R/lib -L/usr/local/lib64 -o nloptr.so init_nloptr.o nloptr.o test-C-API.o test-runner.o -lopenblas -lgfortran -lm -lquadmath -L/opt/exp_soft/nlopt-2.7.1/lib -lnlopt -L/opt/exp_soft/R-4.1.1/lib64/R/lib -lR
/usr/bin/ld: cannot find -lnlopt
collect2: error: ld returned 1 exit status
make: *** [nloptr.so] Error 1
ERROR: compilation failed for package ‘nloptr’
* removing ‘/home/rdxmig002/programs/r/lib/4.1/nloptr’
* restoring previous ‘/home/rdxmig002/programs/r/lib/4.1/nloptr’

Here is the output from sessionInfo():

R version 4.1.1 (2021-08-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /usr/lib64/libopenblas-r0.3.3.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] alwaysloaded_0.2.0

loaded via a namespace (and not attached):
[1] compiler_4.1.1 magrittr_2.0.2 tools_4.1.1
eddelbuettel commented 2 years ago

You are close, and you seem to diagnose this correctly. The automated process ask pkg-config about the location, it does not seem to get the right answer.

As a one-off, you could always a) remove script configure and b) hand-edit src/Makevars from src/Makevars.in to suit your location.

eddelbuettel commented 2 years ago

Sorry, I was wrong / too fast here. pkg-config finds it: 

checking if pkg-config knows NLopt... yes
checking for pkg-config checking NLopt version... >= 2.7.0

What happens now looks like the dynamic (system-wide) linker not finding it. You can try setting LD_LIBRARY_PATH and alike; a higher-end solution on some systems is adding a file ending in .conf in the directory /etc/ld.so.conf.d/.

In simply terms, because we use dynamic linking, your nlopt installation needs to add its non-standard location to the linker's knowledge about paths to look in for it.

MiguelRodo commented 2 years ago

Great, thanks! I'm very new to these concepts, so I'll read up on your suggestions and revert (and thanks for the additional explanation).

eddelbuettel commented 2 years ago

It's more-or-less standard "non-standard" system administration. You made most of the right steps, happy to help for the last one.

MiguelRodo commented 2 years ago

Thanks! It's 6pm here in SA, so I'll have to get onto this tomorrow (you reply so quickly I felt I had to say that!:P thanks very much).

MiguelRodo commented 2 years ago

Hi again. So, I tried the tack of prepending two paths inside the nlopt folder to LD_LIBRARY_PATH. I added the following to .bashrc:

export LD_LIBRARY_PATH=/opt/exp_soft/nlopt-2.7.1/include:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/exp_soft/nlopt-2.7.1/lib64:$LD_LIBRARY_PATH

I suppose that only the lib64 folder is required, so I tried with adding the include folder and without adding the include folder and got the same result.

I get the same error as before (I think exactly), specifically cannot find -lnlopt. But it seems like that would be inside lib64 of the nlopt-2.7.1​ directory, as shown by ls /opt/exp_soft/nlopt-2.7.1/*​ yielding this:

/opt/exp_soft/nlopt-2.7.1/include:
nlopt.f  nlopt.h  nlopt.hpp

/opt/exp_soft/nlopt-2.7.1/lib64:
cmake  libnlopt.so  libnlopt.so.0  libnlopt.so.0.11.1  pkgconfig

/opt/exp_soft/nlopt-2.7.1/share:
man

I notice that the final line before the error message about not finding -libnlopt mentions /opt/exp_soft/nlopt-2.7.1/lib​. This folder doesn't exist, though - it's lib64. I'm not sure if this is a problem, since I think the -L in front means this is a link and I don't think the link name has to match the target (reading up on this seems to say that the actual target is stored "in" the link as text).

Should I handle LD_LIBRARY_PATH differently, or try one of its "friends" or asking the sysadmin to add that .conf file?

Thanks again for the help.

MiguelRodo commented 2 years ago

So, our sysadmin created a symlink from lib to lib64. The package now installs and runs!

So, in conclusion for anyone with one or all of our problems, here are how they were solved (others should feel free to suggest edits):

I would close the issue, but I'll leave that to the package maintainer as the lib64 problem seems like a bug and this message might merit editing.

eddelbuettel commented 2 years ago

Thanks for working through this. The lib != lib64 issue seems related to the (upstream) NLopt project and its package config output on CentOS. The nloptr package is just a consumer of that information and trusts it (and you appear to have corrected an infelicity in the right place).

MiguelRodo commented 2 years ago

Oh, okay, right. I'll fine an issue on the stevengh/nlopt repo then, letting them know about it. Thanks for the LD_LIBRARY_PATH suggestion and explaining what's happened!