search

mrc-ide / heartbeatr

:heartpulse::computer::heartpulse: Redis heartbeat support
https://mrc-ide.github.io/heartbeatr
Other
2 stars 1 forks source link

problem installing RedisHeartbeat #5

Closed jscamac closed 4 years ago

jscamac commented 8 years ago

Hi @richfitz I'm having some issues installing RedisHeartbeat. Below are details: Im using OSX 10.11.3. R version 3.2.4 Redis 3.0.4 Hiredis 0.13.3

When I install usingdevtools::install_github("richfitz/RedisHeartbeat") I get:

Downloading GitHub repo richfitz/RedisHeartbeat@master
from URL https://api.github.com/repos/richfitz/RedisHeartbeat/zipball/master
Installing RedisHeartbeat
Skipping 2 unavailable packages: RedisAPI, redux
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ  \
  --no-save --no-restore CMD INSTALL  \
  '/private/var/folders/gx/kvkkz_rs42b5x6nj2wq5p6880000gn/T/Rtmp7Ox3pI/devtools346d76578c7/richfitz-RedisHeartbeat-4ce0b55'  \
  --library='/Library/Frameworks/R.framework/Versions/3.2/Resources/library'  \
  --install-tests 

* installing *source* package ‘RedisHeartbeat’ ...
** libs
clang++ -arch x86_64 -ftemplate-depth-256 -I/Library/Frameworks/R.framework/Resources/include     -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include -I"/Library/Frameworks/R.framework/Versions/3.2/Resources/library/Rcpp/include"  -D_FILE_OFFSET_BITS=64 -I/usr/local/include -I../inst/include -fPIC  -O3 -c RcppExports.cpp -o RcppExports.o
clang++ -arch x86_64 -ftemplate-depth-256 -I/Library/Frameworks/R.framework/Resources/include     -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include -I"/Library/Frameworks/R.framework/Versions/3.2/Resources/library/Rcpp/include"  -D_FILE_OFFSET_BITS=64 -I/usr/local/include -I../inst/include -fPIC  -O3 -c heartbeat.cpp -o heartbeat.o
clang++ -arch x86_64 -ftemplate-depth-256 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o RedisHeartbeat.so RcppExports.o heartbeat.o -L/usr/local/lib -lhiredis -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Library/Frameworks/R.framework/Versions/3.2/Resources/library/RedisHeartbeat/libs
** R
** inst
** tests
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/Library/Frameworks/R.framework/Versions/3.2/Resources/library/RedisHeartbeat/libs/RedisHeartbeat.so':
  dlopen(/Library/Frameworks/R.framework/Versions/3.2/Resources/library/RedisHeartbeat/libs/RedisHeartbeat.so, 6): Library not loaded: libhiredis.0.13.dylib
  Referenced from: /Library/Frameworks/R.framework/Versions/3.2/Resources/library/RedisHeartbeat/libs/RedisHeartbeat.so
  Reason: image not found
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/3.2/Resources/library/RedisHeartbeat’
Error: Command failed (1)

redis and hiredis are installed and I've added DYLD_LIBRARY_PATH=/usr/local/lib to my profile.

echo $DYLD_LIBRARY_PATH 
/usr/local/lib

The missing library also appears to be present:

➜  ~  ls $DYLD_LIBRARY_PATH/libhi*
/usr/local/lib/libhiredis.0.13.dylib /usr/local/lib/libhiredis.dylib
/usr/local/lib/libhiredis.a

Any ideas what is going on here?

jscamac commented 8 years ago

Also here is the output when running make followed by sudo make install in the latest hiredis release

cc -std=c99 -pedantic -c -O3 -fPIC  -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb  net.c
cc -std=c99 -pedantic -c -O3 -fPIC  -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb  hiredis.c
cc -std=c99 -pedantic -c -O3 -fPIC  -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb  sds.c
cc -std=c99 -pedantic -c -O3 -fPIC  -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb  async.c
cc -std=c99 -pedantic -c -O3 -fPIC  -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb  read.c
cc -shared -Wl,-install_name,libhiredis.0.13.dylib -o libhiredis.dylib  net.o hiredis.o sds.o async.o read.o
ar rcs libhiredis.a net.o hiredis.o sds.o async.o read.o
cc -std=c99 -pedantic -c -O3 -fPIC  -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb  test.c
cc -O3 -fPIC  -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb  -o hiredis-test   test.o libhiredis.a
Generating hiredis.pc for pkgconfig...
mkdir -p /usr/local/include/hiredis /usr/local/lib
cp -a hiredis.h async.h read.h sds.h adapters /usr/local/include/hiredis
cp -a libhiredis.dylib /usr/local/lib/libhiredis.0.13.dylib
cd /usr/local/lib && ln -sf libhiredis.0.13.dylib libhiredis.dylib
cp -a libhiredis.a /usr/local/lib
mkdir -p /usr/local/lib/pkgconfig
cp -a hiredis.pc /usr/local/lib/pkgconfig
jscamac commented 8 years ago

Ok we've discovered what the problem is. For some reason z shell correctly finds the DYLD_LIBRARY_PATH at /usr/local/lib

However, in R this path doesn't exist. No idea what is going on.

jscamac commented 8 years ago

Ok this may be an issue with El Captain no longer allowing you to set the DYLD path. https://github.com/oracle/node-oracledb/issues/231 https://groups.google.com/forum/#!topic/caffe-users/waugt62RQMU http://www.postgresql.org/message-id/561E73AB.8060800@gmx.net

dfalster commented 8 years ago

Hmm, well diagnosed James. This does indeed seem to be a problem with El Capitan. Following this thread, for another piece of software with similar issue, i came up with this MWE to illustrate the problem, which is that path variables are not being forwarded when spawning new shells:

echo $DYLD_LIBRARY_PATH 
sh -c 'echo $DYLD_LIBRARY_PATH'
Rscript -e 'system("echo $DYLD_LIBRARY_PATH")'

Prediction is that the last two won't work on James's laptop.

RESULTS: On my Imac (Yosemite):

➜  ~  echo $DYLD_LIBRARY_PATH 
/usr/local/lib
➜  ~  sh -c 'echo $DYLD_LIBRARY_PATH'
/usr/local/lib
➜  ~  Rscript -e 'system("echo $DYLD_LIBRARY_PATH")'
/usr/local/lib

On James's imac (Yosemite): 

➜  ~  echo $DYLD_LIBRARY_PATH 
/usr/local/lib
➜  ~  sh -c 'echo $DYLD_LIBRARY_PATH'
/usr/local/lib
➜  ~  Rscript -e 'system("echo $DYLD_LIBRARY_PATH")'
/usr/local/lib

On James's laptop (El Capitan): 

➜  ~  echo $DYLD_LIBRARY_PATH 
/usr/local/lib
➜  ~  sh -c 'echo $DYLD_LIBRARY_PATH'

➜  ~  Rscript -e 'system("echo $DYLD_LIBRARY_PATH")'

➜  ~
dfalster commented 8 years ago

@richfitz It seems likely you've encountered this issue elsewhere by now. Suggested fixes?

dfalster commented 8 years ago

One possible fix is to disable the "System Integrity Protection in El Capitan" -- http://www.macworld.com/article/2986118/security/how-to-modify-system-integrity-protection-in-el-capitan.html

richfitz commented 8 years ago

Can you try setting this via Renviron? Traveling at the moment and don't actually have an El cap install

jscamac commented 8 years ago

ok. Added DYLD_LIBRARY_PATH=/usr/local/lib to ~/.Renviron

Doing this R now can see the correct path

➜  ~  Rscript -e ' Sys.getenv("DYLD_LIBRARY_PATH")' 
[1] "/usr/local/lib"

However, still cannot install RedisHeartbeat

devtools::install_github('richfitz/RedisHeartbeat')
Downloading GitHub repo richfitz/RedisHeartbeat@master
from URL https://api.github.com/repos/richfitz/RedisHeartbeat/zipball/master
Installing RedisHeartbeat
Skipping 2 unavailable packages: RedisAPI, redux
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ  \
  --no-save --no-restore CMD INSTALL  \
  '/private/var/folders/gx/kvkkz_rs42b5x6nj2wq5p6880000gn/T/RtmpHlIzSV/devtools1eec73787f74/richfitz-RedisHeartbeat-4ce0b55'  \
  --library='/Library/Frameworks/R.framework/Versions/3.2/Resources/library'  \
  --install-tests 

* installing *source* package ‘RedisHeartbeat’ ...
** libs
clang++ -arch x86_64 -ftemplate-depth-256 -I/Library/Frameworks/R.framework/Resources/include     -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include -I"/Library/Frameworks/R.framework/Versions/3.2/Resources/library/Rcpp/include"  -D_FILE_OFFSET_BITS=64 -I/usr/local/include -I../inst/include -fPIC  -O3 -Wall -pedantic -Wextra -Wno-padded -Wunused -isystem /Library/Frameworks/R.framework/Versions/3.2/Resources/library/Rcpp/include -isystem /Library/Frameworks/R.framework/Versions/3.2/Resources/library/BH/include   -c RcppExports.cpp -o RcppExports.o
clang++ -arch x86_64 -ftemplate-depth-256 -I/Library/Frameworks/R.framework/Resources/include     -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include -I"/Library/Frameworks/R.framework/Versions/3.2/Resources/library/Rcpp/include"  -D_FILE_OFFSET_BITS=64 -I/usr/local/include -I../inst/include -fPIC  -O3 -Wall -pedantic -Wextra -Wno-padded -Wunused -isystem /Library/Frameworks/R.framework/Versions/3.2/Resources/library/Rcpp/include -isystem /Library/Frameworks/R.framework/Versions/3.2/Resources/library/BH/include   -c heartbeat.cpp -o heartbeat.o
In file included from heartbeat.cpp:2:
../inst/include/tthread/tinythread.h:631:9: warning: 'long long' is a C++11
      extension [-Wc++11-long-long]
typedef long long __intmax_t;
        ^
In file included from heartbeat.cpp:5:
In file included from /usr/local/include/hiredis/hiredis.h:36:
/usr/local/include/hiredis/read.h:75:50: warning: 'long long' is a C++11
      extension [-Wc++11-long-long]
    void *(*createInteger)(const redisReadTask*, long long);
                                                 ^
In file included from heartbeat.cpp:5:
In file included from /usr/local/include/hiredis/hiredis.h:40:
/usr/local/include/hiredis/sds.h:47:10: warning: flexible array members are a
      C99 feature [-Wc99-extensions]
    char buf[];
         ^
/usr/local/include/hiredis/sds.h:92:21: warning: 'long long' is a C++11
      extension [-Wc++11-long-long]
sds sdsfromlonglong(long long value);
                    ^
In file included from heartbeat.cpp:5:
/usr/local/include/hiredis/hiredis.h:114:5: warning: 'long long' is a C++11
      extension [-Wc++11-long-long]
    long long integer; /* The integer when type is REDIS_REPLY_INTEGER */
    ^
/usr/local/include/hiredis/hiredis.h:136:20: warning: commas at the end of
      enumerator lists are a C++11 extension [-Wc++11-extensions]
    REDIS_CONN_UNIX,
                   ^
6 warnings generated.
clang++ -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o RedisHeartbeat.so RcppExports.o heartbeat.o -L/usr/local/lib -lhiredis -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Library/Frameworks/R.framework/Versions/3.2/Resources/library/RedisHeartbeat/libs
** R
** inst
** tests
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/Library/Frameworks/R.framework/Versions/3.2/Resources/library/RedisHeartbeat/libs/RedisHeartbeat.so':
  dlopen(/Library/Frameworks/R.framework/Versions/3.2/Resources/library/RedisHeartbeat/libs/RedisHeartbeat.so, 6): Library not loaded: libhiredis.0.13.dylib
  Referenced from: /Library/Frameworks/R.framework/Versions/3.2/Resources/library/RedisHeartbeat/libs/RedisHeartbeat.so
  Reason: image not found
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/3.2/Resources/library/RedisHeartbeat’
Error: Command failed (1)
jscamac commented 8 years ago

Disabling the System Integrity Protection in El Capitan following the approach @dfalster linked to works. Redis is installed. and I've reactivated it.

This is a temporary solution for now ;)

richfitz commented 8 years ago

I have El Cap on one laptop and don't see any problem installing, but also don't know what I might have done to configure things differently. I also don't see the compilation warnings you see. I don't remember disabling the SIP thing that Daniel linked (looks like something I'd remember). However the laptop is also a bit out of date (running 10.11.1 (15B42)) so perhaps things have become more locked down?

jscamac commented 8 years ago

Not sure. Though disabling the SIP fixed the installation problem for me. SIP also screwed with mactex and pdflatex https://tug.org/mactex/UpdatingForElCapitan.pdf. So this problem appears to be quiet widespread.

Not sure what the long-term fix to this issue is.

Re: the extra compile warnings. I think I'm getting them as Daniel must have changed my profile to make it more verbose.

dfalster commented 8 years ago

Yes, the compiler warnings come from extra settings in .R/Makevars. i copied my settings over to james, which in turn came from Rich at some point. perhaps these aren’t needed anymore.

On Sat, Apr 9, 2016 at 12:05 PM, James Camac notifications@github.com wrote:

Not sure. Though disabling the SIP fixed the installation problem for me. SIP also screwed with mactex and pdflatex https://tug.org/mactex/UpdatingForElCapitan.pdf. So this problem appears to be quiet widespread.

Not sure what the long-term fix to this issue is.

Re: the extra compile warnings. I think I'm getting them as Daniel must have changed my profile to make it more verbose.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/richfitz/RedisHeartbeat/issues/5#issuecomment-207683250

AaronTao1990 commented 7 years ago

disabling the SIP will bring new problems.

There must be some way else out there

richfitz commented 7 years ago

I agree, but I also can't replicate this problem on my El Cap system :disappointed:

richfitz commented 4 years ago

There is a travis job for an el cap install now - please open a new issue if problems persist