Error on require in Debian

duckontheweb commented 7 years ago

PRETTY_NAME="Debian GNU/Linux 8 (jessie)"
NAME="Debian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=debian

Node v4.7.0
NPM v2.15.11
HDF5 v1.8.18
gcc (Debian 4.9.2-10) 4.9.2

I installed hdf5.node successfully using npm install hdf5 --fallback-to-build --hdf5_home_linux=/usr/local/hdf5 within a Node project. However, I get the following error when trying to require:

> var hdf5 = require('hdf5');
Error: libhdf5.so.100: cannot open shared object file: No such file or directory
    at Error (native)
    at Object.Module._extensions..node (module.js:434:18)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/root/test-app/node_modules/hdf5/index.js:4:25)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)

rimmartin commented 7 years ago

http://hdf-ni.github.io/hdf5.node/doc/install-setup.html Give

npm install hdf5 --fallback-to-build –hdf5_home_mac=/usr/local/hdf5

a try. If it still fails we'll investigate further

rimmartin commented 7 years ago

Clean up by getting rid of the build folder and it's linux project placed there by node-pre-gyp

rimmartin commented 7 years ago

It should be looking for a libhdf5.dylib instead of libhdf5.so.100

duckontheweb commented 7 years ago

npm install hdf5 --fallback-to-build --hdf5_home_mac=/usr/local/hdf5 installs without error, but I now get a different error on require:

> var h = require("hdf5")
Error: Cannot find module './build/Release/hdf5'
    at Function.Module._resolveFilename (module.js:325:15)
    at Function.Module._load (module.js:276:25)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/opt/hdf5-test/node_modules/hdf5/index.js:4:25)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Module.require (module.js:353:17)

rimmartin commented 7 years ago

I can't repeat this on my linux box but after work today I'll have access to a mac and will investigate then

rimmartin commented 7 years ago

to get to the hdf5 module you will need var h = require("hdf5").hdf5;

but I don't expect that to fix this. This project has modules for different parts like var h5im = require("hdf5").h5im; get the image interface

duckontheweb commented 7 years ago

Okay, thanks. I'm running this on Linux as well (Debian), so I'm not sure if experimenting on a Mac will help or not. I'll try to troubleshoot the latest error a bit more and let you know if I find anything.

On Fri, Dec 9, 2016 at 11:38 AM, rimmartin notifications@github.com wrote:

to get to the hdf5 module you will need var h = require("hdf5").hdf5;

but I don't expect that to fix this. This project has modules for different parts like var h5im = require("hdf5").h5im; get the image interface

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/HDF-NI/hdf5.node/issues/34#issuecomment-266088135, or mute the thread https://github.com/notifications/unsubscribe-auth/AGjzeZ4pzEuqtWL2sLYMt3xVt8wgPuleks5rGaASgaJpZM4LIbqg .

duckontheweb commented 7 years ago

I just wiped out my build and started over using npm install hdf5 --fallback-to-build --hdf5_home_mac=/usr/local/hdf5 and the build is now failing with the following error:

In file included from ../src/hdf5.cc:2:0:
../src/file.h:11:18: fatal error: hdf5.h: No such file or directory
 #include "hdf5.h"
                  ^
compilation terminated.
hdf5.target.mk:104: recipe for target 'Release/obj.target/hdf5/src/hdf5.o' failed

This is what I was getting when I was using the wrong path to my HDF5 install, so I'm guessing that the --hdf5_home_mac just isn't picking up the library.

duckontheweb commented 7 years ago

In case it's helpful, here's the full output for a successful install. This is the one that yields the original error.

root@44089d5f6762:/opt/test-project# npm install hdf5 --fallback-to-build --hdf5_home_linux=/usr/local/hdf5
npm info it worked if it ends with ok
npm info using npm@2.15.11
npm info using node@v4.7.0
npm WARN package.json test-project@1.0.0 No description
npm WARN package.json test-project@1.0.0 No repository field.
npm WARN package.json test-project@1.0.0 No README data
npm info addNameTag [ 'hdf5', 'latest' ]
npm info attempt registry request try #1 at 12:34:52 AM
npm http request GET https://registry.npmjs.org/hdf5
npm http 304 https://registry.npmjs.org/hdf5
npm info install hdf5@0.2.1 into /opt/test-project
npm info installOne hdf5@0.2.1
npm info preinstall hdf5@0.2.1
npm info package.json node-pre-gyp@0.6.11 license should be a valid SPDX license expression
npm info build /opt/test-project/node_modules/hdf5
npm info linkStuff hdf5@0.2.1
npm info build /opt/test-project/node_modules/hdf5/node_modules/node-pre-gyp
npm info preinstall node-pre-gyp@0.6.11
npm info linkStuff node-pre-gyp@0.6.11
npm info build /opt/test-project/node_modules/hdf5/node_modules/node-pre-gyp/node_modules/mkdirp
npm info preinstall mkdirp@0.5.1
npm info linkStuff mkdirp@0.5.1
npm info install mkdirp@0.5.1
npm info postinstall mkdirp@0.5.1
npm info build /opt/test-project/node_modules/hdf5/node_modules/node-pre-gyp/node_modules/nopt
npm info preinstall nopt@3.0.4
npm info linkStuff nopt@3.0.4
npm info install nopt@3.0.4
npm info postinstall nopt@3.0.4
npm info build /opt/test-project/node_modules/hdf5/node_modules/node-pre-gyp/node_modules/npmlog
npm info preinstall npmlog@1.2.1
npm info linkStuff npmlog@1.2.1
npm info install npmlog@1.2.1
npm info postinstall npmlog@1.2.1
npm info build /opt/test-project/node_modules/hdf5/node_modules/node-pre-gyp/node_modules/rc
npm info preinstall rc@1.1.1
npm info linkStuff rc@1.1.1
npm info install rc@1.1.1
npm info postinstall rc@1.1.1
npm info build /opt/test-project/node_modules/hdf5/node_modules/node-pre-gyp/node_modules/request
npm info preinstall request@2.63.0
npm info linkStuff request@2.63.0
npm info install request@2.63.0
npm info postinstall request@2.63.0
npm info build /opt/test-project/node_modules/hdf5/node_modules/node-pre-gyp/node_modules/rimraf
npm info preinstall rimraf@2.4.3
npm info linkStuff rimraf@2.4.3
npm info install rimraf@2.4.3
npm info postinstall rimraf@2.4.3
npm info build /opt/test-project/node_modules/hdf5/node_modules/node-pre-gyp/node_modules/semver
npm info preinstall semver@5.0.3
npm info linkStuff semver@5.0.3
npm info install semver@5.0.3
npm info postinstall semver@5.0.3
npm info build /opt/test-project/node_modules/hdf5/node_modules/node-pre-gyp/node_modules/tar
npm info preinstall tar@2.2.1
npm info linkStuff tar@2.2.1
npm info install tar@2.2.1
npm info postinstall tar@2.2.1
npm info build /opt/test-project/node_modules/hdf5/node_modules/node-pre-gyp/node_modules/tar-pack
npm info preinstall tar-pack@2.0.0
npm info linkStuff tar-pack@2.0.0
npm info install tar-pack@2.0.0
npm info postinstall tar-pack@2.0.0
npm info install node-pre-gyp@0.6.11
npm info postinstall node-pre-gyp@0.6.11
npm info install hdf5@0.2.1

> hdf5@0.2.1 install /opt/test-project/node_modules/hdf5
> node-pre-gyp install --fallback-to-build

node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@0.6.11
node-pre-gyp info using node@4.7.0 | linux | x64
node-pre-gyp info check checked for "/opt/test-project/node_modules/hdf5/lib/binding/hdf5.node" (not found)
node-pre-gyp http GET https://hdf5.node.s3-us-west-2.amazonaws.com/v0.2.1/hdf5-v0.2.1-node-v46-linux-x64.tar.gz
node-pre-gyp http Hostname/IP doesn't match certificate's altnames: "Host: hdf5.node.s3-us-west-2.amazonaws.com. is not in the cert's altnames: DNS:s3-us-west-2.amazonaws.com, DNS:*.s3-us-west-2.amazonaws.com, DNS:s3.us-west-2.amazonaws.com, DNS:*.s3.us-west-2.amazonaws.com, DNS:s3.dualstack.us-west-2.amazonaws.com, DNS:*.s3.dualstack.us-west-2.amazonaws.com, DNS:*.s3.amazonaws.com" (falling back to source compile with node-gyp) 
gyp info it worked if it ends with ok
gyp info using node-gyp@3.4.0
gyp info using node@4.7.0 | linux | x64
gyp info ok 
gyp info it worked if it ends with ok
gyp info using node-gyp@3.4.0
gyp info using node@4.7.0 | linux | x64
gyp info spawn /usr/bin/python2
gyp info spawn args [ '/usr/local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/opt/test-project/node_modules/hdf5/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/usr/local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/root/.node-gyp/4.7.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/root/.node-gyp/4.7.0',
gyp info spawn args   '-Dnode_gyp_dir=/usr/local/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=node.lib',
gyp info spawn args   '-Dmodule_root_dir=/opt/test-project/node_modules/hdf5',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
gyp info ok 
gyp info it worked if it ends with ok
gyp info using node-gyp@3.4.0
gyp info using node@4.7.0 | linux | x64
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/opt/test-project/node_modules/hdf5/build'
  CXX(target) Release/obj.target/hdf5/src/hdf5.o
  CXX(target) Release/obj.target/hdf5/src/h5_file.o
  CXX(target) Release/obj.target/hdf5/src/h5_group.o
  SOLINK_MODULE(target) Release/obj.target/hdf5.node
  COPY Release/hdf5.node
  COPY /opt/test-project/node_modules/hdf5/lib/binding/hdf5.node
  TOUCH Release/obj.target/action_after_build.stamp
  CXX(target) Release/obj.target/h5ds/src/h5ds.o
  SOLINK_MODULE(target) Release/obj.target/h5ds.node
  COPY Release/h5ds.node
  CXX(target) Release/obj.target/h5im/src/h5im.o
  SOLINK_MODULE(target) Release/obj.target/h5im.node
  COPY Release/h5im.node
  CXX(target) Release/obj.target/h5lt/src/h5lt.o
  SOLINK_MODULE(target) Release/obj.target/h5lt.node
  COPY Release/h5lt.node
  CXX(target) Release/obj.target/h5pt/src/h5pt.o
  CXX(target) Release/obj.target/h5pt/src/h5_pt.o
  SOLINK_MODULE(target) Release/obj.target/h5pt.node
  COPY Release/h5pt.node
  CXX(target) Release/obj.target/h5tb/src/h5tb.o
  SOLINK_MODULE(target) Release/obj.target/h5tb.node
  COPY Release/h5tb.node
make: Leaving directory '/opt/test-project/node_modules/hdf5/build'
gyp info ok 
node-pre-gyp info ok 
npm info postinstall hdf5@0.2.1
hdf5@0.2.1 node_modules/hdf5
npm info ok

duckontheweb commented 7 years ago

What flavor of Linux are you using? Maybe I can try on a different distro and see if I can get it to work that way.

rimmartin commented 7 years ago

Ubuntu and CentOS. I'll be at a mac this morning and testing it there

zszszsz commented 7 years ago

I guess you have no hdf5 lib installed?

rimmartin commented 7 years ago

gcc (Debian 4.9.2-10) 4.9.2 is really a clang toolchain installed on your mac? It isn't a gcc installed by homebrew or compiled there?

My mac is older 10.7 and it has gcc compiled and built on it because the clang was too old and the other software of my company is gcc portable.

But up further it looked like npm install hdf5 --fallback-to-build --hdf5_home_mac=/usr/local/hdf5 did install and build. you don't want the --hdf5_home_linux because you'll get *.so dependency.

In your /usr/local/hdf5/lib folder are the dynamic libraries .so or .dylib? Needs to be *.dylib

duckontheweb commented 7 years ago

Thanks for helping me troubleshoot this. Just wanted to be clear again that I am running this is on Linux and not on a Mac.

The HDF5 library installs successfully and passes all make check and make check-install tests and is located at /usr/local/hdf5. I think I had not cleared my previous build properly the first time I tried installing with the --hdf5_home_mac flag, which is why the build appeared to be successful. When I fully removed the hdf5.node install and started over I got the error noted a few posts ago that it could not find hdf5.h.

It looks like all of the dynamic libraries in /usr/local/hdf5/lib are in fact *.so. I've been following the HDF5 in section 2.1 of this HDF5 documentation to install the HDF5 libraries. Is there something I should do differently to get *.dylib extensions on a Linux machine?

I should also mention that these HDF5 libraries are working with the Python module h5py without issue on this machine.

duckontheweb commented 7 years ago

Okay, I think I finally got it.

I was confused by the "Environment Variables" section of the documentation, which said to use add this at runtime:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./hdf5.node/../hdf5/lib

I couldn't find the hdf5.node directory anywhere on my file system, and I wasn't sure if I was supposed to be adding a path to something inside of the hdf5 node module or to something in the C libraries. This was the command that worked for me:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path_to_HDF5_installation>/lib

which in my case is

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/hdf5/lib

What is the "hdf.node" portion of that path in the documentation supposed to be pointing to?

rimmartin commented 7 years ago

Sorry, I mistook this as a mac build issue

rimmartin commented 7 years ago

npm install hdf5 --fallback-to-build --hdf5_home_linux=/usr/local/hdf5 build is good Next do you have /usr/local/hdf5/lib on your LD_LIBRARY_PATH?

rimmartin commented 7 years ago

"hdf.node" is the actual name of this project in github https://github.com/HDF-NI/hdf5.node

I'll revise the documentation to put in your more clear wording

HDF-NI / hdf5.node

Error on require in Debian #34