Issue running tfjs-node-gpu under Windows

eggerand13 commented 5 years ago

TensorFlow.js version

tfjs-node-gpu 1.2.7

Node version

v12.9.0

Describe the problem or feature request

When trying to run a file that only contains the following line: const tf = require('@tensorflow/tfjs-node-gpu');

I will receive the following error:

internal/modules/cjs/loader.js:977
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: The specified module could not be found.
\\?\C:\development\JS ML\node_modules\@tensorflow\tfjs-node-gpu\lib\napi-v4\tfjs_binding.node
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:977:18)
    at Module.load (internal/modules/cjs/loader.js:790:32)
    at Function.Module._load (internal/modules/cjs/loader.js:703:12)
    at Module.require (internal/modules/cjs/loader.js:830:19)
    at require (internal/modules/cjs/helpers.js:68:18)
    at Object.<anonymous> (C:\Entwicklung_neu\JS ML\node_modules\@tensorflow\tfjs-node-gpu\dist\index.js:44:16)
    at Module._compile (internal/modules/cjs/loader.js:936:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:947:10)
    at Module.load (internal/modules/cjs/loader.js:790:32)
    at Function.Module._load (internal/modules/cjs/loader.js:703:12)

I tried to find out why there is this strange "\\?" prepended to the path but I couldn't really find out why it is happening. The code will run without any problem when I only use the CPU module (tfjs-node).

I have a CUDA compatible graphics card (GTX960M) and followed the complete installation guide. When installing the module I have only Python 2.7 installed. The installation of the module also seems to work as expected:

> @tensorflow/tfjs-node-gpu@1.2.7 install C:\development\JS ML\node_modules\@tensorflow\tfjs-node-gpu
> node scripts/install.js gpu download

GPU-windows-1.2.7.zip
* Downloading libtensorflow
[==============================] 3508656/bps 100% 0.0s
[==============================] 1862362/bps 100% 0.0s
* Building TensorFlow Node.js bindings
npm WARN js-ml@1.0.0 No description
npm WARN js-ml@1.0.0 No repository field.

added 86 packages from 83 contributors and audited 150 packages in 98.859s
found 0 vulnerabilities

I would be very glad if anyone could help me with this issue.

chenqing commented 5 years ago

The same issue in windows 10. Error: Cannot find module 'C:\Users\zx\work\escher-project\escher-util\node_modules\_@tensorflow_tfjs-node@1.2.8@@tensorflow\tfjs-node\lib\napi-v4\tfjs_binding.node' at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15) at Function.Module._load (internal/modules/cjs/loader.js:562:25) at Module.require (internal/modules/cjs/loader.js:690:17) at require (internal/modules/cjs/helpers.js:25:18) at Object.<anonymous> (C:\Users\zx\work\escher-project\escher-util\node_modules\_@tensorflow_tfjs-node@1.2.8@@tensorflow\tfjs-node\dist\index.js:44:16) at Module._compile (internal/modules/cjs/loader.js:776:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10) at Module.load (internal/modules/cjs/loader.js:653:32) at tryModuleLoad (internal/modules/cjs/loader.js:593:12) at Function.Module._load (internal/modules/cjs/loader.js:585:3)

kangyizhang commented 5 years ago

@eggerand13 @chenqing I just updated the addon module for windows. Please try @tensorflow/tfjs-node @ 1.2.8.

eggerand13 commented 5 years ago

@eggerand13 @chenqing I just updated the addon module for windows. Please try @tensorflow/tfjs-node @ 1.2.8.

Thank you very much for your help. Could you point me in the right direction HOW I could get the version 1.2.8? NPM doesn't know about ("No matching version found for @tensorflow/tfjs-node-gpu@^1.2.8") and I also see that the latest commit in tfjs-node is already 17 hours old and it's not about this issue. Also in your private fork of tfjs-node I don't see any updates.

I hope am not missing something obvious here.

Thank you.

d3x0r commented 5 years ago

I had this same error. I further went to bulid @tensorflow/tfjs-node-gyp using node-gyp configure

M:\javascript\tensorflow\node_modules\@tensorflow\tfjs-node-gpu>node-gyp configure
gyp info it worked if it ends with ok
gyp info using node-gyp@5.0.3
gyp info using node@12.9.0 | win32 | x64
gyp info find Python using Python version 2.7.11 found at "c:\tools\Python27\python.exe"
gyp info find VS using VS2015 (14.0) found at:
gyp info find VS "C:\Program Files (x86)\Microsoft Visual Studio 14.0"
gyp info find VS run with --verbose for detailed information
gyp info spawn c:\tools\Python27\python.exe
gyp info spawn args [
gyp info spawn args   'C:\\tools\\nodejs\\node_modules\\node-gyp\\gyp\\gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'msvs',
gyp info spawn args   '-I',
gyp info spawn args   'M:\\javascript\\tensorflow\\node_modules\\@tensorflow\\tfjs-node-gpu\\build\\config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\tools\\nodejs\\node_modules\\node-gyp\\addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\d3x0r\\AppData\\Local\\node-gyp\\Cache\\12.9.0\\include\\node\\common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=C:\\Users\\d3x0r\\AppData\\Local\\node-gyp\\Cache\\12.9.0',
gyp info spawn args   '-Dnode_gyp_dir=C:\\tools\\nodejs\\node_modules\\node-gyp',
gyp info spawn args   '-Dnode_lib_file=C:\\Users\\d3x0r\\AppData\\Local\\node-gyp\\Cache\\12.9.0\\<(target_arch)\\node.lib',
gyp info spawn args   '-Dmodule_root_dir=M:\\javascript\\tensorflow\\node_modules\\@tensorflow\\tfjs-node-gpu',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'M:\\javascript\\tensorflow\\node_modules\\@tensorflow\\tfjs-node-gpu\\build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp: Undefined variable module_name in binding.gyp while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (C:\tools\nodejs\node_modules\node-gyp\lib\configure.js:344:16)
gyp ERR! stack     at ChildProcess.emit (events.js:209:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
gyp ERR! System Windows_NT 6.1.7601
gyp ERR! command "C:\\tools\\nodejs\\node.exe" "C:\\tools\\nodejs\\node_modules\\node-gyp\\bin\\node-gyp.js" "configure"
gyp ERR! cwd M:\javascript\tensorflow\node_modules\@tensorflow\tfjs-node-gpu
gyp ERR! node -v v12.9.0
gyp ERR! node-gyp -v v5.0.3
gyp ERR! not ok

which I gues scomes form this part of binding.gyp

  , {
      "target_name": "action_after_build",
      "type": "none",
      "dependencies": [ "<(module_name)" ],
      "copies": [
        {
          "files": [ "<(PRODUCT_DIR)/<(module_name).node" ],
          "destination": "<(module_path)"
        }
      }

which gracking down 'binding.gyp module name.....' seems to be a problem with node-pre-gyp which doesn't appear to be a thing....

I saw the other notes, and the WINDOWS _TROUBLESHOOTING.md but really none of that applies, bulding other things with node-gyp works fine but then I don't use <(module_name).

kangyizhang commented 5 years ago

@eggerand13 ah turns out we only published tfjs-node@1.2.8 but missed tfjs-node-gpu. I just published it. Please give it a try.

FYI we are hosting the pre compiled addon module in google cloud platform, so it's not a change in github.

kangyizhang commented 5 years ago

@d3x0r we switched to node-pre-gyp instead of node-gyp, that's why you are seeing the error. Please try tfjs-node-gpu @ 1.2.8 again.

FYI in future if you want to build the module by yourself, you can just run yarn or npm install, the install script (tfjs-node/scripts/install.js) will do all the work.

d3x0r commented 5 years ago

@kangyizhang npm install iin... myproject/node_modules/@tensorflow/tfjs-node-gpu will then install everything in myproject/node_modules/@tensorflow/ again in myproject/node_modules/@tensorflow/tfjs-node-gpu/node_modules/@tensorflow/... including tfjs-node-gpu again :)

npm install in an addon is not a good idea :)

I was just doing npm update in an existing project that (almost)) worked with 1.2.4..


> @tensorflow/tfjs-node-gpu@1.1.2 install M:\javascript\tensorflow\node_modules\@tensorflow\tfjs-node-gpu
> node scripts/install.js gpu download

* Downloading libtensorflow

* Building TensorFlow Node.js bindings
M:\javascript\tensorflow\node_modules\@tensorflow\tfjs-node-gpu\scripts\install.js:165
      throw new Error('node-gyp rebuild failed with: ' + err);
      ^

Error: node-gyp rebuild failed with: Error: Command failed: node-gyp rebuild
gyp ERR! build error 
gyp ERR! stack Error: `msbuild` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Users\Panther\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:197:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:254:12)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "h:\\dev2\\nodejs\\node.exe" "C:\\Users\\Panther\\AppData\\Roaming\\npm\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd M:\javascript\tensorflow\node_modules\@tensorflow\tfjs-node-gpu
gyp ERR! node -v v11.9.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok 

    at cp.exec (M:\javascript\tensorflow\node_modules\@tensorflow\tfjs-node-gpu\scripts\install.js:165:13)
    at ChildProcess.exithandler (child_process.js:304:5)
    at ChildProcess.emit (events.js:197:13)
    at maybeClose (internal/child_process.js:978:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:265:5)
npm WARN rollup-plugin-visualizer@1.1.1 requires a peer of rollup@>=0.60.0 but none is installed. You must install peer dependencies yourself.
npm WARN tensorflow@1.0.0 No description
npm WARN tensorflow@1.0.0 No repository field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @tensorflow/tfjs-node-gpu@1.1.2 install: `node scripts/install.js gpu download`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @tensorflow/tfjs-node-gpu@1.1.2 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Panther\AppData\Roaming\npm-cache\_logs\2019-05-06T10_53_38_142Z-debug.log

This is a pretty clean system, there was a dependancy of rollup

M:\javascript\tensorflow>npm install @tensorflow/tfjs-node-gpu
(node:11204) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 drain listeners added to [TLSSocket]. Use emitter.setMaxListeners() to increase limit
(node:11204) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 drain listeners added to [TLSSocket]. Use emitter.setMaxListeners() to increase limit
(node:11204) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 drain listeners added to [TLSSocket]. Use emitter.setMaxListeners() to increase limit
(node:11204) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 drain listeners added to [TLSSocket]. Use emitter.setMaxListeners() to increase limit
(node:11204) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 drain listeners added to [TLSSocket]. Use emitter.setMaxListeners() to increase limit
(node:11204) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 drain listeners added to [TLSSocket]. Use emitter.setMaxListeners() to increase limit

> @tensorflow/tfjs-node-gpu@1.2.8 install M:\javascript\tensorflow\node_modules\@tensorflow\tfjs-node-gpu
> node scripts/install.js gpu download

GPU-windows-1.2.8.zip
* Downloading libtensorflow
[==============================] 2506038/bps 100% 0.0s
[==============================] 658990/bps 100% 0.0s
* Building TensorFlow Node.js bindings
node-pre-gyp install failed with error: Error: Command failed: node-pre-gyp install --fallback-to-build
node-pre-gyp WARN Using needle for node-pre-gyp https download
node-pre-gyp WARN Tried to download(404): https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v4/1.2.8/tfjs_binding-v1.2.8-node-v72-win32-x64.tar.gz
node-pre-gyp WARN Pre-built binaries not found for @tensorflow/tfjs-node-gpu@1.2.8 and node@12.9.0 (node-v72 ABI, unknown) (falling back to source compile with node-gyp)
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\MSBuild\14.0\bin\MSBuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\tools\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:196:23)
gyp ERR! stack     at ChildProcess.emit (events.js:209:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
gyp ERR! System Windows_NT 6.1.7601
gyp ERR! command "C:\\tools\\nodejs\\node.exe" "C:\\tools\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "build" "--fallback-to-build" "--module=M:\\javascript\\tensorflow\\node_modules\\@tensorflo
w\\tfjs-node-gpu\\lib\\napi-v4\\tfjs_binding.node" "--module_name=tfjs_binding" "--module_path=M:\\javascript\\tensorflow\\node_modules\\@tensorflow\\tfjs-node-gpu\\lib\\napi-v4" "--napi_version=4" "--node_abi_napi=napi"
 "--napi_build_version=4" "--node_napi_label=napi-v4"
gyp ERR! cwd M:\javascript\tensorflow\node_modules\@tensorflow\tfjs-node-gpu
gyp ERR! node -v v12.9.0
gyp ERR! node-gyp -v v5.0.3
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute 'C:\tools\nodejs\node.exe C:\tools\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js build --fallback-to-build --module=M:\javascript\tensorflow\node_modules\@
tensorflow\tfjs-node-gpu\lib\napi-v4\tfjs_binding.node --module_name=tfjs_binding --module_path=M:\javascript\tensorflow\node_modules\@tensorflow\tfjs-node-gpu\lib\napi-v4 --napi_version=4 --node_abi_napi=napi --napi_bui
ld_version=4 --node_napi_label=napi-v4' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (M:\javascript\tensorflow\node_modules\node-pre-gyp\lib\util\compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:209:13)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:1021:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5)
node-pre-gyp ERR! System Windows_NT 6.1.7601
node-pre-gyp ERR! command "C:\\tools\\nodejs\\node.exe" "M:\\javascript\\tensorflow\\node_modules\\node-pre-gyp\\bin\\node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd M:\javascript\tensorflow\node_modules\@tensorflow\tfjs-node-gpu
node-pre-gyp ERR! node -v v12.9.0
node-pre-gyp ERR! node-pre-gyp -v v0.13.0
node-pre-gyp ERR! not ok

npm WARN tensorflow@1.0.0 No description
npm WARN tensorflow@1.0.0 No repository field.

+ @tensorflow/tfjs-node-gpu@1.2.8
updated 1 package and audited 150 packages in 131.814s
found 0 vulnerabilities

chenqing commented 5 years ago

@eggerand13 @chenqing I just updated the addon module for windows. Please try @tensorflow/tfjs-node @ 1.2.8.

https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v4/1.2.8/tfjs_binding-v1.2.8-node-v64-win32-x64.tar.gz 404 now

kangyizhang commented 5 years ago

@chenqing the module is uploaded to https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v4/1.2.8/GPU-windows-1.2.8.zip

I notice @d3x0r get the same error. I'll take a look.

eggerand13 commented 5 years ago

@kangyizhang Thank you for providing a new version and handling this issue. Unfortunately there was no change with the 1.2.8 version. I still receive the same error as in my initial question.

kangyizhang commented 5 years ago

@eggerand13 can you try const tf = require('@tensorflow/tfjs-node-gpu'); and let me know if it works? I think you installed tfjs-node-gpu but you are importing tfjs-node.

eggerand13 commented 5 years ago

@kangyizhang Ohh I just realized that in my initial post I accidentially wrote tfjs-node instead of tfjs-node-gpu. I edited that part. I do import tfjs-node-gpu. It was just a copy-paste error when writing the issue post. So no, unfortunately that is not the issue.

As you can see from the error, the module is actually loaded. It is in node_modules\@tensorflow\tfjs-node-gpu\dist\index.js:44 that the error appears. It seems that somewhere here the problem occurs.

var bindingPath = binary.find(path.resolve(path.join(__dirname, '/../package.json')));
// tslint:disable-next-line:no-require-imports
var bindings = require(bindingPath);

When it tries to load the \node_modules\@tensorflow\tfjs-node-gpu\lib\napi-v4\tfjs_binding.node file (which is referenced in the package.json).

kangyizhang commented 5 years ago

This is a bug in tfjs-node@1.2.8 and will be fixed in the next release.

Arturbarth commented 5 years ago

Solution? Use:

npm install @tensorflow/tfjs@1.2.7 and

npm install @tensorflow/tfjs-node@1.2.7

This solved my problem.

eggerand13 commented 5 years ago

@Arturbarth Do you mean using those two installs BEFORE/AFTER using the @tensorflow/tfjs-node-gpu install? Because those two alone will not install tfjs-node-gpu module which this issue here is all about.

I tried using the two installs you proposed BEFORE the @tensorflow/tfjs-node-gpu install which unfortunately did not help.

kangyizhang commented 5 years ago

@eggerand13 it should work when you only install @tensorflow/tfjs-node-gpu@1.2.7. We will release a new version soon.

@Arturbarth You only need to install @tensorflow/tfjs-node. @tensorflow/tfjs will be installed when you install @tensorflow/tfjs-node.

d3x0r commented 5 years ago

Just a small note on mine - I did get node-pre-gyp to build it... but it still wouldn't load, even though the file was there. But in my case, this is a Radeon Mobile HD 5850 (from 2009), and being AMD it doesn't have CUDA. (Cuda doesn't have a fall back tp pure CPU mode either, apparently) Just out of curiosity; any work done to use OpenCL instead of CUDA?

rthadur commented 5 years ago

@d3x0r thank you,There is no plan for supporting OpenCL but it depends on overall TF Project , you can follow the discussion here. CC @tafsiri

Arturbarth commented 5 years ago

@kangyizhang Thanks.

@eggerand13 reinstall

npm uninstall @tensorflow/tfjs-node-gpu npm uninstall @tensorflow/tfjs-node

npm update

npm install @tensorflow/tfjs-node@1.2.7 npm install @tensorflow/tfjs-node-gpu@1.2.7

patheros commented 5 years ago

I still get the above error when I use version 1.2.7 I get a very similar error when I use version 1.2.6 Is there another workaround?

nnt0 commented 5 years ago

I'm also getting this issue using tfjs-node-gpu 1.2.9. tfjs-node also fails but with a different error. I tried older versions but they wont work either.

Cannot find module 'd:\development\node_modules\@tensorflow\tfjs-node-gpu\lib\napi-v4\tfjs_binding.node'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
    at Function.Module._load (internal/modules/cjs/loader.js:562:25)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (d:\development\node_modules\@tensorflow\tfjs-node-gpu\dist\index.js:44:16)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)

kangyizhang commented 5 years ago

@nnt0 I can not reproduce this error on my machine. Can you try the following:

remove node_module and reinstall the packages.
if it's still not working, can you check what's in path d:\development\node_modules\@tensorflow\tfjs-node-gpu\lib
go to d:\development\node_modules\@tensorflow\tfjs-node-gpu and run yarn or npm install

also what's the error you see for tfjs-node?

nnt0 commented 5 years ago

Thanks for the fast reply.

I deleted the node_modules folder and reinstalled everything. Since i still got the same error i checked the path you gave me and found out that the lib folder doesn't even exist.

I ran npm install in d:\development\node_modules\@tensorflow\tfjs-node-gpu and got this output

> @tensorflow/tfjs-node-gpu@1.2.9 install D:\development\node_modules\@tensorflow\tfjs-node-gpu
> node scripts/install.js gpu download

GPU-windows-1.2.9.zip
* Downloading libtensorflow
[==============================] 10903954/bps 100% 0.0s
(node:25632) UnhandledPromiseRejectionWarning: RangeError: Array buffer allocation failed
    at new ArrayBuffer (<anonymous>)
    at new Uint8Array (<anonymous>)
    at new FastBuffer (buffer.js:79:1)
    at Function.alloc (buffer.js:283:10)
    at decompress (D:\development\node_modules\@tensorflow\tfjs-node-gpu\node_modules\adm-zip\zipEntry.js:56:27)
    at Object.getData (D:\development\node_modules\@tensorflow\tfjs-node-gpu\node_modules\adm-zip\zipEntry.js:242:12)
    at D:\development\node_modules\@tensorflow\tfjs-node-gpu\node_modules\adm-zip\adm-zip.js:438:25
    at Array.forEach (<anonymous>)
    at Object.extractAllTo (D:\development\node_modules\@tensorflow\tfjs-node-gpu\node_modules\adm-zip\adm-zip.js:432:17)
    at WriteStream.response.on.pipe.on (D:\development\node_modules\@tensorflow\tfjs-node-gpu\scripts\resources.js:70:21)
(node:25632) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:25632) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
npm WARN @tensorflow/tfjs-node-gpu@1.2.9 No license field.

audited 732 packages in 25.007s
found 2 high severity vulnerabilities
  run `npm audit fix` to fix them, or `npm audit` for details

The error i'm getting for tfjs-node is this

Error: \\?\d:\development\node_modules\@tensorflow\tfjs-node\lib\napi-v4\tfjs_binding.node is not a valid Win32 application.
\\?\d:\development\node_modules\@tensorflow\tfjs-node\lib\napi-v4\tfjs_binding.node
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:807:18)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (d:\development\node_modules\@tensorflow\tfjs-node\dist\index.js:44:16)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)

kangyizhang commented 5 years ago

@nnt0

For the tfjs-node-gpu error, it seems like your computer does not have enough memory for unzipping the libtensorflow file. Can you clean up your memory and try again? Also can you check what's in the D:\development\node_modules\@tensorflow\tfjs-node-gpu\deps folder?

For the tfjs-node error, it seems like a prefix \\? is added in __dirname. If you just print out path.resolve(path.join(__dirname,'./package.json')) in a nodejs process, what's the result? This is why I'm asking this.

nnt0 commented 5 years ago

@kangyizhang

I reinstalled tfjs-node-gpu while monitoring my ram usage and still got the exact same error and stack trace as before even though my memory usage stayed under 80% the whole time.

The D:\development\node_modules\@tensorflow\tfjs-node-gpu\deps exists but it's completely empty.

I ran

const path = require("path")
console.log(path.resolve(path.join(__dirname, "./package.json")))

and got d:\development\package.json back.

kangyizhang commented 5 years ago

hi @nnt0 looks like your machine is 32-bit, which is not supported by tensorflow yet: https://www.tensorflow.org/install/lang_c#supported_platforms

nnt0 commented 5 years ago

@kangyizhang

I'm very sure that this isn't the issue especially since i am running Windows 10 64 bit.

kangyizhang commented 5 years ago

@eggerand13 the prefix \\? in the addon path is because of the space in your path (the project is in folder JS ML). Space in path is not supported by the binding tool node-gyp. Please try to move it to a folder without space and let me know if the error still exist.

kangyizhang commented 5 years ago

@nnt0

For GPU, it seems like the process failed to unzip the libtensorflow, which is 200+ MB. I can not tell or reproduce the root cause of Array buffer allocation failed, but most likely it relates to memory exhausted. I would recommend you to monitor the memory used by node process, and see is there any patterns.

For CPU error, it looks like node.js kernel is trying to load the addon as Win32 application.Can you try to run node -p "process.arch"?

nnt0 commented 5 years ago

@kangyizhang

Okay, i ran node -p "process.arch" and got ia32 back. I then reinstalled Node.js 64 bit, ran the command again and got x64 back. I guess i must have accidentally installed a 32 bit version of node at some point.

I then reinstalled tfjs-node as well as tfjs-node-gpu which completed without any error.

But trying to run either still produced errors. After some testing i found out that everything worked fine in a different directory. After some more testing i found out that a - that i had in a part of a directory caused it to fail. Once i removed the - from the directory everything ran fine.

I am a bit confused about how exactly this - caused everything to fail. I've used the same directory with tfjs-node 1.2.1 a while back and it didn't cause any errors.

Anyhow, thank you very much for helping me figure this out.

kangyizhang commented 5 years ago

@nnt0 Glad to hear it works and thank you for update your investigation.

We were using node-gyp as the binding tool previously, and switched to node-pre-gyp after 1.2.4 to avoid compiling issues on user's machine (the issues in the WINDOWS_TROUBLESHOOTING doc). It may be related with path resolving in the node-pre-gyp package. Though I tried with - and/or space in the directory name on my machine, I can not reproduce it.

I am gonna close this issue, but feel free to reopen/update if you have any issue.

AionDev commented 5 years ago

same issue on windows : "@tensorflow/tfjs-node-gpu": "^1.2.9", this line is from my package.json

kangyizhang commented 5 years ago

@AionDev did you install the NVIDIA software for gpu support (CUDA and cuDNN)? Misconfigured gpu software has been the reason for most of the errors.

emosGambler commented 5 years ago

Hey, I've managed to remove all the console issues while npm install by doing the following:

1) Remove node_modules/ 2) npm install -g yarn // because it is required to install some packages 3) Install python 2.7 4) Install Visual Studio + Desktop development with c++ 5) restart your PC 6) npm install @tensorflow/tfjs-node-gpu @tensorflow/tfjs

Problem went away after doing that.

tensorflow / tfjs