tensorflow / tfjs

A WebGL accelerated JavaScript library for training and deploying ML models.
https://js.tensorflow.org
Apache License 2.0
18.39k stars 1.92k forks source link

tfjs-node@4.2.0 install fails on Windows machine #7341

Open enyineer opened 1 year ago

enyineer commented 1 year ago

System information

When installing tfjs-node ^4.2.0 node-gyp fails to download the prebuilt binaries for napi-v8:

https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.2.0/CPU-linux-4.2.0.tar.gz exists, but https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.2.0/CPU-windows-4.2.0.zip doesn't.

Command: yarn add @tensorflow/tfjs-node

Any other info / logs

# This file contains the result of Yarn building a package (@tensorflow/tfjs-node@npm:4.2.0)
# Script name: install

CPU-windows-4.2.0.zip
* Building TensorFlow Node.js bindings
node-pre-gyp install failed with error: Error: Command failed: node-pre-gyp install --fallback-to-build
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@1.0.9
node-pre-gyp info using node@18.12.1 | win32 | x64
node-pre-gyp info check checked for "F:\Projekte\ebk-ml\node_modules\@tensorflow\tfjs-node\lib\napi-v8\tfjs_binding.node" (not found)
node-pre-gyp http GET https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.2.0/CPU-windows-4.2.0.zip
node-pre-gyp ERR! install response status 404 Not Found on https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.2.0/CPU-windows-4.2.0.zip 
node-pre-gyp WARN Pre-built binaries not installable for @tensorflow/tfjs-node@4.2.0 and node@18.12.1 (node-v108 ABI, unknown) (falling back to source compile with node-gyp) 
node-pre-gyp WARN Hit error response status 404 Not Found on https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.2.0/CPU-windows-4.2.0.zip 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute 'node-gyp.cmd clean' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (F:\Projekte\ebk-ml\node_modules\@mapbox\node-pre-gyp\lib\util\compile.js:89:23)
node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1091:16)
node-pre-gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:302:5)
node-pre-gyp ERR! System Windows_NT 10.0.22000
node-pre-gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "F:\\Projekte\\ebk-ml\\node_modules\\@mapbox\\node-pre-gyp\\bin\\node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd F:\Projekte\ebk-ml\node_modules\@tensorflow\tfjs-node
node-pre-gyp ERR! node -v v18.12.1
node-pre-gyp ERR! node-pre-gyp -v v1.0.9
node-pre-gyp ERR! not ok 
kevo1ution commented 1 year ago

I got the same error. I think this is because https://storage.googleapis.com/tf-builds/pre-built-binary/napi-v8/4.2.0/CPU-windows-4.2.0.zip doesn't exist. I went to https://storage.googleapis.com/tf-builds/ to see what binaries exist and searched for windows. The latest version I see is pre-built-binary/napi-v8/3.18.0/CPU-windows-3.18.0.zip.

The workaround I had to do to get this working is to just use the older version 3.18.0 so:

# save-exact flag so we don't allow any other version installed until correct binary is uploaded to tf-builds
npm install --save-exact @tensorflow/tfjs-node@3.18.0
neyoung commented 1 year ago

@kevo1ution I ran into the same issue and noticed the link for version 4.2.0 for windows didn't exist. I didn't think to install an older version. Thanks for sharing!

gaikwadrahul8 commented 1 year ago

Hi, @enyineer

Apologize for the delayed response and I checked this link and pre-built-binary/napi-v8/4.2.0/CPU-windows-4.2.0.zip file is not available at the moment for windows and if someone is looking to use on Windows so last supported file is available pre-built-binary/napi-v8/3.18.0/CPU-windows-3.18.0.zip at the moment and for linux CPU and GPU both are available for pre-built-binary/napi-v8/4.2.0/

Hi, @pyu10055 Do you have any update about tfjs-node@4.2.0support for windows please ? Thank you!

TobCraft3521 commented 1 year ago

I still have the same error (I think) - using npm install --save-exact @tensorflow/tfjs-node@3.18.0 I have all the required stuff installed and - get this error

`npm ERR! code 1 npm ERR! path C:\Users\tobia\tfjs\node_modules\@tensorflow\tfjs-node npm ERR! command failed npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node scripts/install.js npm ERR! CPU-windows-3.18.0.zip npm ERR! https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-windows-x86_64-2.7.0.zip npm ERR! Downloading libtensorflow npm ERR! npm ERR! Building TensorFlow Node.js bindings npm ERR! symlink ./lib/napi-v9 failed: Error: Command failed: node scripts/deps-stage.js symlink ./lib/napi-v9 npm ERR! Symlink of lib\napi-v9\tensorflow.dll failed, creating a copy on disk. npm ERR! node:internal/process/promises:289 npm ERR! triggerUncaughtException(err, true / fromPromise */); npm ERR! ^ npm ERR! npm ERR! [Error: ENOENT: no such file or directory, copyfile 'C:\Users\tobia\tfjs\node_modules\@tensorflow\tfjs-node\deps\lib\tensorflow.dll' -> 'C:\Users\tobia\tfjs\node_modules\@tensorflow\tfjs-node\lib\napi-v9\tensorflow.dll'] { npm ERR! errno: -4058, npm ERR! code: 'ENOENT', npm ERR! syscall: 'copyfile', npm ERR! path: 'C:\Users\tobia\tfjs\node_modules\@tensorflow\tfjs-node\deps\lib\tensorflow.dll', npm ERR! dest: 'C:\Users\tobia\tfjs\node_modules\@tensorflow\tfjs-node\lib\napi-v9\tensorflow.dll' npm ERR! } npm ERR! npm ERR! Node.js v20.5.0 npm ERR! npm ERR! at ChildProcess.exithandler (node:child_process:421:12) npm ERR! at ChildProcess.emit (node:events:514:28) npm ERR! at maybeClose (node:internal/child_process:1105:16) npm ERR! at ChildProcess._handle.onexit (node:internal/child_process:305:5) { npm ERR! code: 1, npm ERR! killed: false, npm ERR! signal: null, npm ERR! cmd: 'node scripts/deps-stage.js symlink ./lib/napi-v9' npm ERR! }

npm ERR! A complete log of this run can be found in: C:\Users\tobia\AppData\Local\npm-cache_logs\2023-09-10T09_33_52_091Z-debug-0.log`

I dont use any antivirus software or other stuff like that

amacsmith commented 1 year ago

Having the same issue

$ npm install
npm ERR! code 1
npm ERR! path C:\Users\thedude\dev\someproject\node_modules\@tensorflow\tfjs-node
npm ERR! command failed
npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node scripts/install.js
npm ERR! CPU-windows-4.11.0.zip
npm ERR! https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-windows-x86_64-2.9.1.zip
npm ERR! * Downloading libtensorflow
npm ERR!
npm ERR! * Building TensorFlow Node.js bindings
npm ERR! symlink ./lib/napi-v9 failed:  Error: Command failed: node scripts/deps-stage.js symlink ./lib/napi-v9
npm ERR!   * Symlink of lib\napi-v9\tensorflow.dll failed, creating a copy on disk.
npm ERR! node:internal/process/promises:289
npm ERR!             triggerUncaughtException(err, true /* fromPromise */);
npm ERR!             ^
npm ERR!
npm ERR! [Error: ENOENT: no such file or directory, copyfile 'C:\Users\thedude\dev\someproject\node_modules\@tensorflow\tfjs-node\deps\lib\tensorflow.dll' -> 'C:\Users\thedude\dev\someproject\node_modules\@tensorflow\tfjs-node\lib\napi-v9\tensorflow.dll'] {
npm ERR!   errno: -4058,
npm ERR!   code: 'ENOENT',
npm ERR!   syscall: 'copyfile',
npm ERR!   path: 'C:\\Users\\thedude\\dev\\someproject\\node_modules\\@tensorflow\\tfjs-node\\deps\\lib\\tensorflow.dll',
npm ERR!   dest: 'C:\\Users\\thedude\\dev\\someproject\\node_modules\\@tensorflow\\tfjs-node\\lib\\napi-v9\\tensorflow.dll'
npm ERR! }
npm ERR!
npm ERR! Node.js v20.7.0
npm ERR!
npm ERR!     at ChildProcess.exithandler (node:child_process:422:12)
npm ERR!     at ChildProcess.emit (node:events:514:28)
npm ERR!     at maybeClose (node:internal/child_process:1105:16)
npm ERR!     at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
npm ERR!   code: 1,
npm ERR!   killed: false,
npm ERR!   signal: null,
npm ERR!   cmd: 'node scripts/deps-stage.js symlink ./lib/napi-v9'
npm ERR! }

npm ERR! A complete log of this run can be found in: C:\Users\thedude\AppData\Local\npm-cache\_logs\2023-09-19T06_58_31_377Z-debug-0.log

Would love some additional information on this error.

georgzoeller commented 1 year ago

Having the same issue

$ npm install
npm ERR! code 1
npm ERR! path C:\Users\thedude\dev\someproject\node_modules\@tensorflow\tfjs-node
npm ERR! command failed
npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node scripts/install.js
npm ERR! CPU-windows-4.11.0.zip
npm ERR! https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-windows-x86_64-2.9.1.zip
npm ERR! * Downloading libtensorflow
npm ERR!
npm ERR! * Building TensorFlow Node.js bindings
npm ERR! symlink ./lib/napi-v9 failed:  Error: Command failed: node scripts/deps-stage.js symlink ./lib/napi-v9
npm ERR!   * Symlink of lib\napi-v9\tensorflow.dll failed, creating a copy on disk.
npm ERR! node:internal/process/promises:289
npm ERR!             triggerUncaughtException(err, true /* fromPromise */);
npm ERR!             ^
npm ERR!
npm ERR! [Error: ENOENT: no such file or directory, copyfile 'C:\Users\thedude\dev\someproject\node_modules\@tensorflow\tfjs-node\deps\lib\tensorflow.dll' -> 'C:\Users\thedude\dev\someproject\node_modules\@tensorflow\tfjs-node\lib\napi-v9\tensorflow.dll'] {
npm ERR!   errno: -4058,
npm ERR!   code: 'ENOENT',
npm ERR!   syscall: 'copyfile',
npm ERR!   path: 'C:\\Users\\thedude\\dev\\someproject\\node_modules\\@tensorflow\\tfjs-node\\deps\\lib\\tensorflow.dll',
npm ERR!   dest: 'C:\\Users\\thedude\\dev\\someproject\\node_modules\\@tensorflow\\tfjs-node\\lib\\napi-v9\\tensorflow.dll'
npm ERR! }
npm ERR!
npm ERR! Node.js v20.7.0
npm ERR!
npm ERR!     at ChildProcess.exithandler (node:child_process:422:12)
npm ERR!     at ChildProcess.emit (node:events:514:28)
npm ERR!     at maybeClose (node:internal/child_process:1105:16)
npm ERR!     at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
npm ERR!   code: 1,
npm ERR!   killed: false,
npm ERR!   signal: null,
npm ERR!   cmd: 'node scripts/deps-stage.js symlink ./lib/napi-v9'
npm ERR! }

npm ERR! A complete log of this run can be found in: C:\Users\thedude\AppData\Local\npm-cache\_logs\2023-09-19T06_58_31_377Z-debug-0.log

Would love some additional information on this error.

Same, Win11. The directories exist, as do the file

georgzoeller commented 1 year ago

Actually it is because the napi-v9 folder in libs does not exist and doesn't have a bindings file

Iust1n2 commented 1 year ago

Having the same issue

$ npm install
npm ERR! code 1
npm ERR! path C:\Users\thedude\dev\someproject\node_modules\@tensorflow\tfjs-node
npm ERR! command failed
npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node scripts/install.js
npm ERR! CPU-windows-4.11.0.zip
npm ERR! https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-windows-x86_64-2.9.1.zip
npm ERR! * Downloading libtensorflow
npm ERR!
npm ERR! * Building TensorFlow Node.js bindings
npm ERR! symlink ./lib/napi-v9 failed:  Error: Command failed: node scripts/deps-stage.js symlink ./lib/napi-v9
npm ERR!   * Symlink of lib\napi-v9\tensorflow.dll failed, creating a copy on disk.
npm ERR! node:internal/process/promises:289
npm ERR!             triggerUncaughtException(err, true /* fromPromise */);
npm ERR!             ^
npm ERR!
npm ERR! [Error: ENOENT: no such file or directory, copyfile 'C:\Users\thedude\dev\someproject\node_modules\@tensorflow\tfjs-node\deps\lib\tensorflow.dll' -> 'C:\Users\thedude\dev\someproject\node_modules\@tensorflow\tfjs-node\lib\napi-v9\tensorflow.dll'] {
npm ERR!   errno: -4058,
npm ERR!   code: 'ENOENT',
npm ERR!   syscall: 'copyfile',
npm ERR!   path: 'C:\\Users\\thedude\\dev\\someproject\\node_modules\\@tensorflow\\tfjs-node\\deps\\lib\\tensorflow.dll',
npm ERR!   dest: 'C:\\Users\\thedude\\dev\\someproject\\node_modules\\@tensorflow\\tfjs-node\\lib\\napi-v9\\tensorflow.dll'
npm ERR! }
npm ERR!
npm ERR! Node.js v20.7.0
npm ERR!
npm ERR!     at ChildProcess.exithandler (node:child_process:422:12)
npm ERR!     at ChildProcess.emit (node:events:514:28)
npm ERR!     at maybeClose (node:internal/child_process:1105:16)
npm ERR!     at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
npm ERR!   code: 1,
npm ERR!   killed: false,
npm ERR!   signal: null,
npm ERR!   cmd: 'node scripts/deps-stage.js symlink ./lib/napi-v9'
npm ERR! }

npm ERR! A complete log of this run can be found in: C:\Users\thedude\AppData\Local\npm-cache\_logs\2023-09-19T06_58_31_377Z-debug-0.log

Would love some additional information on this error.

Same, Win11. The directories exist, as do the file

Same here. It appears that the @tensorflow folder is empty

Iust1n2 commented 1 year ago

I still have the same error (I think) - using npm install --save-exact @tensorflow/tfjs-node@3.18.0 I have all the required stuff installed and - get this error

`npm ERR! code 1 npm ERR! path C:\Users\tobia\tfjs\node_modules@tensorflow\tfjs-node npm ERR! command failed npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node scripts/install.js npm ERR! CPU-windows-3.18.0.zip npm ERR! https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-windows-x86_64-2.7.0.zip npm ERR! Downloading libtensorflow npm ERR! npm ERR! Building TensorFlow Node.js bindings npm ERR! symlink ./lib/napi-v9 failed: Error: Command failed: node scripts/deps-stage.js symlink ./lib/napi-v9 npm ERR! Symlink of lib\napi-v9\tensorflow.dll failed, creating a copy on disk. npm ERR! node:internal/process/promises:289 npm ERR! triggerUncaughtException(err, true / fromPromise */); npm ERR! ^ npm ERR! npm ERR! [Error: ENOENT: no such file or directory, copyfile 'C:\Users\tobia\tfjs\node_modules@tensorflow\tfjs-node\deps\lib\tensorflow.dll' -> 'C:\Users\tobia\tfjs\node_modules@tensorflow\tfjs-node\lib\napi-v9\tensorflow.dll'] { npm ERR! errno: -4058, npm ERR! code: 'ENOENT', npm ERR! syscall: 'copyfile', npm ERR! path: 'C:\Users\tobia\tfjs\node_modules\@tensorflow\tfjs-node\deps\lib\tensorflow.dll', npm ERR! dest: 'C:\Users\tobia\tfjs\node_modules\@tensorflow\tfjs-node\lib\napi-v9\tensorflow.dll' npm ERR! } npm ERR! npm ERR! Node.js v20.5.0 npm ERR! npm ERR! at ChildProcess.exithandler (node:child_process:421:12) npm ERR! at ChildProcess.emit (node:events:514:28) npm ERR! at maybeClose (node:internal/child_process:1105:16) npm ERR! at ChildProcess._handle.onexit (node:internal/child_process:305:5) { npm ERR! code: 1, npm ERR! killed: false, npm ERR! signal: null, npm ERR! cmd: 'node scripts/deps-stage.js symlink ./lib/napi-v9' npm ERR! }

npm ERR! A complete log of this run can be found in: C:\Users\tobia\AppData\Local\npm-cache_logs\2023-09-10T09_33_52_091Z-debug-0.log`

I dont use any antivirus software or other stuff like that

I get the same errors. I think this has something to do with the fact that the @tensorflow folder is empty after the installation

gaikwadrahul8 commented 11 months ago

Hi, @TobCraft3521, @amacsmith , @georgzoeller, @Iust1n2

Apologize for the delayed response and I see you're using the Node.js v20.5.0 and Node.js v20.7.0 versions so could you please give it try by downgrading to Node.js v18.16.1 with python 3.8 or python 3.9 and try with npm install --save-exact @tensorflow/tfjs-node@3.18.0 it seems like solving the issue please refer this comment

Meanwhile, our developer team will try to solve this issue with latest version ofNode.js versions and will update you soon. Thank you!

gaikwadrahul8 commented 9 months ago

Hi, @TobCraft3521, @georgzoeller , @amacsmith, @Iust1n2

I apologize for the delayed response and I'm able to install tfjs-node on my Windows 10 system with these Node.js versions v19.9.0 and v18.16.1 with this command npm i @tensorflow/tfjs-node and it's installing the latest version of tfjs-node and I see currently you're using Node.js versions v20.7.0 and v20.5.0 and getting symlink ./lib/napi-v9 failed error message which is known issue to us and our relevant team is working on this issue to fix it as soon as possible.

I would request you to please give it try from your end and let us know is it working as expected or not with Node.js versions v19.9.0 and v18.16.1 ?

If you face any issue while installing tfjs-node on Windows system please feel free to create new issue with your error log, it would be helpful for us to investigate this issue further from our end and fix it as soon as possible.

@enyineer, You also please give it try and let us know, is it working as expected.

Thank you for your understanding and patience.

YaserFazil commented 9 months ago

Alright @gaikwadrahul8 , that's great, but I need the support of the latest version of NodeJS, so, is there an other way to solve this problem, please?

0x5ca1ab1e commented 9 months ago

My problem on windows was that ./lib/napi-v9 directory didn't exist and ./scripts/deps-stage.js on line 60 needed the path to symlink or copy the dll.

My fix was to make sure that the directory napi-v9 exists with: await fs.mkdir(path.dirname(destLibTensorFlowPath), {recursive: true});

I'm solving the copy issue with this patch(Edit: my patch was wrong)

diff --git a/scripts/deps-stage.js b/scripts/deps-stage.js
index 31c8dc397a993103cf0d055c0705fb6d93a5deda..75b69387e231b07a1f5e8aac065d685f7ad5dde8 100644
--- a/scripts/deps-stage.js
+++ b/scripts/deps-stage.js
@@ -22,6 +22,7 @@ const copy = util.promisify(fs.copyFile);
 const os = require('os');
 const rename = util.promisify(fs.rename);
 const symlink = util.promisify(fs.symlink);
+const mkdir = util.promisify(fs.mkdir);
 const {
   depsLibTensorFlowFrameworkPath,
   depsLibTensorFlowPath,
@@ -57,6 +58,7 @@ async function symlinkDepsLib() {
     throw new Error('Destination path not supplied!');
   }
   try {
+    await mkdir(path.dirname(destLibTensorFlowPath), {recursive: true});
     await symlink(
         path.relative(
             path.dirname(destLibTensorFlowPath), depsLibTensorFlowPath),
salos1982 commented 1 week ago

I have the same issue. Dll for windows in located in napi-v9 folder but binding in napi-v8. And when I load the tfjs-node node searches in napi-v8. So coping from napi-v9 to napi-v8 resolves the issue.