search

projectkudu / kudu

Kudu is the engine behind git/hg deployments, WebJobs, and various other features in Azure Web Sites. It can also run outside of Azure.
Apache License 2.0
3.12k stars 652 forks source link

kudu deployment.sh started failing on selectnodeversion #3189

Closed timleyden closed 6 months ago

timleyden commented 4 years ago

See #3057 my deploy.sh has been working for months without issues recently i started getting the same behaviour referenced in the above issue. I worked around it be defining WEBSITE_NPM_DEFAULT_VERSION in configuration. I'd like to understand why it started failing all of a sudden. After exploring selectNodeVersion.js it looks like the script is failing on trying to open the path vbar npmVersionSpecFileForResolvedNodeVersion = path.resolve(nodePath, '../../npm.txt'); what is npm.txt? I cant find it anywhere in the filesystem.

Thanks

suwatch commented 4 years ago

npm.txt contains the info about npm version matching nodejs. Could you echo what is nodePath value?

timleyden commented 4 years ago

@suwatch Kudu deployment log shows: Resolved to version 12.16.3 Detecting npm version spec... An error has occurred during web site deployment. select node version failed ENOENT: no such file or directory, open '/opt/nodejs/12.16.3/npm.txt'

so i assume nodePath is /opt/nodejs/12.16.3/bin/node I searched under other version of nodejs and cannot find this file...

Thanks

suwatch commented 4 years ago

Umm .. is this Linux or Windows?

timleyden commented 4 years ago

Linux

suwatch commented 4 years ago

Unfortunately, the generated deployment script does not support Linux due to many assumptions on Windows VM.

timleyden commented 4 years ago

okay can you help me understand what changed as its been working for well over 6 months

suwatch commented 4 years ago

@sanchitmehta could you help with this? npm.txt is very specific to Windows and I doubt this ever works in Linux. It could be that the script never needs to hit this code path before - and now some condition led to this.

pesimeao commented 4 years ago

It seems that if you set these environment variables in you App Service Settings, everything runs smoothly. You just have to use versions that match.

WEBSITE_NODE_DEFAULT_VERSION=12
WEBSITE_NPM_DEFAULT_VERSION=6

This should be fixed in the script. Following the azure tutorial doesn't work because of this unfortunately.

ggirodda commented 3 years ago

I already tested this solution, with WEBSITE_NODE_DEFAULT_VERSION=14. The problem is still there, I had to comment those lines

SELECT_NODE_VERSION="$KUDU_SELECT_NODE_VERSION_CMD \"$DEPLOYMENT_SOURCE\" \"$DEPLOYMENT_TARGET\" \"$DEPLOYMENT_TEMP\""
eval $SELECT_NODE_VERSION
exitWithMessageOnError "select node version failed"

It would be nice if someone of the kudu project give us a real working solution, I have still problem with this.

sanchitmehta commented 3 years ago

We depend on Microsoft Oryx for base Image. You could load any stack version in Kudu via benv node=14 <command> this would ensure that the context for the particular stack version is loaded.

This would also work with benv dotnet=2.2

Let us know if you still face problem with this

sanchitmehta commented 3 years ago

Please also refer to: https://github.com/Azure-App-Service/KuduLite/wiki/SDK-Not-Present-in-Kudu to load an SCM into build context.

danechitoaie commented 3 years ago

Trying to make my NodeJS app work for hours... Why is the deployment process so hard and complicated? Unfortunately I'm forced to use Azure by our IT as else I'd be on a more decent platform.

danechitoaie commented 3 years ago

Anyone has any idea how to deploy a hello world nodejs app? Any working example would be appreciated.

strindhaug commented 3 years ago

@suwatch ehhh.. what do you mean by "npm.txt is very specific to Windows " ?

Assuming this is the correct source for selectNodeVersion, the only reference to npm.txt is within a if (process.platform === "linux") { block (notice first and last line in these lines):

https://github.com/projectkudu/kudu/blob/70763db2bd23ff1e54453d9b1fcbc638f5bf9153/Kudu.Core/Scripts/selectNodeVersion.js#L181-L214

Why would the linux specific part of the code specifically use a windows only file?!

I'm having the same issue, I need to update from the ancient v10.22.0 of node (https://endoflife.date/nodejs) which is the default version if I don't specify any "engines" in package.json to something at least above v12 in order for one of the other node packages to work.

But if I specify any "engines" in package.json it just fails (though annoyingly it also manages to delete all the files from the previous deploy before failing, so my develop app is now completely offline) after listing several node versions and seemingly successfully resolving to one of them:

remote: Detecting node version spec...        
remote: Using package.json engines.node value: >=14.17.4        
remote: Node.js versions available on the platform are: 8.0.0, 8.1.4, 8.2.1, 8.8.1, 8.9.4, 8.11.4, 8.12.0, 8.15.1, 8.17.0, 9.4.0, 10.1.0, 10.10.0, 10.14.2, 10.21.0, 10.23.0, 12.18.2, 12.22.4, 14.17.4.        
remote: ENOENT: no such file or directory, scandir '/opt/npm'        
remote: Resolved to version 14.17.4        
remote: Detecting npm version spec...        
remote: Using package.json engines.npm value: *        
remote: An error has occurred during web site deployment.        
remote: select node version failed        
remote: Deployment Logs : 'https://[redacted].scm.azurewebsites.net/jsonviewer?view_url=/api/deployments/[redacted]/log'

The only thing I think could be a clue for whatever goes wrong is the mysterious ENOENT: no such file or directory, scandir '/opt/npm' which is similar enough that google finds this issue when googling for it.

eanders-ms commented 2 years ago

I'm also seeing this issue. I deploy to an Azure App Service for Linux using a custom deploy.sh script, with only minor changes from the default. In my project's package.json, I specify engine versions for node and npm:

  "engines": {
      "node": ">=14",
      "npm": ">=6"
  }

When Kudu runs the script /opt/Kudu/Scripts/selectNodeVersion.js, it sets npmRootPath to /opt/npm, but that path doesn't exist on the machine:

Kudu Remote Execution Console
Type 'exit' to reset this console.
/home>ls -al /opt
total 88
drwxr-xr-x  31 root root  4096 Jan  5 02:12 .
drwxr-xr-x 121 root root  4096 Jan  5 02:12 ..
drwxr-xr-x   6 root root 20480 Dec 11 00:54 Kudu
drwxr-xr-x   3 root root 16384 Aug 26 03:32 buildscriptgen
drwxr-xr-x   7 root root  4096 Aug 26 03:38 dotnet
drwxr-xr-x   3 root root  4096 Aug 26 03:33 hugo
drwxr-xr-x  23 root root  4096 Aug 26 03:38 nodejs
drwxr-xr-x   2 root root  4096 Aug 26 03:35 oryx
drwxr-xr-x   7 root root  4096 Aug 26 03:39 php
drwxr-xr-x   3 root root  4096 Aug 26 03:34 php-composer
drwxr-xr-x   6 root root  4096 Aug 26 03:39 python
drwxr-xr-x   2 root root  4096 Aug 26 03:37 startupcmdgen
drwxr-xr-x   4 root root  4096 Aug 26 03:32 tmp
drwxr-xr-x   4 root root  4096 Jan  5 02:12 webssh
drwxr-xr-x   3 root root  4096 Aug 26 03:34 yarn
/home>

My build output:

Detecting node version spec...
Using package.json engines.node value: >=14
Node.js versions available on the platform are: 8.0.0, 8.1.4, 8.2.1, 8.8.1, 8.9.4, 8.11.4, 8.12.0, 8.15.1, 8.17.0, 9.4.0, 10.1.0, 10.10.0, 10.14.2, 10.21.0, 10.23.0, 12.18.2, 12.22.4, 14.17.4.
ENOENT: no such file or directory, scandir '/opt/npm'
Resolved to version 14.17.4
Detecting npm version spec...
Using package.json engines.npm value: >=6
An error has occurred during web site deployment.
select node version failed
ENOENT: no such file or directory, scandir '/opt/npm'\n/opt/Kudu/Scripts/starter.sh bash deploy.sh

This appears to be a bug in the selectNodeVersion.js script, around here: https://github.com/projectkudu/kudu/blob/70763db2bd23ff1e54453d9b1fcbc638f5bf9153/Kudu.Core/Scripts/selectNodeVersion.js#L150

eanders-ms commented 2 years ago

In the meantime, here's my workaround. I needed Node 14, so in deploy.sh:

# 2. Select node version
#selectNodeVersion
NPM_JS_PATH=/opt/nodejs/14.17.4/lib/node_modules/npm/bin/npm-cli.js
NODE_EXE=/opt/nodejs/14.17.4/bin/node
NPM_CMD="\"$NODE_EXE\" \"$NPM_JS_PATH\""
FreakinWard commented 2 years ago

In the meantime, here's my workaround. I needed Node 14, so in deploy.sh:

# 2. Select node version
#selectNodeVersion
NPM_JS_PATH=/opt/nodejs/14.17.4/lib/node_modules/npm/bin/npm-cli.js
NODE_EXE=/opt/nodejs/14.17.4/bin/node
NPM_CMD="\"$NODE_EXE\" \"$NPM_JS_PATH\""

Thanks for the suggeston, @eanders-ms. I'm trying it myself but kudu still seems to reference npm v6.x. I'm curious if I've missed something or if others are experiencing similar behaviors.

// deploy.sh

#selectNodeVersion
NODE_EXE=/opt/nodejs/14.17.4/bin/node
NPM_CMD="node /opt/nodejs/14.17.4/bin/npm"

# 3. Install npm packages
if [ -e "$DEPLOYMENT_TARGET/package.json" ]; then
  cd "$DEPLOYMENT_TARGET"
  echo NODE_EXE -v
  echo $NPM_CMD -v
  echo "Running $NPM_CMD install --production"
  eval $NPM_CMD install --production
  eval $NPM_CMD run build
  exitWithMessageOnError "npm failed"
  cd - > /dev/null
fi

From kudu bash, node -v returns v14.17.4 as expected. However, npm -v returns v6.14.14

// kudu bash cli

/opt/nodejs/14.17.4>/opt/nodejs/14.17.4/bin/node -v
v14.17.4
/opt/nodejs/14.17.4>/opt/nodejs/14.17.4/bin/npm -v
6.14.14
/opt/nodejs/14.17.4>

With the changes, the script appears to be referencing the expected node v14.17.4 path. 

// azure-pipeline log

0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node',
1 verbose cli   '/opt/nodejs/14.17.4/bin/npm',
1 verbose cli   'run',
1 verbose cli   'build' ]
2 info using npm@6.14.14
3 info using node@v10.22.0
4 verbose run-script [ 'prebuild', 'build', 'postbuild' ]
5 info lifecycle nextjs-boilerplate@0.1.0~prebuild: nextjs-boilerplate@0.1.0
6 info lifecycle nextjs-boilerplate@0.1.0~build: nextjs-boilerplate@0.1.0
7 verbose lifecycle nextjs-boilerplate@0.1.0~build: unsafe-perm in lifecycle true
8 verbose lifecycle nextjs-boilerplate@0.1.0~build: PATH: /opt/nodejs/14.17.4/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/home/site/wwwroot/node_modules/.bin:/home/site/deployments/tools:/opt/Kudu/Scripts:/usr/bin:/usr/bin:/usr/local/bin:/usr/local/bin:/usr/local/bin:/usr/local/bin:/usr/local/.rbenv/bin:/usr/local:/opt/oryx:/opt/nodejs/lts/bin:/opt/dotnet/lts:/opt/python/latest/bin:/opt/php/lts/bin:/opt/php-composer:/opt/yarn/stable/bin:/opt/hugo/lts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/nodejs/9/bin
9 verbose lifecycle nextjs-boilerplate@0.1.0~build: CWD: /home/site/wwwroot
10 silly lifecycle nextjs-boilerplate@0.1.0~build: Args: [ '-c', 'next build' ]
11 silly lifecycle nextjs-boilerplate@0.1.0~build: Returned: code: 1  signal: null
12 info lifecycle nextjs-boilerplate@0.1.0~build: Failed to exec build script
13 verbose stack Error: nextjs-boilerplate@0.1.0 build: `next build`
13 verbose stack Exit status 1
13 verbose stack     at EventEmitter.<anonymous> (/opt/nodejs/14.17.4/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
13 verbose stack     at EventEmitter.emit (events.js:198:13)
13 verbose stack     at ChildProcess.<anonymous> (/opt/nodejs/14.17.4/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack     at ChildProcess.emit (events.js:198:13)
13 verbose stack     at maybeClose (internal/child_process.js:982:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
14 verbose pkgid nextjs-boilerplate@0.1.0
15 verbose cwd /home/site/wwwroot
16 verbose Linux 5.4.0-1056-azure
17 verbose argv "/usr/local/bin/node" "/opt/nodejs/14.17.4/bin/npm" "run" "build"
18 verbose node v10.22.0
19 verbose npm  v6.14.14
20 error code ELIFECYCLE
21 error errno 1
22 error nextjs-boilerplate@0.1.0 build: `next build`
22 error Exit status 1
23 error Failed at the nextjs-boilerplate@0.1.0 build script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]

// https://{sitename}.scm.azurewebsites.net/api/diagnostics/runtime

{
  "nodejs": [

  ],
  "system": {
    "os_name": "Unix 5.4.0.1056",
    "os_build_lab_ex": "",
    "cores": 1
  }
}
eanders-ms commented 2 years ago

npm v6 is expected with node v14, so that looks right. But these two things in your output look suspicious:

1 verbose cli [ '/usr/local/bin/node',
1 verbose cli   '/opt/nodejs/14.17.4/bin/npm',

and

18 verbose node v10.22.0
19 verbose npm  v6.14.14

It appears to be using the default path for Node, but the explicit path for npm.

eanders-ms commented 2 years ago

I think I see the problem. Use this value for NPM_CMD:

NPM_CMD="\"$NODE_EXE\" \"$NPM_JS_PATH\""
FreakinWard commented 2 years ago

I think I see the problem. Use this value for NPM_CMD:

NPM_CMD="\"$NODE_EXE\" \"$NPM_JS_PATH\""

All in the details, this works, thanks @eanders-ms

In addition to making these changes, kudu deployments continued to fail. So, from the UI, I was able to remove all of the deployed files. Deploy and install then fired off perfectly 👍

jvano commented 6 months ago

Hi

If the problem persists and is related to running it on Azure App Service, please open a support incident in Azure: https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request

This way we can better track and assist you on this case

Thanks,

Joaquin Vano Azure App Service