search

projectkudu / kudu

Kudu is the engine behind git/hg deployments, WebJobs, and various other features in Azure Web Sites. It can also run outside of Azure.
Apache License 2.0
3.12k stars 655 forks source link

KuduScript repo is obsolete and misleading #3479

Closed aawalton closed 1 year ago

aawalton commented 1 year ago

For deployment issues, please provide us with the following information:

Repro steps.

your project built successfully on your dev machine but failed on Azure? please write down your build tools and their versions (ie Msbuild 15.1.0.0)

What we're trying to accomplish.

 We're migrating a complex production server from Heroku to Azure App Service. We're unable to use the standard build automation for the following reasons. 1. We've standardized on `yarn` instead of `npm`. 1. The standard build automation using npm install. 1. While yarn and npm have a similar interface, there are enough differences that our repo will not build correctly with npm install.

Why we're using a custom Kudu deployment script.

 We started with the Azure docs [here](https://learn.microsoft.com/en-us/azure/app-service/configure-language-nodejs?pivots=platform-linux#customize-build-automation). We then followed the link to the Oryx configuration [here](https://github.com/microsoft/Oryx/blob/main/doc/configuration.md). While the Oryx configuration supports a `CUSTOM_BUILD_COMMAND`, we've found no evidence of a parallel `CUSTOM_INSTALL_COMMAND`. Since we need a custom install command, we can't use the standard build automation. We then following the link to the Azure deployment docs [here](https://learn.microsoft.com/en-us/azure/app-service/deploy-zip?tabs=cli#enable-build-automation-for-zip-deploy). From there we went to the Kudu documentation [here](https://github.com/projectkudu/kudu/wiki/Deploying-from-a-zip-file-or-url). This lead to the docs for taking over the deployment [here](https://github.com/projectkudu/kudu/wiki/Configurable-settings#take-over-the-whole-deployment-script), which lead to the Customizing deployments document [here](https://github.com/projectkudu/kudu/wiki/Customizing-deployments). While learning more about Kudu, we found the Kudu custom script generator instructions [here](https://github.com/projectkudu/kudu/wiki/Custom-Deployment-Script). Using the command `kuduscript -y --node` to generate a deployment script was mostly successful, with the exception of `selectNodeVersion` failing on deploy.

Project structures.

in order to reproduce your issue at our end we need a simple github repository that highlights structure of the project

How we created the reproduction repo

 [Current commit for reproduction repo](https://github.com/latitudegames/latitude-api-test/tree/7b96b9808350d2b1a9bb6fb105a562c171ac6fb4) 1. Initialized a new repo with a README from GitHub. 2. `yarn init` 3. Added empty `yarn.lock` 4. `yarn install` 5. `kuduscript -y --node`

How we created the azure app service.

 Created a new Azure App Service for Linux instance with default settings except the following. 1. Selected Node 16 LTS runtime stack (Node 18 LTS also appears to be affected) 2. Turned on GitHub Actions Workflow (which added the deployment workflow and publishing profile env vars to the repo).

The log/error given by the failure.

Normally this include a stack trace, error code and some more information.

Specific error messages.

 ```bash Run azure/webapps-deploy@v2 with: app-name: latitude-api-test slot-name: Production publish-profile: *** package: . Package deployment using ZIP Deploy initiated. Updating submodules. Preparing deployment for commit id '367bce[2](https://github.com/latitudegames/latitude-api-test/actions/runs/5762956206/job/15623785710#step:3:2)9-2'. PreDeployment: context.CleanOutputPath False PreDeployment: context.OutputPath /home/site/wwwroot Running custom deployment command... Not setting execute permissions for bash deploy.sh Running deployment command... Command: bash deploy.sh Handling node.js deployment. Kudu sync from: '/tmp/zipdeploy/extracted' to: '/home/site/wwwroot' Ignoring: .deployment Copying file: '.gitignore' Copying file: '.pnp.cjs' Copying file: '.pnp.loader.mjs' Copying file: 'README.md' Ignoring: deploy.sh Copying file: 'package-lock.json' Copying file: 'package.json' Copying file: 'yarn.lock' Ignoring: .git Copying file: '.github/workflows/main_latitude-api-test.yml' Copying file: '.yarn/install-state.gz' Copying file: 'node_modules/.package-lock.json' Copying file: 'node_modules/.bin/prebuild-install' Copying file: 'node_modules/.bin/rc' Copying file: 'node_modules/.bin/semver' Copying file: 'node_modules/b4a/LICENSE' Copying file: 'node_modules/b4a/README.md' Copying file: 'node_modules/b4a/browser.js' Copying file: 'node_modules/b4a/index.js' Copying file: 'node_modules/b4a/package.json' Copying file: 'node_modules/b4a/lib/ascii.js' Copying file: 'node_modules/b4a/lib/base64.js' Copying file: 'node_modules/b4a/lib/hex.js' Copying file: 'node_modules/b4a/lib/utf16le.js' Copying file: 'node_modules/b4a/lib/utf8.js' Copying file: 'node_modules/base64-js/LICENSE' Copying file: 'node_modules/base64-js/README.md' Copying file: 'node_modules/base64-js/base64js.min.js' Copying file: 'node_modules/base64-js/index.d.ts' Copying file: 'node_modules/base64-js/index.js' Copying file: 'node_modules/base64-js/package.json' Copying file: 'node_modules/bl/.travis.yml' Copying file: 'node_modules/bl/BufferList.js' Copying file: 'node_modules/bl/LICENSE.md' Copying file: 'node_modules/bl/README.md' Copying file: 'node_modules/bl/bl.js' Copying file: 'node_modules/bl/package.json' Copying file: 'node_modules/bl/test/convert.js' Copying file: 'node_modules/bl/test/indexOf.js' Copying file: 'node_modules/bl/test/isBufferList.js' Copying file: 'node_modules/bl/test/test.js' Copying file: 'node_modules/buffer/AUTHORS.md' Copying file: 'node_modules/buffer/LICENSE' Copying file: 'node_modules/buffer/README.md' Copying file: 'node_modules/buffer/index.d.ts' Copying file: 'node_modules/buffer/index.js' Copying file: 'node_modules/buffer/package.json' Copying file: 'node_modules/chownr/LICENSE' Omitting next output lines... /opt/Kudu/Scripts/selectNodeVersion.js:166 throw new Error('Unable to locate node.js installation directory at ' + nodejsDir); ^ Error: Unable to locate node.js installation directory at /opt/nodejs at Object. (/opt/Kudu/Scripts/selectNodeVersion.js:166:11) at Module._compile (internal/modules/cjs/loader.js:1085:14) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10) at Module.load (internal/modules/cjs/loader.js:950:[3](https://github.com/latitudegames/latitude-api-test/actions/runs/5762956206/job/15623785710#step:3:3)2) at Function.Module._load (internal/modules/cjs/loader.js:790:12) at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:76:12) at internal/main/run_main_module.js:17:[4](https://github.com/latitudegames/latitude-api-test/actions/runs/5762956206/job/15623785710#step:3:4)7 An error has occurred during web site deployment. select node version failed /opt/Kudu/Scripts/selectNodeVersion.js:166\n throw new Error('Unable to locate node.js installation directory at ' + nodejsDir);\n ^\n\nError: Unable to locate node.js installation directory at /opt/nodejs\n at Object. (/opt/Kudu/Scripts/selectNodeVersion.js:166:11)\n at Module._compile (internal/modules/cjs/loader.js:108[5](https://github.com/latitudegames/latitude-api-test/actions/runs/5762956206/job/15623785710#step:3:5):14)\n at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)\n at Module.load (internal/modules/cjs/loader.js:950:32)\n at Function.Module._load (internal/modules/cjs/loader.js:790:12)\n at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:7[6](https://github.com/latitudegames/latitude-api-test/actions/runs/5762956206/job/15623785710#step:3:6):12)\n at internal/main/run_main_module.js:1[7](https://github.com/latitudegames/latitude-api-test/actions/runs/5762956206/job/15623785710#step:3:8):[47](https://github.com/latitudegames/latitude-api-test/actions/runs/5762956206/job/15623785710#step:3:48)\n/opt/Kudu/Scripts/starter.sh bash deploy.sh Deployment Failed. deployer = GITHUB_ZIP_DEPLOY deploymentPath = ZipDeploy. Extract zip. Error: Failed to deploy web package to App Service. Error: Deployment Failed, Package deployment using ZIP Deploy failed. Refer logs for more details. App Service Application URL: https://latitude-api-test.azurewebsites.net/ ```

Debug your Azure website remotely.

it is recommanded that you share your Web App name, directly or indirectly we can take a look at what's going on.

Here's the App Service where our reproduction repo is currently deployed.

Mention any other details that might be useful.

I'm guessing we went wrong by trusting that the kuduscript generator was correct, as the blog post is 6 years old and the kuduscript generator repo hasn't been updated in 3 years. The instructions to download a deployment script from the Kudu portal in App Service also appear to be obsolete, since the Kudu portal doesn't have a Tools section. We're currently looking for a way to download the default Kudu script from Azure to get a correct starting point for the custom deployment script.

Thanks! We'll be in touch soon.

aawalton commented 1 year ago

Ok, I've found the actual deployment script in the Kudu portal at ~/site/deployments/tools/deploy.sh, and the generated script definitely does not have the selectNodeVersion function that was failing with the error, but it does show as using Node 16 LTS as the node version. It even does the install and build steps, but only if SCM_DO_BUILD_DURING_DEPLOYMENT=true, which means it's still relying on the code outside of the generated deploy script in order to do the install and build steps. Copying the deploy.sh script into the repo and using that as a custom build command does not do the install and build steps, even with SCM_DO_BUILD_DURING_DEPLOYMENT=true. Using a custom build script also does not set the node version correctly, which suggests that the node version, install, and build steps are being handled by the Oryx build, and not by kudu.

aawalton commented 1 year ago

For my use case, I'm going to try to use only the standard build automation with customization through Oryx instead of using a custom deployment script. Without updates to the KuduScript repo, I don't see any way to use a custom deployment script for this use case.