Closed mahmoudajawad closed 1 year ago
I tried to get around this by creating a .npmrc
file on the fly in the build command, but this does not work either
app_build_command: 'echo "registry=https://npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}" > .npmrc && npm install'
Nice thought @davemurphysf which asserts for some reason npm
command in SWA step is ignoring .npmrc
file completely.
So oryx has an environment variable that we can try, can you try adding this below the 'with' block for the build and deploy step? It might need quotes, haven't actually tried this out yet env: NPM_REGISTRY_URL: https://npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}
No luck for me. I tried using quotes and no quotes on the value of the environment variable NPM_REGISTRY_URL
. I also tried using a generated PAT instead of the GITHUB_TOKEN
. I also tried to do the build first and disable Oryx from building via DISABLE_NODEJS_BUILD: true
but then it complained that it could not determine the project type.
Here is my action yaml:
name: Azure Static Web Apps CI/CD
on:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened, closed]
branches:
- master
jobs:
build_and_deploy_job:
if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.action != 'closed')
runs-on: self-hosted
name: Build and Deploy Job
steps:
- uses: actions/checkout@v2
- name: Build And Deploy
id: builddeploy
uses: Azure/static-web-apps-deploy@v0.0.1-preview
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_SALMON_CLIFF_0735E8210 }}
repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
action: 'upload'
###### Repository/Build Configurations - These values can be configured to match you app requirements. ######
# For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
app_location: '/' # App source code path
app_artifact_location: 'build' # Built app content directory - optional
###### End of Repository/Build Configurations ######
env:
NPM_REGISTRY_URL: https://npm.pkg.github.com/:_authToken=${{ secrets.GPR_PAT }}
close_pull_request_job:
if: github.event_name == 'pull_request' && github.event.action == 'closed'
runs-on: ubuntu-latest
name: Close Pull Request Job
steps:
- name: Close Pull Request
id: closepullrequest
uses: Azure/static-web-apps-deploy@v0.0.1-preview
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_SALMON_CLIFF_0735E8210 }}
action: 'close'
Raw Log Output
2020-06-12T23:20:43.1564859Z ##[group]Run Azure/static-web-apps-deploy@v0.0.1-preview
2020-06-12T23:20:43.1564984Z with:
2020-06-12T23:20:43.1565430Z azure_static_web_apps_api_token: ***
2020-06-12T23:20:43.1565812Z repo_token: ***
2020-06-12T23:20:43.1565901Z action: upload
2020-06-12T23:20:43.1565979Z app_location: /
2020-06-12T23:20:43.1566059Z app_artifact_location: build
2020-06-12T23:20:43.1566140Z env:
2020-06-12T23:20:43.1566303Z NPM_REGISTRY_URL: https://npm.pkg.github.com/:_authToken=***
2020-06-12T23:20:43.1566411Z ##[endgroup]
2020-06-12T23:20:43.1586683Z ##[command]/usr/bin/docker run --name dbc4ec7617c60f44f1da6fa505890ffdec0_73a2b9 --label 549dbc --workdir /github/workspace --rm -e NPM_REGISTRY_URL -e INPUT_AZURE_STATIC_WEB_APPS_API_TOKEN -e INPUT_REPO_TOKEN -e INPUT_ACTION -e INPUT_APP_LOCATION -e INPUT_APP_ARTIFACT_LOCATION -e INPUT_API_BUILD_COMMAND -e INPUT_API_LOCATION -e INPUT_APP_BUILD_COMMAND -e INPUT_ROUTES_LOCATION -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/dave/actions-runner/_work/_temp/_github_home":"/github/home" -v "/home/dave/actions-runner/_work/_temp/_github_workflow":"/github/workflow" -v "/home/dave/actions-runner/_work/ft_web/ft_web":"/github/workspace" 549dbc:4ec7617c60f44f1da6fa505890ffdec0
2020-06-12T23:20:43.9544390Z [37mDeploymentId: 6398ef9d-4634-494f-a939-654b0ead882b[0m
2020-06-12T23:20:43.9544801Z [37m[0m
2020-06-12T23:20:44.0206926Z [37mApp Directory Location: '/' was found.[0m
2020-06-12T23:20:44.0207353Z [37mNo Api directory specified. Azure Functions will not be created.[0m
2020-06-12T23:20:45.1439330Z [37mStarting to build app with Oryx[0m
2020-06-12T23:20:45.1442019Z [33mAzure Static Web Apps utilizes Oryx to build both static applications and Azure Functions. You can find more details on Oryx here: https://github.com/microsoft/Oryx[0m
2020-06-12T23:20:45.1442646Z [37m---Oryx build logs---[0m
2020-06-12T23:20:45.1443131Z [37m[0m
2020-06-12T23:20:45.1443442Z [37m[0m
2020-06-12T23:20:45.5892281Z [37mBuild orchestrated by Microsoft Oryx, https://github.com/Microsoft/Oryx[0m
2020-06-12T23:20:45.5892917Z [37mYou can report issues at https://github.com/Microsoft/Oryx/issues[0m
2020-06-12T23:20:45.5893115Z [37m[0m
2020-06-12T23:20:45.5900460Z [37mOryx Version : 0.2.20200521.2, Commit: 409adf05fdb5f3d6e9b99c6c60597c58eb409524, ReleaseTagName: 20200521.2[0m
2020-06-12T23:20:45.5900903Z [37mBuild Operation ID: |PorEe0AEbGA=.6574302c_[0m
2020-06-12T23:20:45.5901090Z [37m[0m
2020-06-12T23:20:46.5933021Z [37mUsing intermediate directory '/bin/staticsites/ss-oryx/app-int'.[0m
2020-06-12T23:20:46.5955355Z [37m[0m
2020-06-12T23:20:46.5957876Z [37mIntermediate directory doesn't exist, creating it...'[0m
2020-06-12T23:20:46.5958960Z [37m[0m
2020-06-12T23:20:46.5959285Z [37mCopying files to the intermediate directory...[0m
2020-06-12T23:20:46.6897143Z [37mDone in 0 sec(s).[0m
2020-06-12T23:20:46.6897853Z [37m[0m
2020-06-12T23:20:46.6899636Z [37mSource directory : /bin/staticsites/ss-oryx/app-int[0m
2020-06-12T23:20:46.6900509Z [37mDestination directory: /bin/staticsites/ss-oryx/app[0m
2020-06-12T23:20:46.6902003Z [37m[0m
2020-06-12T23:20:46.6902842Z [37m[0m
2020-06-12T23:20:46.6903361Z [37mDownloading and extracting nodejs version 12.16.3 to /tmp/oryx/platforms/nodejs/12.16.3...[0m
2020-06-12T23:20:47.3093285Z [37mDownloaded in 1 sec(s).[0m
2020-06-12T23:20:47.3094301Z [37mVerifying checksum...[0m
2020-06-12T23:20:47.3721997Z [37mExtracting contents...[0m
2020-06-12T23:20:47.8482713Z [37mDone in 1 sec(s).[0m
2020-06-12T23:20:47.8483396Z [37m[0m
2020-06-12T23:20:47.8596382Z [37m[0m
2020-06-12T23:20:47.8597112Z [37mUsing Node version:[0m
2020-06-12T23:20:47.8614659Z [37mv12.16.3[0m
2020-06-12T23:20:47.8621492Z [37m[0m
2020-06-12T23:20:47.8622883Z [37mUsing Npm version:[0m
2020-06-12T23:20:47.9755707Z [37m6.14.4[0m
2020-06-12T23:20:47.9816372Z [37m[0m
2020-06-12T23:20:47.9816980Z [37mInstalling production dependencies in '/bin/staticsites/ss-oryx/app-int/__oryx_prod_node_modules'...[0m
2020-06-12T23:20:47.9817476Z [37m[0m
2020-06-12T23:20:47.9818099Z [37mRunning 'npm install --unsafe-perm --production'...[0m
2020-06-12T23:20:47.9818985Z [37m[0m
2020-06-12T23:20:53.8088934Z [31mnpm WARN tar ENOENT: no such file or directory, open '/bin/staticsites/ss-oryx/app-int/__oryx_prod_node_modules/node_modules/.staging/minipass-pipeline-bca0161e/LICENSE'[0m
<lots more no such file errors>
@kichalla any ideas here?
I have a theory. It seems that sometime during the build, Orxy copies the package.json to another directory to only install the production dependencies:
I stumbled upon this because I have a postinstall
script that runs a JS file in my project which is missing in the GitHub Actions build.
Installing production dependencies in '/bin/staticsites/ss-oryx/app-int/__oryx_prod_node_modules'...
Running 'npm install --unsafe-perm --production'...
> @ngxp/demo@0.0.1 postinstall /bin/staticsites/ss-oryx/app-int/__oryx_prod_node_modules
> node ./decorate-angular-cli.js && ngcc --properties es2015 browser module main --first-only --create-ivy-entry-points
internal/modules/cjs/loader.js:960
throw err;
^
Error: Cannot find module '/bin/staticsites/ss-oryx/app-int/__oryx_prod_node_modules/decorate-angular-cli.js'
However, I have no idea what to do about that...
@miwebst @kichalla, updates on whether this is resolvable or not?
@mahmoudajawad Do you have a repository with the repro project that I can try with?
@davemurphysf If possible could you share your repro project?
Unfortunately @kichalla , it's private code.
@mahmoudajawad from your original post, so you are still seeing the package being tried from the NPM registry and not the GitHub registry. I just now tried a scenario with the following content in .npmrc file and it worked.
Example content in the repo's .npmrc file
registry=https://registry.npmjs.org/
@kichalla:registry=https://npm.pkg.github.com/
In your GitHub actions workflow, before the static web apps actions get executed you could do the following:
echo "registry=https://npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}" >> .npmrc
Ignore my earlier example, the following content in .npmrc file worked fine for me:
Here the ${GITHUB_TOKEN}
is expanded by npm when ‘npm install’ is called. You just need to make sure this variable's value is available as an environment variable.
registry=https://registry.npmjs.org/
@kichalla:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}
@kichalla, this doesn't seem to work when the package is in another repo.
@kichalla, this doesn't seem to work when the package is in another repo.
Ok, just curious...irrespective of static web apps, how does it work when you do it locally on your development machine?
.npmrc
file is all what is needed. On local machine or Azure DevOps, it works internally because release pipelines have access to all repos the user created the release pipeline have access to. I have never run into such a scenario on Github Action to confirm, but I could spare sometime and create a separate Actions pipeline that attempts to build project artifacts and prepare them for download using simple build commands and see the results. Shall I do this?
@kichalla has suggested a resolution for this issue which has been spotted and diagnosed by @getfatday, and it has worked with me. The resolution was to move the private-hosted package from dependencies
to devDependencies
in package.json
and it got the workflow to successfully build and deploy the app.
Ref: https://github.com/microsoft/Oryx/issues/636#issuecomment-657013125
Hi all.
I am still having this issue after trying out all your tips. The repo i am trying to add is the Fontawesome Pro ones. this is the error i am getting..
npm ERR! code E404 npm ERR! 404 Not Found - GET https://registry.npmjs.org/@fortawesome%2fpro-light-svg-icons - Not found npm ERR! 404 npm ERR! 404 '@fortawesome/pro-light-svg-icons@5.13.0' is not in the npm registry. npm ERR! 404 You should bug the author to publish it (or use the name yourself!) npm ERR! 404 It was specified as a dependency of '__oryx_prod_node_modules' npm ERR! 404 npm ERR! 404 Note that you can also install from a npm ERR! 404 tarball, folder, http url, or git url.
i have the .npmrc file working locally and it is in the my repo i tried running the config commands again in app_build_command i also tried what @getfatday suggested yet nothing seems to be working.
any thing else i could try
@alikazai, try to build your project on clean container or VM so you can confirm the settings of the project are correct, and not some cached credentials and tokens are allowing you to access locally, and not on Github Actions. This is common issue. Alas, you could be hitting on another problem that could be or could be not related to this issue.
I am seeing this as well, using a gitlab's npm registry. It appears that the .npmrc file is being ignored as well as the environment variables sent to the docker execution. To cover all bases, I added the following in the environment settings:
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
NPM_REGISTRY_URL: https://gitlab.com/api/v4/packages/npm/:_authToken=${{ secrets.NPM_TOKEN }}
INPUT_REGISTRY-URL: https://gitlab.com/api/v4/packages/npm/:_authToken=${{ secrets.NPM_TOKEN }}
They are environment variables set as a part of the docker run syntax. In addition, what I observed is that while GitHub Action syntax permits
registry-url: https://gitlab.com/api/v4/packages/npm/:_authToken=${{ secrets.NPM_TOKEN }}
the SWA setup does not. It does not honor the .npmrc file, whether it is at the top of the project, or inserted as previous comments have suggested. And yet, it is clearly processed, because using an invalid environment variable within it, causes the build to fail immediately. So it appears that GitHub Action is performing its substitution.
Thanks @jracc that seemed to have fixed it.
Hey @alikazai - what fixed what?
@jracc
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
NPM_REGISTRY_URL: https://gitlab.com/api/v4/packages/npm/:_authToken=${{ secrets.NPM_TOKEN }}
INPUT_REGISTRY-URL: https://gitlab.com/api/v4/packages/npm/:_authToken=${{ secrets.NPM_TOKEN }}
you said to use this i did that but tweaked it for fontawesome and it worked the build from the private npm packages worked fine.
@alikazai - thanks. I'm glad it helped you. @kichalla - Any reason why the build ignores my .npmrc and still assumes the registry is registry.npmjs.org?
Has anyone made progress in solving this issue? I'm trying to deploy an Angular app that has an extra library in a private github package that fails on the npm install portion of the Build and Deploy step.
I tried adding a step before it that made use of the node-setup action and was able to successfully have it run npm install with the correct repos referenced. But it appears that the SWA step ignores those already installed node_modules and runs the npm install itself again and fails.
Here's my Actions workflow in a hope it would help those who are still having issues:
name: Azure Static Web Apps CI/CD
on:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened, closed]
branches:
- master
jobs:
build_and_deploy_job:
if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.action != 'closed')
runs-on: ubuntu-latest
name: Build and Deploy Job
steps:
- uses: actions/checkout@v2
- name: NPM Token
run: |
echo "@ORG:registry=https://npm.pkg.github.com/ORG
//npm.pkg.github.com/:_authToken=$NPM_TOKEN
always-auth=true" > .npmrc
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
- name: Confirm NPM Token
run: cat .npmrc
- name: Build And Deploy
id: builddeploy
uses: Azure/static-web-apps-deploy@v0.0.1-preview
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_XXXXXXXXXXX }}
repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
action: 'upload'
###### Repository/Build Configurations - These values can be configured to match you app requirements. ######
# For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
app_location: '.' # App source code path
app_artifact_location: './dist/OUTPUT' # Built app content directory - optional
###### End of Repository/Build Configurations ######
close_pull_request_job:
if: github.event_name == 'pull_request' && github.event.action == 'closed'
runs-on: ubuntu-latest
name: Close Pull Request Job
steps:
- name: Close Pull Request
id: closepullrequest
uses: Azure/static-web-apps-deploy@v0.0.1-preview
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_XXXXXXXXXXX }}
action: 'close'
Notice, there's a step that prints out .npmrc
file. I've added this later when it was suggested .npmrc
file is not being copied over by Oryx. This would allow you to make sure you have the correct .npmrc
file in place.
I have struggle really long time now and is frustrated that this is not documented anywhere? Tried all you suggestions above but none of them work for me. It all works locally as my .npmrc file is then being read as it should!
My private repo is at: https://pkgs.dev.azure.com/XXXXX/_packaging/MYnpm/npm/registry/
My yml-file looks like this:
name: Azure Static Web Apps CI/CD
on:
push:
branches:
- main
pull_request:
types: [opened, synchronize, reopened, closed]
branches:
- main
jobs:
build_and_deploy_job:
if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.action != 'closed')
runs-on: ubuntu-latest
name: Build and Deploy Job
steps:
- uses: actions/checkout@v2
with:
submodules: true
- name: NPM Token
run: |
echo "@ORG:registry=https://pkgs.dev.azure.com/XXXXX/_packaging/MYnpm/npm/registry/
//pkgs.dev.azure.com/:_authToken=$NPM_TOKEN
always-auth=true" > .npmrc
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
- name: Confirm NPM Token
run: cat .npmrc
- name: Build And Deploy
id: builddeploy
uses: Azure/static-web-apps-deploy@v0.0.1-preview
with:
app_build_command: "npm run build-production"
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_XXXXX }}
repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
action: "upload"
###### Repository/Build Configurations - These values can be configured to match you app requirements. ######
# For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
app_location: "/" # App source code path
api_location: "api" # Api source code path - optional
output_location: "dist" # Built app content directory - optional
###### End of Repository/Build Configurations ######
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
NPM_REGISTRY_URL: https://pkgs.dev.azure.com/XXXXX/_packaging/MYnpm/npm/registry/:_authToken=${{ secrets.NPM_TOKEN }}
INPUT_REGISTRY-URL: https://pkgs.dev.azure.com/XXXXX/_packaging/MYnpm/npm/registry/:_authToken=${{ secrets.NPM_TOKEN }}
close_pull_request_job:
if: github.event_name == 'pull_request' && github.event.action == 'closed'
runs-on: ubuntu-latest
name: Close Pull Request Job
steps:
- name: Close Pull Request
id: closepullrequest
uses: Azure/static-web-apps-deploy@v0.0.1-preview
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_XXXXX }}
action: "close"
I'm on the same boat as @persyl.
This has been very frustrating -- I've got an Azure Static Web App trying to get it deployed through GitHub Actions that need to authenticate with our private NPM Registry stored in Azure DevOps but I always get the now deeply-hated
npm ERR! code E401 npm ERR! Unable to authenticate, need: *** Basic realm="https://pkgsprodcus1.pkgs.visualstudio.com/", TFS-Federated
error.
I've tried every single tip/idea in this and lots of other posts/threads; however, I'm still stuck.
Has anyone found a solution/workaround?
You guys can try install all the packages outside Azure/static-web-apps-deploy@v1, and only use azure web app GitHub action to deploy it.
+1
Hi, Guys. I have gotten this same problem with Azure Pipelines. After some testing, it works for me:
trigger:
- poc/static-web-app
pool:
vmImage: ubuntu-latest
steps:
- checkout: self
submodules: true
- task: NodeTool@0
displayName: 'Use Node 16.17.1'
inputs:
versionSpec: 16.17.1
- task: DownloadSecureFile@1
name: npmrc
inputs:
secureFile: 'npmrc_2021_12'
- script: |
mv $(npmrc.secureFilePath) .npmrc
displayName: 'Move secure file'
- task: AzureStaticWebApp@0
inputs:
app_build_command: 'yarn build:dev --verbose'
app_location: '/'
output_location: 'dist/development'
azure_static_web_apps_api_token: $(AZURE_STATIC_WEB_APPS_API_TOKEN_PROUD_ROCK_0313AD910)
Hope it helps someone. In a nutshell, it started working when I put the npmrc task after this:
- checkout: self
submodules: true
Similar to @persyl I was able to get it to work by committing the .npmrc
file to the root of the project:
@our-scope:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=${NPM_GITHUB_AUTH_TOKEN}
Then set the NPM_GITHUB_AUTH_TOKEN
in the github secrets.
To send env variables over to azure they can be defined under the with
section, as noted in the documentation:
- name: Build And Deploy
id: builddeploy
uses: Azure/static-web-apps-deploy@v1
with:
app_build_command: "npm run build"
.
.
.
env:
NPM_GITHUB_AUTH_TOKEN: ${{ secrets.NPM_GITHUB_AUTH_TOKEN }}
I'm going to close this now as I believe the different proposed solutions already cover all scenarios.
I also encountered the same issue. However, in my case, it might be slightly different. It's not a private package; instead, it's a personal package that I hosted on Github Packages. Thus, it has a different registry URL. I am also using Docker to package my application. My solution to this issue was to copy the '.npmrc' file before running npm install, like this:
FROM node:16-alpine as dependencies
WORKDIR /app
# This is what i changed. (Notice the forward slash after the dot, it matters when you want to copy multiple files
COPY package.json .npmrc ./
RUN npm i
COPY . .
FROM dependencies as builder
RUN npm run build
EXPOSE 5000
CMD npm run start
This is my .npmrc
file:
registry=https://registry.npmjs.org/
@<organization>:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=<auth-token>
Hello.
I'm giving SWA some advanced scenario where I've published a private package on Github Packages Registry in a repo I own, then I created an app repo that uses this package in it. Locally this is working very well with a
.npmrc
file that has a PAT issued with the correct scopes. However, SWA Actions step fails to detect the package is from another registry. I'm not sure why because.npmrc
should be enough to handle this, unless at time of files process by SWA Actions something is happening and it's causing.npmrc
file not to be taken into consideration.Error message in Actions log: