microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.49k stars 2.61k forks source link

vsts-npm-auth returns wrong Basic Realm (visualstudio.com instead of azure.com) #11649

Closed DynaSpan closed 4 years ago

DynaSpan commented 4 years ago

(Didn't know in which repo to create this issue so I chose this one)

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: bug

Enter Task Name: vsts-npm-auth

Environment

Azure DevOps npm v6.12.0 vsts-npm-auth v0.37.0 node v11.10.0 Windows 1903

Issue Description

npm install returns the error: Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/" while our package URL is https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/.

This does only happen on some projects; not all. We've migrated all .npmrc in the projects and the user's .npmrc to the dev.azure.com realm a couple of months ago.

vsts-npm-auth was run on all projects .npmrc and works for most projects. Forcing credential updates doesn't resolve it either, whether applied to the project's .npmrc or user's .npmrc.

vsts-npm-auth -C C:\Users\[xxx]\.npmrc -F                  
vsts-npm-auth v0.37.0.0
-----------------------
Getting new credentials for source:https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/, scope:vso.packaging_write vso.drop_write

Troubleshooting

I've tried clearing all NPM caches and node_modules, updating NPM & vsts-npm-auth, regenerating the credentials, removing package-lock.json. But nothing works unfortunately. Updating the credentials returns the correct URL:

PS C:\Users\[xxx]\AppData\Roaming\npm> vsts-npm-auth -C C:\Users\[xxx]\.npmrc

vsts-npm-auth v0.37.0.0
-----------------------
Already have credentials for https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/.
PS C:\Users\[xxx]\AppData\Roaming\npm> vsts-npm-auth -C C:\Projects\[xxx]\.npmrc

vsts-npm-auth v0.37.0.0
-----------------------
Already have credentials for https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/.

Error logs

0 info it worked if it ends with ok
1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli   'C:\\Users\\[xxx]\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli   'i' ]
2 info using npm@6.12.0
3 info using node@v11.10.0
4 verbose npm-session bc10c5eb62b72e40
5 silly install runPreinstallTopLevelLifecycles

[..]

38 silly pacote range manifest for @angular/cli@^8.3.9 fetched in 3036ms
39 http fetch GET 401 https://pkgs.dev.azure.com/[xxx]/_packaging/@[xxx]/npm/registry/@[xxx]%2fcore 100ms
40 silly fetchPackageMetaData error for @[xxx]/core@^0.1.1 Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
41 http fetch GET 401 https://pkgs.dev.azure.com/[xxx]/_packaging/@[xxx]/npm/registry/@[xxx]%2fstyle 27ms
42 silly fetchPackageMetaData error for @[xxx]/style@^0.9.0 Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
43 http fetch GET 200 https://registry.npmjs.org/@ngx-translate%2fcore 416ms
44 silly pacote range manifest for @ngx-translate/core@^11.0.1 fetched in 420ms

[..]

2285 silly pacote range manifest for @ngx-translate/http-loader@^4.0.0 fetched in 11ms
2286 silly resolveWithNewModule @ngx-translate/http-loader@4.0.0 checking installable status
2287 silly pacote range manifest for core-js@^3.3.2 fetched in 17ms
2288 silly resolveWithNewModule core-js@3.3.4 checking installable status
2289 silly pacote range manifest for rxjs@~6.4.0 fetched in 18ms
2290 silly resolveWithNewModule rxjs@6.4.0 checking installable status
2291 silly pacote range manifest for tippy.js@^4.3.5 fetched in 14ms
2292 silly resolveWithNewModule tippy.js@4.3.5 checking installable status
2293 silly pacote range manifest for tslib@^1.10.0 fetched in 15ms
2294 silly resolveWithNewModule tslib@1.10.0 checking installable status
2295 silly pacote range manifest for zone.js@~0.9.1 fetched in 14ms
2296 silly resolveWithNewModule zone.js@0.9.1 checking installable status
2297 http fetch GET 401 https://pkgs.dev.azure.com/[xxx]/_packaging/@[xxx]/npm/registry/@[xxx]%2fstyle 97ms
2298 http fetch GET 401 https://pkgs.dev.azure.com/[xxx]/_packaging/@[xxx]/npm/registry/@[xxx]%2fcore 98ms
2299 silly fetchPackageMetaData error for @[xxx]/style@^0.9.0 Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
2300 silly fetchPackageMetaData error for @[xxx]/core@^0.1.1 Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
2301 timing stage:rollbackFailedOptional Completed in 1ms
2302 timing stage:runTopLevelLifecycles Completed in 23371ms
2303 silly saveTree is-ph-meter@0.0.0
2303 silly saveTree +-- @angular/animations@8.2.12
2303 silly saveTree | `-- tslib@1.10.0
2303 silly saveTree +-- @angular/common@8.2.12
2303 silly saveTree +-- @angular/compiler@8.2.12
2303 silly saveTree +-- @angular/core@8.2.12
2303 silly saveTree +-- @angular/forms@8.2.12
2303 silly saveTree +-- @angular/platform-browser-dynamic@8.2.12
2303 silly saveTree +-- @angular/platform-browser@8.2.12
2303 silly saveTree +-- @angular/router@8.2.12
2303 silly saveTree +-- @aspnet/signalr@1.1.4
2303 silly saveTree +-- @fortawesome/fontawesome-pro@5.11.2
2303 silly saveTree +-- @ngx-translate/core@11.0.1
2303 silly saveTree +-- @ngx-translate/http-loader@4.0.0
2303 silly saveTree +-- chart.js@2.9.1
2303 silly saveTree +-- core-js@3.3.4
2303 silly saveTree +-- rxjs@6.4.0
2303 silly saveTree +-- tippy.js@4.3.5
2303 silly saveTree +-- tslib@1.10.0
2303 silly saveTree `-- zone.js@0.9.1
2304 verbose stack Error: Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
2304 verbose stack     at res.buffer.catch.then.body (C:\Users\[xxx]\AppData\Roaming\npm\node_modules\npm\node_modules\npm-registry-fetch\check-response.js:94:17)
2304 verbose stack     at processTicksAndRejections (internal/process/next_tick.js:81:5)
2305 verbose statusCode 401
2306 verbose pkgid @[xxx]/style@^0.9.0
2307 verbose cwd C:\my\project
2308 verbose Windows_NT 10.0.18362
2309 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\[xxx]\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "i"
2310 verbose node v11.10.0
2311 verbose npm  v6.12.0
2312 error code E401
2313 error Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
2314 verbose exit [ 1, true ]
DynaSpan commented 4 years ago

I've somehow managed to fix it by deleting the registry keys at HKEY_CURRENT_USER\SOFTWARE\Microsoft\VSCommon\14.0\ClientServices\TokenStorage\VisualStudio\VssApp and spamming the vsts-npm-auth tool till it finally worked. It still throws an exception but it finally generates a token and I can run npm install :).

PS C:\Projects\My-Project> vsts-npm-auth -F -R -V Detailed -C C:\Projects\My-Project\.npmrc -T C:\Users\[xxx]\.npmrc

vsts-npm-auth v0.37.0.0
-----------------------
Parameters:
  AuthenticationProviders=wia,federated
  Config=C:\Projects\My-Project\.npmrc
  Help=False
  NonInteractive=False
  TargetConfig=C:\Users\[xxx]\.npmrc
  ExpirationMinutes=129600
  ReadOnly=True
  Force=True
  Verbosity=Detailed
Creating npmrc file. Path: C:\Users\[xxx]\.npmrc
INI (resolved): @[xxx]:registry=https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/
INI (resolved): always-auth=true
Probing https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/
Probe response code: 401 Unauthorized
Credential type: Sps.
Has valid credentials: False.
Getting new credentials for source:https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/, scope:vso.packaging
Getting authentication token from https://vssps.dev.azure.com/[xxx]/
Trying authentication provider Windows Integrated Authentication via Azure AD...
Authorization failure while retrieving a session token
Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: VS30063: You are not authorized to access https://vssps.dev.azure.com.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.<SendAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__46.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__43`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__27`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__26`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.DelegatedAuthorization.Client.DelegatedAuthorizationHttpClient.<CreateSessionToken>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.DelegatedAuthorization.Client.DelegatedAuthorizationHttpClient.<CreateSessionToken>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.TaskExtensions.SyncResult[T](Task`1 task)
   at VSS.NuGet.Authentication.SpsAuthTokenProvider.GetAuthToken(Uri spsUri, TimeSpan sessionLength, Boolean allowInteractive, String scope, Boolean discardExistingCredentials)
Trying authentication provider Browser-based federated authentication...
SPS CreateSessionToken Activity ID: [xxx]
CreateSessionToken result: {"ClientId":"00000000-0000-0000-0000-000000000000","AccessId":"[xxx]","AuthorizationId":"[xxx]","HostAuthorizationId":"00000000-0000-0000-0000-000000000000","UserId":"[xxx]","ValidFrom":"2019-10-29T15:10:31.971952Z","ValidTo":"2020-01-27T15:20:18.9719678Z","DisplayName":"[xxx]","Scope":"vso.packaging","TargetAccounts":["[xxx]"],"Token":"[Redacted]","AlternateToken":null,"IsValid":true,"IsPublic":false,"PublicData":null,"Source":null}
Acquired JWT token valid from 29/10/2019 15:10:32 to 27/01/2020 15:20:19 (90.00:09:47 total, 89.23:59:39.9869983 remaining). Note that the server may reject the token before the end date due to revocation, etc.
bastienJS commented 4 years ago

I have the exact same error message(s) with the workaround he tried.. My user`s password had expired in azure devops thus the token in my npmrc file was invalid. I just generated a new PAT convert to base64 string and pasted it 2 times in the npmrc. Then npm install worked!

In my case there was no key in the registry set!

edgarrs commented 4 years ago

Please re-open it if this is still happening

aucampia commented 3 years ago

I was having the same error:

$ npm install
npm ERR! code E401
npm ERR! Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"

For me this was because my PAT on azure devops expired.

webbiffy commented 3 years ago

In my case the command vsts-npm-auth -config .npmrc generates PAT. Howerver when running npm install I got an error "403 Forbidden - In most cases, you or one of your dependencies are requesting a package version that is forbidden by your security policy." When I look at my PAT I can see there's read/write access in packaging, anyone having the same issue?

JamesHough commented 3 years ago

I had the same error message as the OP.

The only way that I could get it to work was to force the generation of a new token using the -f flag. I didn't delete any files or change the registry at all before running. If you are only downloading packages from your private npm on Azure DevOps, then the token can be read-only, you don't need a full access token. Read-only worked for me.

I used these switches to force a new read-only token to be generated on my account. This was the only way I could get the login dialog to appear so that I could provide the correct account to use with my company repo (not my personal login account).

vsts-npm-auth -config .npmrc -r -f -v normal

DynaSpan commented 3 years ago

So after having it working for a while, today I ran into this issue again The acquired token is not a JWT:

PS C:\Users\milan_drossaerts> vsts-npm-auth -C .npmrc -F -V Detailed -R

vsts-npm-auth v0.41.0.0
-----------------------
Parameters:
  AuthenticationProviders=wia,federated
  Config=.npmrc
  Help=False
  NonInteractive=False
  TargetConfig=
  ExpirationMinutes=129600
  ReadOnly=True
  Force=True
  Verbosity=Detailed
INI (resolved): //pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/registry/:username=VssSessionToken
INI (resolved): //pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/registry/:_password=<redacted ---sekretz--->
INI (resolved): //pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/registry/:email=not-used@example.com
INI (resolved): @[company-name]:registry=https://pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/registry/
INI (resolved): always-auth=true
INI (resolved): //pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/registry/:username=[org-name]
INI (resolved): //pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/registry/:_password=<redacted ---sekretz--->
INI (resolved): //pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/registry/:email=my.company@email.com
INI (resolved): //pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/:username=[org-name]
INI (resolved): //pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/:_password=<redacted ---sekretz--->
INI (resolved): //pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/:email=my.company@email.com
Probing https://pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/registry/
Probe response code: 401 Unauthorized
Credential type: Sps.
Has valid credentials: False.
Getting new credentials for source:https://pkgs.dev.azure.com/[org-name]/[project-name]/_packaging/[feed-name]/npm/registry/, scope:vso.packaging
Getting authentication token from https://vssps.dev.azure.com/[org-name]/
Trying authentication provider Windows Integrated Authentication via Azure AD...
Authorization failure while retrieving a session token
Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: VS30063: You are not authorized to access https://vssps.dev.azure.com.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.<SendAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__51.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.TaskExtensions.SyncResult[T](Task`1 task)
   at VSS.NuGet.Authentication.SpsAuthTokenProvider.GetAuthToken(Uri feedUri, Uri spsUri, TimeSpan sessionLength, Boolean allowInteractive, String scope, Boolean discardExistingCredentials)
Trying authentication provider Browser-based federated authentication...
SPS CreateSessionToken Activity ID: 092ae2eb-cf1b-40e7-a2b5-375070f864d6
The acquired token is not a JWT.

My .npmrc in my users folder:

@my-company:registry=https://pkgs.dev.azure.com/[company-name]/[project-name]/_packaging/[feed-name]/npm/registry/
always-auth=true
; begin auth token
//pkgs.dev.azure.com/[company-name]/[project-name]/_packaging/[feed-name]/npm/registry/:username=[company-name]
//pkgs.dev.azure.com/[company-name]/[project-name]/_packaging/[feed-name]/npm/registry/:_password=<-- redacted -->
//pkgs.dev.azure.com/[company-name]/[project-name]/_packaging/[feed-name]/npm/registry/:email=my.company@email.com
//pkgs.dev.azure.com/[company-name]/[project-name]/_packaging/[feed-name]/npm/:username=[company-name]
//pkgs.dev.azure.com/[company-name]/[project-name]/_packaging/[feed-name]/npm/:_password=<-- redacted -->
//pkgs.dev.azure.com/[company-name]/[project-name]/_packaging/[feed-name]/npm/:email=my.company@email.com
; end auth token

I don't see any dialog whatsoever prompting me for a login. I also have access to the feeds, because I can use the package URL npm i throws to download the package directly from the browser.

EDIT: So I can see it does create a new PAT everytime I run vsts-npm-auth (and places it in the .npmrc), however, I still end up with the error of it not being a JWT token. If I run the manual steps from DevOps (Artifact Feed > Connect to feed > NPM > Other) and follow steps 1 till 4, I still end up with a non-authorized setup....

DynaSpan commented 3 years ago

@edgarrs any workaround for this? I can't download my work packages meaning I can't do anything useful at work right now.

LukeGarrigan commented 3 years ago

Running vsts-npm-auth wasn't updating the password in my .npmrc file, so I did the following:

I regenerated my personal access token using:

vsts-npm-auth -config .npmrc -r -f -v detailed

I grabbed that personal access token and plugged into powershell:

[Convert]::ToBase64String([system.Text.Encoding]::UTF8.GetBytes("YOUR_PAT_GOES_HERE"))

I then put the output of that in the password of my .npmrc and reran

npm install

shuebner20 commented 2 years ago

I also experienced this problem with vsts-npm-auth v0.41.0.0. Even if I specify -T/-TargetConfig and -f as stated in several posts here, the existing .npmrc file in the project folder (which is also shell's current working folder) will not be updated at any time.

A fairly easy workaround was the following command:

vsts-npm-auth -config .npmrc -f -T .npmrc2

This will create a new file (.npmrc2) - just open it and copy the contents to the corresponding auth section in your existing .npmrc. There is no need to transform the PAT in any way (like base64 encoding).

lukos commented 2 years ago

Thanks @shuebner20 this was the only way I could get it to reset. This is honestly garbage if a flag like "force" does nothing. I uninstalled vsts-npm-auth and then reinstalled it, followed the instructions on Azure Devops and it still didn't work.

rafek1241 commented 4 months ago

Still issue exists, this workaround helped me.