search

npm / cli

the package manager for JavaScript
https://docs.npmjs.com/cli/
Other
8.36k stars 3.09k forks source link

[BUG] Integrity check fails for azure artifacts (and apparently other private registries) NPM packages #1800

Closed PaulVrugt closed 3 years ago

PaulVrugt commented 4 years ago

Current Behavior:

We are trying to upgrate to NPM 7, but we run into problems with getting packages from azure artifacts. The following log is shown, when running npm i --verbose:

npm WARN tarball tarball data for https://infolandcode.pkgs.visualstudio.com/_packaging/Shared/npm/registry/xxx/xxx.tgz (sha1-ZTIGEke6SQtyObZXul6THKgWSz0=) seems to be corrupted. Trying again.
npm WARN tarball tarball data for https://infolandcode.pkgs.visualstudio.com/_packaging/Shared/npm/registry/xxx/xxx.tgz (sha1-oYbrH56D5xdd2LzPKGiYWv/1Z9w=) seems to be corrupted. Trying again.
npm WARN tarball tarball data for https://infolandcode.pkgs.visualstudio.com/_packaging/Shared/npm/registry/xxx/xxx.tgz (sha1-ZTIGEke6SQtyObZXul6THKgWSz0=) seems to be corrupted. Trying again.
npm WARN tarball tarball data for https://infolandcode.pkgs.visualstudio.com/_packaging/Shared/npm/registry/xxx/xxx.tgz (sha1-oYbrH56D5xdd2LzPKGiYWv/1Z9w=) seems to be corrupted. Trying again.
npm timing reify:rollback:createSparse Completed in 2ms
npm timing reify:rollback:retireShallow Completed in 1ms
npm timing command:install Completed in 1784ms
npm verb stack Error: sha1-ZTIGEke6SQtyObZXul6THKgWSz0= integrity checksum failed when using sha1: wanted sha1-ZTIGEke6SQtyObZXul6THKgWSz0= but got sha512-1HAETTUJsxZ0ZT/s/kYXiDJgmm4bXD7co77wzewb4t8BBW7Vhk0ONkOs5orSToPCtu41U30n/e50gqTgia8anA== sha1-oCrDJJcm5EvGPkto9uzdg5hsPC0=. (307 bytes)

Expected Behavior:

I expect the packages to be downloaded correctly, just as they did using npm 6

Steps To Reproduce:

Reference packages to azure artifacts in your package.json. Try to run npm install

expected: packages install

actual: error as shown above

Environment:

leandro-manifesto commented 4 years ago

I'm also getting this with Gemfury's private registry, but the signature is a SHA512 in my case.

Both NPM 6.14.6 and Yarn 1.22.5 work properly, but NPM 7.0.0-beta.10 is calculating a different signature and failing the integrity check.

ValeryVS commented 3 years ago

Also getting such error with private registry and npm@7.0.0-beta.12. Install with npm@6.14.6, npm@6.14.8 and yarn@1.22.4 works properly.

jayphelps commented 3 years ago

Also getting this with a custom package installed from the github npm registry https://npm.pkg.github.com. Hopefully this is helpful correlation that it seems to be non-npm official registry related. Downgrading from 7.0.0-beta.12 to v6 worked.

integrity checksum failed when using sha512:
wanted sha512-X99IsgB5ElySriQwf/O4bimqNVHS9Wwij3KZLURUkC3dxO/028WcbiQRO2NtE2YJQgLkoFl8evwf0sR8jmxRFQ==
but got sha512-z4PhNX7vuL3xVChQ1m2AB9Yg5AULVxXcg/SpIdNs6c5H0NE8XYXysP+DGNKHfuwvY7kxvUdBeoGlODJ6+SfaPg==

Since this is a private package, I'm happy to do a Zoom with any team member and give them access if they are unable to reproduce independently.

PaulVrugt commented 3 years ago

@jayphelps I've tested downgrading to 7.0.0-beta.6 but this didn't help for me. The same issue:

npm ERR! sha1-ZTIGEke6SQtyObZXul6THKgWSz0= integrity checksum failed when using sha1: 
wanted sha1-ZTIGEke6SQtyObZXul6THKgWSz0= 
but got sha512-1HAETTUJsxZ0ZT/s/kYXiDJgmm4bXD7co77wzewb4t8BBW7Vhk0ONkOs5orSToPCtu41U30n/e50gqTgia8anA== sha1-oCrDJJcm5EvGPkto9uzdg5hsPC0=.
jayphelps commented 3 years ago

@PaulVrugt sorry, I meant v6.x.x, the current stable release, not a beta for v7

darcyclarke commented 3 years ago

We've potentially got a fix for this based on some work @nlf is about to land in a point release. I'll let him follow up though to confirm once we've got that out to have you try to reproduce.

nlf commented 3 years ago

We released v7.0.0-rc.2 last Friday and it includes some fixes that may relate to this, can you try updating to that version with npm i -g npm@next-7 and seeing if your issue is fixed?

leandro-manifesto commented 3 years ago

Installed npm@v7.0.0-rc2 and the integrity check still fails.

jayphelps commented 3 years ago

@nlf rc.2 solved the issue for me, or at least the problem "went away". Thank you!

Unclear why it fixed my issue but not for @leandro-manifesto. That's unfortunate.

leandro-manifesto commented 3 years ago

Just to be sure I ran npm i again just like before, with a clean cache, empty node_modules and no package-lock.json or yarn.lock. Still didn't work.

Here is the log: https://gist.github.com/leandro-manifesto/50bffbfb2af4d433630d388310923af3

PaulVrugt commented 3 years ago

I'll take a look in the morning to see if it is solved for us

PaulVrugt commented 3 years ago

@nlf I've updated to RC2 and I'm afraid the issue remains, just as with @leandro-manifesto

bonkydog commented 3 years ago

Hello, @jayphelps. I'd like to pair with you on diagnosing this problem. Is there a convenient time for you tomorrow (Wed, Oct 7) ? (Please include time zone.)

PaulVrugt commented 3 years ago

@bonkydog jayphelps stated that his problem went away with rc2

bonkydog commented 3 years ago

@PaulVrugt Would it be possible to share the tarball and and package.json files downloaded for the packages being reported as corrupt? This might help us figure out what is going on.

PaulVrugt commented 3 years ago

It's night time here now. I'm at gmt+2. I'll take a look in the morning at what I can share

bonkydog commented 3 years ago

OK, thanks!

PaulVrugt commented 3 years ago

@bonkydog

I've attachted the tarball of one of the packages failing. Is this what you're looking for? I doubt the issue has something to do with the tarball itself, since all packages downloaded from azure artifacts seem to fail. I've also included the logfile of the install attempt

The exact error:

npm WARN tarball tarball data for https://xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/@xxx/testmonkey-integration-script/-/testmonkey-integration-script-1.0.1.tgz (sha1-ZTIGEke6SQtyObZXul6THKgWSz0=) seems to be corrupted. Trying again.
npm WARN tarball tarball data for https://xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/@xxx/testmonkey-integration-script/-/testmonkey-integration-script-1.0.1.tgz (sha1-ZTIGEke6SQtyObZXul6THKgWSz0=) seems to be corrupted. Trying again.
npm ERR! code EINTEGRITY
npm ERR! sha1-ZTIGEke6SQtyObZXul6THKgWSz0= integrity checksum failed when using sha1: wanted sha1-ZTIGEke6SQtyObZXul6THKgWSz0= but got sha512-1HAETTUJsxZ0ZT/s/kYXiDJgmm4bXD7co77wzewb4t8BBW7Vhk0ONkOs5orSToPCtu41U30n/e50gqTgia8anA== sha1-oCrDJJcm5EvGPkto9uzdg5hsPC0=. (307 bytes)

testmonkey-integration-script-1.0.1.tgz.zip 2020-10-08T06_31_09_578Z-debug.log

bonkydog commented 3 years ago

Thanks, Paul!

jayphelps commented 3 years ago

Still want to pair? My problem did go away for what ever reason.

nlf commented 3 years ago

we published v7.0.0-rc.4 today that i believe will resolve this.

you can update with npm i -g npm@next-7 and please let us know how it goes.

i suspect at least some of you are going to get different errors now, but they'll be a better indicator about what's actually going wrong!

PaulVrugt commented 3 years ago

Cool! I'll give it a go and let you know the results

leandro-manifesto commented 3 years ago

Hey, tested with 7.0.0-rc.4, and now I'm getting 404 when fetching the tarball (the other requests are ok).

I'm just not sure if Gemfury is rejecting the authorization or if it really is a bad url.

nlf commented 3 years ago

If you curl the url (or try to fetch it with a browser) does it work? If so, it sounds like an issue with authentication

PaulVrugt commented 3 years ago

@nlf It seems indeed to be an authentication issue. We get a 401 response while trying to download packages from our private registry. We use the vsts-npm-auth package to authenticate, but it seems not to work with npm 7. Is this an issue with npm 7 or vsts-npm-auth?

the documentation in Azure devops states: Add a .npmrc to your project, in the same directory as your package.json

registry=https://pkgs.dev.azure.com/xxx_packaging/xxx/npm/registry/ 
always-auth=true

Then, run vsts-npm-auth to get an Azure Artifacts token added to your user-level .npmrc file

vsts-npm-auth -config .npmrc
nlf commented 3 years ago

I'm not familiar with that package at all, and I'm unable to find any code that I could review, so I'm not sure.

Do you know if there are implementation details documented anywhere? How does it modify the .npmrc file when you run it? Can you share the .npmrc file? Please be sure to remove any auth tokens first.

PaulVrugt commented 3 years ago

I can't find any information about it either. I'll post a topic in the microsoft github about it and link it here. The .npmrc doesn't get modified, it simply contains the url to the registry where it should get the packages.

nlf commented 3 years ago

It doesn't add a line like //pkgs.dev.azure.com/:_authToken=YOURTOKENHERE?

If not, and it's not doing something like replacing the registry url with a proxy that performs the authentication for you, then it's likely the auth is failing because we're not able to figure out how to actually do the authentication because it's not configured anywhere.

I think following up with that team is the right way to go. Please do get back to us and we'll see what we can do to help make this work correctly.

PaulVrugt commented 3 years ago

Microsoft issue: https://github.com/microsoft/azure-pipelines-tasks/issues/13697

PaulVrugt commented 3 years ago

No it doesn't add anything to the .npmrc file, so it probably hooks in somewhere else. I've reverted to npm6, and then the authentication is working as expected, and still no change to the .npmrc file. As you can see, I logged an issue with microsoft. Let's see what they come up with

Edit: As stated in the post below, it DOES make changes to the .npmrc, but only to the .npmrc file in the user profile

PaulVrugt commented 3 years ago

@nlf The plot thickens a bit. I've tried to use the .npmrc for authentication without using the vsts-npm-auth package. The contents of the .npmrc is now:

@xxx:registry=https://xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/

//xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/:username=VssToken
//xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/:_password=***
//xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/:email=VssEmail
//xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/:always-auth=true

and when installing packages, I see that is recognizes the auth, since it lists the same as the above in the log. But the authentication still fails. With the same configuration, npm 6 works like a charm.

963 verbose stack Error: Unable to authenticate, need: Bearer authorization_uri=https://login.windows.net/<guid>, Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/", TFS-Federated
963 verbose stack     at C:\npm\prefix\node_modules\npm\node_modules\npm-registry-fetch\check-response.js:113:17
963 verbose stack     at processTicksAndRejections (internal/process/task_queues.js:97:5)
964 verbose statusCode 401
965 verbose pkgid https://xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/@xxx/testmonkey-integration-script/-/testmonkey-integration-script-1.0.1.tgz

So even without the vsts-npm-auth the authentication seems to fail if we add authentication to the .npmrc file

Update: I discovered that the vsts-npm-auth package injects the credentials in the user .npmrc file (located at %userprofile%.npmrc). The content of the user .npmrc file is:

//xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/:username=VssSessionToken
; This is an unencrypted authentication token. Treat it like a password. DO NOT share this value with anyone, including Microsoft support.
//xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/:_password=xxx
; The npm client won't use username and _password without an email set, but the service doesn't use the email value. The following is an arbitrarily made-up address.
//xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/:email=not-used@example.com

So I think this means the authentication is set correctly in the .npmrc file, but npm 7 somehow doesn't use it, although when I check the logging, the .npmrc file from the user profile IS loaded during npm install

nlf commented 3 years ago

i think i see a bug in the code.. try removing the path segment of your registry url for the configuration values, but keep it in place for the registry= line and let me know if that helps

so

@xxx:registry=https://xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/
//xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/:username=VssSessionToken
//xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/:_password=xxx
//xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/:email=not-used@example.com

becomes

@xxx:registry=https://xxx.pkgs.visualstudio.com/_packaging/Shared/npm/registry/
//xxx.pkgs.visualstudio.com/:username=VssSessionToken
//xxx.pkgs.visualstudio.com/:_password=xxx
//xxx.pkgs.visualstudio.com/:email=not-used@example.com
PaulVrugt commented 3 years ago

@nlf

I've tried it, but it seems to make no difference at all. The error remain exactly the same :(

nlf commented 3 years ago

Thanks for giving it a try, I'll keep looking and see if I can figure out what's going on

nlf commented 3 years ago

Just to rule out a problem with auth, can I have you try one more thing..

The value after _password should be base64 encoded, can you decode that and try to make the request using curl with the username and password in the configuration and see if you receive the same 401 error?

You can decode the base64 with something like:

node -pe "Buffer.from('xxx', 'base64').toString('utf8')"

which will give you a plain text password, after which something like

curl -svo /dev/null 'https://username:password@the-full-and-real.url/to/the/file'

should print out all the response headers and the status code.

If that is also returning a 401 then the issue is outside of the scope of npm, however if that works then it would seem we're mishandling the auth somewhere.

PaulVrugt commented 3 years ago

I have a few days off work right now, but I'll try to give this a go today or tomorrow if my wife and children let me 😉

PaulVrugt commented 3 years ago

So, I found the time. And following your instructions, see the result of the Curl command below. It returns a 303 with a redirect. When I change the password in the curl command it returns a 401. So the credentials seem to be fine.

Edit: When adding the --location flag, it actually follows the redirect and downloads the package

*   Trying 13.107.42.18...
* TCP_NODELAY set
* Connected to xxx.pkgs.visualstudio.com (13.107.42.18) port 443 (#0)
* schannel: SSL/TLS connection with xxx.pkgs.visualstudio.com port 443 (step 1/3)
* schannel: checking server certificate revocation
* schannel: sending initial handshake data: sending 199 bytes...
* schannel: sent initial handshake data: sent 199 bytes
* schannel: SSL/TLS connection with xxx.pkgs.visualstudio.com port 443 (step 2/3)
* schannel: failed to receive handshake, need more data
* schannel: SSL/TLS connection with xxx.pkgs.visualstudio.com port 443 (step 2/3)
* schannel: encrypted data got 4096
* schannel: encrypted data buffer: offset 4096 length 4096
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with xxx.pkgs.visualstudio.com port 443 (step 2/3)
* schannel: encrypted data got 1024
* schannel: encrypted data buffer: offset 5120 length 5120
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with xxx.pkgs.visualstudio.com port 443 (step 2/3)
* schannel: encrypted data got 1024
* schannel: encrypted data buffer: offset 6144 length 6144
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with xxx.pkgs.visualstudio.com port 443 (step 2/3)
* schannel: encrypted data got 1024
* schannel: encrypted data buffer: offset 7168 length 7168
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with xxx.pkgs.visualstudio.com port 443 (step 2/3)
* schannel: encrypted data got 858
* schannel: encrypted data buffer: offset 8026 length 8192
* schannel: sending next handshake data: sending 158 bytes...
* schannel: SSL/TLS connection with xxx.pkgs.visualstudio.com port 443 (step 2/3)
* schannel: encrypted data got 326
* schannel: encrypted data buffer: offset 326 length 8192
* schannel: SSL/TLS handshake complete
* schannel: SSL/TLS connection with xxx.pkgs.visualstudio.com port 443 (step 3/3)
* schannel: stored credential handle in session cache
* Server auth using Basic with user 'VssSessionToken'
> GET /_packaging/Shared/npm/registry/@xxx/testmonkey-integration-script/-/testmonkey-integration-script-1.0.1.tgz HTTP/1.1
> Host: xxx.pkgs.visualstudio.com
> Authorization: Basic xxx
> User-Agent: curl/7.55.1
> Accept: */*
>
* schannel: client wants to read 102400 bytes
* schannel: encdata_buffer resized 103424
* schannel: encrypted data buffer: offset 0 length 103424
* schannel: encrypted data got 1415
* schannel: encrypted data buffer: offset 1415 length 103424
* schannel: decrypted data length: 1386
* schannel: decrypted data added: 1386
* schannel: decrypted data cached: offset 1386 length 102400
* schannel: encrypted data buffer: offset 0 length 103424
* schannel: decrypted data buffer: offset 1386 length 102400
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 1386
* schannel: decrypted data buffer: offset 0 length 102400
< HTTP/1.1 303 See Other
< Cache-Control: no-cache
< Pragma: no-cache
< Expires: -1
< Location: https://6egvsblobprodsu6weus36.blob.core.windows.net/b-7e5e1ac1118b4eadb4ae3f7165fe853c/1BEED41088483DAD90BC884E56E505D859406194F791A91CF57C09DE12BF9EA600.blob?sv=2019-02-02&sr=b&si=1&sig=cJbL3x91u3lvZnkhnR8d1sv2Umt42e3XnKbZ8FRONk0%3D&spr=https&se=2020-10-15T11%3A58%3A38Z&rscl=x-e2eid-f74f2a7d-d8c74d81-88cc7903-9595d8bd-session-f74f2a7d-d8c74d81-88cc7903-9595d8bd&rscd=attachment%3B%20filename%3D%22testmonkey-integration-script-1.0.1.tgz%22
< P3P: CP="CAO DSP COR ADMa DEV CONo TELo CUR PSA PSD TAI IVDo OUR SAMi BUS DEM NAV STA UNI COM INT PHY ONL FIN PUR LOC CNT"
< X-TFS-ProcessId: d416e68c-4fd0-4669-85eb-03855ef00a73
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< ActivityId: f74f2a7d-d8c7-4d81-88cc-79039595d8bd
< X-TFS-Session: f74f2a7d-d8c7-4d81-88cc-79039595d8bd
< X-VSS-E2EID: f74f2a7d-d8c7-4d81-88cc-79039595d8bd
< X-VSS-UserData: 8b69620c-dc8f-6e39-b800-b63402c40d6b:xxx@xxx.nl
< X-FRAME-OPTIONS: SAMEORIGIN
< X-Packaging-Migration: NpmBlobMetadata
< Request-Context: appId=cid-v1:f5d75a35-28cc-4e72-8007-1cf59e01402f
< Access-Control-Expose-Headers: Request-Context
< X-Content-Type-Options: nosniff
< X-MSEdge-Ref: Ref A: D211137342504A3090A2E577451F4315 Ref B: BRU30EDGE0320 Ref C: 2020-10-14T11:58:34Z
< Date: Wed, 14 Oct 2020 11:58:34 GMT
< Content-Length: 0
<
* Connection #0 to host xxx.pkgs.visualstudio.com left intact
nlf commented 3 years ago

ok @PaulVrugt I think we've finally got it. we released 7.0.1 last night, which you can install with the usual npm i -g npm@next-7 and should hopefully resolve this for you.

please let me know how it goes!

PaulVrugt commented 3 years ago

@nlf you rock! It works now.

raghda94 commented 3 years ago

@nlf I'm facing the same issue with npm v7.0.3 here's an output of what I get npm http fetch GET 200 https://registry.npmjs.org/aws-sdk/-/aws-sdk-2.778.0.tgz 31307ms npm timing reifyNode:node_modules/serverless/node_modules/aws-sdk Completed in 34351ms npm timing reify:rollback:createSparse Completed in 0ms npm timing reify:rollback:retireShallow Completed in 1ms npm timing command:install Completed in 36218ms npm verb stack Error: TAR_BAD_ARCHIVE: Unrecognized archive format npm verb stack at Unpack.warn (/usr/local/lib/node_modules/npm/node_modules/tar/lib/warn-mixin.js:19:40) npm verb stack at Unpack.warn (/usr/local/lib/node_modules/npm/node_modules/tar/lib/unpack.js:188:18) npm verb stack at Unpack.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:82:14) npm verb stack at Unpack.emit (node:events:339:22) npm verb stack at Unpack.[emit] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:286:12) npm verb stack at Unpack.[maybeEnd] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:402:17) npm verb stack at Unpack.[consumeChunk] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:434:21) npm verb stack at Unpack.write (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:365:25) npm verb stack at Unpack.end (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:479:14) npm verb stack at Yallist.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/minipass/index.js:396:18)

jdmarshall commented 3 years ago

I think Artifactory and friends occasionally suffer from connection reset issues when talking to clients. Given that the requests in question usually have a Content-Length header, my suspicion is that pacote does not verify Content-Length before attempting to validate the file it was given. We see this same problem both with NPM install operations and with a piece of code that uses pacote directly.

If the wrong number of bytes are sent there is no way the hash is going to validate, but premature end of stream should be the correct error message, not sha errors. Also parts of npm seem to cache these incomplete files, causing the same problem to happen when running a second time.