Closed adurand-ctie closed 2 months ago
Probably related to https://github.com/nodejs/undici/issues/2200.
Probably related to nodejs/undici#2200.
To overcome this, should I try RUN set NODE_TLS_REJECT_UNAUTHORIZED=0
?
If this is the case, it didn't work :(
Could there be that you need to specify a COREPACK_NPM_REGISTRY
in your environment?
Still no luck, even with COREPACK_NPM_REGISTRY.
However, there is something weird. Because when I set this env variable to a fake value, like RUN set COREPACK_NPM_REGISTRY=https://www.microsoft.com/
(even between quotes), I still see that it does the request towards registry.npmjs.org:
Internal Error: Error when performing the request to https://registry.npmjs.org/pnpm; for troubleshooting help, see https://github.com/nodejs/corepack#troubleshooting
Seeing the same issue here with corepack 0.25.x NODE_EXTRA_CA_CERTS and HTTP_PROXY env variables work as expected in 0.24.x
I've been digging a little more into this issue as I also use a proxy (squid-cache, no proxy auth, no tls inspection).
First, I don't believe this is an injected certificate issue since my proxy only relays connections out without any TLS inspection.
Second, since I fully control my local DNS server and the proxy, I have noticed no connection attempt is made to connect to the defined proxy. However, inspecting the agent
object (https://github.com/nodejs/corepack/blob/v0.25.2/sources/httpUtils.ts#L11), it appears to be valid. So the error is fetch()
is used (https://github.com/nodejs/corepack/blob/v0.25.2/sources/httpUtils.ts#L15)
Rolling backward, I tried to find what were the changes introduced between 0.24.1
(last known working version for me) and 0.25.0
(first version with that issue). Doing a git blame
and I found the PR https://github.com/nodejs/corepack/pull/365 introduced significant changes in how external data is fetched. Previously https.get(...)
was used, but now it's globalThis.fetch(...)
. Either somehow globalThis.fetch()
is broken, or the proxy agent isn't set as expected which causes the internal error.
Last, I found that downgrading to corepack 0.24.1
(npm install -g corepack@0.24.1
), then running yarn --version
will populate ~/.cache/node/corepack
. I believe during this process, tags are retrieved and checked, and the latest known version is stored in ~/.cache/node/corepack/lastKnownGood.json
Then upgrading back to 0.25.2
and running the same command prompts the user to download and install yarn again. Answering yes
leads to the same internal error as I described above. So it seems like using globalThis.fetch()
breaks more corepack functionalities.
Tagging both @merceyz and @aduh95 who have authored the PR mentioned above for visibility.
When dumped, my ProxyAgent looked like this.
ProxyAgent {
_events: [Object: null prototype] {},
_eventsCount: 0,
_maxListeners: undefined,
[Symbol(shapeMode)]: false,
[Symbol(kCapture)]: false,
[Symbol(destroyed)]: false,
[Symbol(onDestroyed)]: null,
[Symbol(closed)]: false,
[Symbol(onClosed)]: [],
[Symbol(proxy agent options)]: { uri: 'http://proxy.local-network.net:3129', protocol: 'https' },
[Symbol(proxy agent)]: Agent {
_events: [Object: null prototype] {},
_eventsCount: 0,
_maxListeners: undefined,
[Symbol(shapeMode)]: false,
[Symbol(kCapture)]: false,
[Symbol(destroyed)]: false,
[Symbol(onDestroyed)]: null,
[Symbol(closed)]: false,
[Symbol(onClosed)]: [],
[Symbol(dispatch interceptors)]: [ [Function (anonymous)] ],
[Symbol(options)]: {
uri: 'http://proxy.local-network.net:3129',
connect: [AsyncFunction: connect],
interceptors: undefined
},
[Symbol(maxRedirections)]: 0,
[Symbol(factory)]: [Function: defaultFactory],
[Symbol(clients)]: Map(0) {},
[Symbol(onDrain)]: [Function (anonymous)],
[Symbol(onConnect)]: [Function (anonymous)],
[Symbol(onDisconnect)]: [Function (anonymous)],
[Symbol(onConnectionError)]: [Function (anonymous)]
},
[Symbol(dispatch interceptors)]: [],
[Symbol(request tls settings)]: undefined,
[Symbol(proxy tls settings)]: undefined,
[Symbol(proxy headers)]: {},
[Symbol(connect endpoint function)]: [Function: connect],
[Symbol(proxy client)]: <ref *1> Pool {
_events: [Object: null prototype] {},
_eventsCount: 0,
_maxListeners: undefined,
[Symbol(shapeMode)]: false,
[Symbol(kCapture)]: false,
[Symbol(destroyed)]: false,
[Symbol(onDestroyed)]: null,
[Symbol(closed)]: false,
[Symbol(onClosed)]: [],
[Symbol(queue)]: FixedQueue {
tail: [FixedCircularBuffer],
head: [FixedCircularBuffer]
},
[Symbol(clients)]: [],
[Symbol(queued)]: 0,
[Symbol(onDrain)]: [Function: onDrain],
[Symbol(onConnect)]: [Function (anonymous)],
[Symbol(onDisconnect)]: [Function (anonymous)],
[Symbol(onConnectionError)]: [Function (anonymous)],
[Symbol(stats)]: PoolStats { [Symbol(pool)]: [Circular *1] },
[Symbol(dispatch interceptors)]: [],
[Symbol(connections)]: null,
[Symbol(url)]: URL {
href: 'http://proxy.local-network.net:3129/',
origin: 'http://proxy.local-network.net:3129',
protocol: 'http:',
username: '',
password: '',
host: 'proxy.local-network.net:3129',
hostname: 'proxy.local-network.net',
port: '3129',
pathname: '/',
search: '',
searchParams: URLSearchParams {},
hash: ''
},
[Symbol(options)]: {
connect: [Function: connect],
allowH2: undefined,
interceptors: undefined
},
[Symbol(factory)]: [Function: defaultFactory]
}
}
Hoping this will be helpful.
Seeing the same issue here with corepack 0.25.x NODE_EXTRA_CA_CERTS and HTTP_PROXY env variables work as expected in 0.24.x
thank you :)
@aureq thanks for the detailed report! Can you also confirm whether you're able to reproduce with a simple fetch
call outside of Corepack? If the bug is in the fetch
implementation, we can write a bug report to Undici issue tracker so it can be fixed upstream.
@aduh95 I'm trying to, but having some typing issues... Could you provide some guidance or help please?
First, I reused most of the code that's present in CorePack to save some time (the code I mentioned in my previous comment).
From the code below, vscode complains with the following error: Object literal may only specify known properties, and 'dispatcher' does not exist in type 'RequestInit'.
when trying to set displatch: agent
on L28 which seems to indicate the property isn't present. I'm not sure what I'm missing here.
But also, I'm a little puzzled as to why dispatcher
of type Dispatcher
would accept agent of type
ProxyAgent` according to my code below, originally taken from https://github.com/nodejs/corepack/blob/v0.25.2/sources/httpUtils.ts#L17.
Any suggestions welcome.
async function getProxyAgent(input: string | URL) {
const {getProxyForUrl} = await import(`proxy-from-env`);
// @ts-expect-error - The internal implementation is compatible with a WHATWG URL instance
const proxy = getProxyForUrl(input);
if (!proxy) return undefined;
// Doing a deep import here since undici isn't tree-shakeable
const {default: ProxyAgent} = (await import(
// @ts-expect-error No types for this specific file
`undici/lib/proxy-agent.js`
)) as { default: typeof import('undici').ProxyAgent };
return new ProxyAgent(proxy);
}
async function myFetch(input: string | URL, init?: RequestInit) {
const agent = await getProxyAgent(input);
console.error(agent);
let response;
try {
response = await globalThis.fetch(input, {
...init,
dispatcher: agent, // <-- error
});
} catch (error) {
console.error(error);
throw new Error(`Error when performing the request to ${input}`);
}
}
fetch("https://checkip.amazonaws.com");
{
"name": "gh-417",
"main": "index.ts",
"engines": {
"node": "^18.17.1 || >=20.10.0"
},
"devDependencies": {
"@types/node": "^20.10.0",
"@types/proxy-from-env": "^1",
"proxy-from-env": "^1.1.0",
"undici": "^6.6.1",
"typescript": "^5.0.4",
"ts-node": "^10.0.0"
}
}
But also, I'm a little puzzled as to why
dispatcher
of typeDispatcher
would acceptagent of type
ProxyAgent` according to my code below
As documented in https://undici.nodejs.org/#/docs/api/ProxyAgent, ProxyAgent
extends Dispatcher
.
From the code below, vscode complains with the following error:
Object literal may only specify known properties, and 'dispatcher' does not exist in type 'RequestInit'.
when trying to setdisplatch: agent
on L28 which seems to indicate the property isn't present. I'm not sure what I'm missing here.
That's probably because dispatcher
is a Undici-specific option. You can see Undici reads the dispatcher
property from the option bag either in https://github.com/nodejs/undici/blob/8dea74494cada360ae9d88a23d55b6296b9016ba/lib/web/fetch/request.js#L82 or in https://github.com/nodejs/undici/blob/8dea74494cada360ae9d88a23d55b6296b9016ba/lib/web/fetch/request.js#L107.
I think it's fine to ignore that error, but if you prefer you can also use setGlobalDispatcher(proxyAgent)
, but I'd expect you'd get the exact same result.
You could also try to reproduce with JS code (instead of TS), to reduce the scope further.
Does the above code reproduce the error? Could you also try to reproduce with URLs Corepack uses, e.g. https://repo.yarnpkg.com/4.0.0/packages/yarnpkg-cli/bin/yarn.js or https://registry.yarnpkg.com/yarn/latest.
@aduh95 Thanks for the information.
It's probably worth mentioning I'm not a JS/TS developer.
With that said, I've simplified my example and the code shared below works as expected. The connection is made to my proxy and the content appears to be retrieved correctly.
So, in my example, undici
appears to be working just fine. The corepack code is a little too complex for me to grasp its nuances.
But FWIW, the issue may be on corepack's side.
import * as proxyFromEnv from "proxy-from-env"
import * as proxyAgent from "undici"
async function getProxyAgent(input: string) {
const proxy = proxyFromEnv.getProxyForUrl(input);
if (!proxy) return undefined;
return new proxyAgent.ProxyAgent(proxy);
}
async function myFetch(input: string, init?: RequestInit) {
const agent = await getProxyAgent(input);
const u = new URL(input);
let response;
try {
proxyAgent.fetch
response = await proxyAgent.fetch(u, {
dispatcher: agent,
});
} catch (error) {
console.error(error);
throw new Error(`Error when performing the request to ${input}`);
}
console.log(response);
}
const res = myFetch("https://checkip.amazonaws.com");
Hi,
I have the same issue and had a look and have a bit more information
Node 18.19.1 Corepack 0.25.2
The error I got initially did not help much:
Internal Error: Error when performing the request to https://repo.yarnpkg.com/4.1.1/packages/yarnpkg-cli/bin/yarn.js; for troubleshooting help, see https://github.com/nodejs/corepack#troubleshooting
at fetch (/opt/containerbase/tools/corepack/0.25.2/18.19.1/node_modules/corepack/dist/lib/corepack.cjs:22886:11)
...
So I added a console.log
to get the original error:
original error TypeError: fetch failed
at Object.fetch (node:internal/deps/undici/undici:11731:11)
at async fetch (/opt/containerbase/tools/corepack/0.25.2/18.19.1/node_modules/corepack/dist/lib/corepack.cjs:22880:16)
at async fetchUrlStream (/opt/containerbase/tools/corepack/0.25.2/18.19.1/node_modules/corepack/dist/lib/corepack.cjs:22918:20)
...
cause: TypeError: this[kClient].connect is not a function
at Client.connect (/opt/containerbase/tools/corepack/0.25.2/18.19.1/node_modules/corepack/dist/lib/corepack.cjs:13364:66)
at socket (/opt/containerbase/tools/corepack/0.25.2/18.19.1/node_modules/corepack/dist/lib/corepack.cjs:12289:29)
at ...
}
So what we get is TypeError: this[kClient].connect is not a function
apparently, from this line : https://github.com/nodejs/undici/blob/v6.6.2/lib/proxy-agent.js#L93
This error is internal to undici (the bundled library) but is triggered when calling globalThis.fetch
One thing I noticed is that Corepack bundles Undici 6.6.2 but Node.JS bundles version 5.28.3 : https://github.com/nodejs/node/releases/tag/v18.19.1
But from what I found next this seems to have no implication
{
"devDependencies": {
"proxy-from-env": "^1.1.0",
"undici": "6.6.2"
}
}
import { getProxyForUrl } from "proxy-from-env";
import ProxyAgent from "undici/lib/proxy-agent.js";
//import { fetch } from "undici/lib/fetch/index.js";
//import { ProxyAgent, fetch } from "undici";
async function getProxyAgent(input) {
const proxy = getProxyForUrl(input);
if (!proxy) return undefined;
return new ProxyAgent(proxy);
}
async function myFetch(input, init) {
const agent = await getProxyAgent(input);
try {
const response = await globalThis.fetch(input, {
dispatcher: agent,
});
console.log(response.status);
} catch (error) {
console.error(error);
throw new Error(`Error when performing the request to ${input}`);
}
}
myFetch("https://repo.yarnpkg.com/4.1.1/packages/yarnpkg-cli/bin/yarn.js");
Using this with a proxy will yield the error we saw above:
TypeError: fetch failed
at Object.fetch (node:internal/deps/undici/undici:11731:11)
at async myFetch (file:///home/sgoetz/workspace/sq/repro/fetch.mjs:18:20) {
cause: TypeError: this[kClient].connect is not a function
But by changing the following import, it works OK :
- import ProxyAgent from "undici/lib/proxy-agent.js";
+ import { ProxyAgent } from "undici";
My understanding is that some other file imported by undici has side effects and would get the library to work fine.
This really needs to be fixed ... we do have the exact same error ...
@boindil As suggested, if this is blocking you, you have 2 options.
0.24.x
as suggested by meAnd if this is a pressing issue, then feel free to submit a pull request for the team to review.
I have the same issue:
Internal Error: Error when performing the request to https://registry.npmjs.org/pnpm; for troubleshooting help, see https://github.com/nodejs/corepack#troubleshooting
at ClientRequest.<anonymous> (C:\Program Files\nodejs\node_modules\corepack\dist\lib\corepack.cjs:42195:14)
But my corepack version is 0.23.0, so downgrade to 24 is not possible, I have an even lower version.
I Upgraded a project from Node 16 to 20. Our build is made in a custom Docker image. I think that the bugged version of corepack is shipped by default in Node 20. I ended up upgradind corepack to 0.27.0
For reference:
RUN corepack enable && npm install -g corepack@0.27.0 && corepack install --global yarn@1.22.22
Works well behind our proxy.
Still encountering this issue. Everything works fine locally. It was also working fine in the cloud too. Now, I only encounter the issue when i'm deploying my api (docker image) onto GCP cloud run. i get this error :
My corepack version locally : 0.17.2 (installed using home brew)
Error: Error when performing the request to https://registry.npmjs.org/pnpm/latest; for troubleshooting help, see https://github.com/nodejs/corepack#troubleshooting .fetch ( /usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:22762 )
@devbydixon as stated above, you need to upgrade your corepack to 0.27.0
or above. If you're still having a problem, please open a new issue as this one is now closed.
Hi, I'm willing to use API Platform (https://api-platform.com/) and the Dockerfile of the pwa component requires corepack:
The base image is node:20-alpine.
I'm behind a corporate proxy, so I've added the following before using corepack:
We also have a "traffic interceptor" that watches the whole traffic, so I added its certificate (which is already injected in the image):
RUN set NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/etat.crt
However, I'm still running into issues and receive the error below upon executing
corepack prepare --activate pnpm@latest
:Anybody could help me figure out what I'm missing here?
Thanks in advance. Adrien.