Open Cherry opened 11 months ago
It appears that this ends up running into some fundamental TypeScript issues. I am not an expert here, but what I understand is happening is that TypeScript imports your tsconfig.json
types
field first, then overrides it with anything imported from a dependency. If you have even a single dependency that overrides a global type that you’re using from @cloudflare/workers-types
, then too bad, that type is overridden.
This is a big and hard to solve problem, because TypeScript automatically emits /// <reference types="node" />
any time a dependency uses a Node global. My pnpm why @types/node
is showing tens of projects with these globals; it’s infeasible for me to personally patch or PR each of these dependencies to fix their projects, even if you buy that those projects have made some mistake in the first place.
Unfortunately, TypeScript can’t fix this problem without a much bigger rearchitecture (microsoft/TypeScript#18588; the strict-env
proposal has also stalled). If there’s any silver lining to this, I guess it’s that it would also break other WinterCG projects that use TypeScript like this (IIRC the only one would be @edge-runtime/types
, and only if they have a similar type incompatibility; Deno and others do their global type checking differently or just use @types/node
).
A slightly better workaround than the above would be to add the following to a global.d.ts
in your project:
declare module "@types/node" {
declare const value: unknown;
export default value;
}
This will limit the damage of neutralising @types/node
to just the projects you’re using Workers in, and allow you to keep @types/node
unpinned for actual node projects (such as scripts or other packages in a monorepo).
TypeScript 5.0 includes a new compilerOptions
directive, customConditions
. This means the compiler can now resolve package.json exports
conditions to locate types depending on the runtime/conditions that are present. In short, this means that packages which emit a /// <reference types="node" />
can scope this to just be imported by the node
condition, or to remove it for the workerd
condition. This is the TypeScript equivalent of a bundler’s conditions
directive (ex. ESBuild), and you use it in much the same way.
However, as we’ve found with the workerd
condition in the ecosystem, this requires every package to update to split their exports. I’m not convinced this is feasible. An easier solution would be for @types/node
to add a workerd
condition with a compatible export, like so:
"exports": {
"workerd": {
"node": {
"types": "./index.workerd.d.ts",
},
"types": "./noop.d.ts"
},
"types": "./index.d.ts"
}
By default, this would just be a no-op, which trusts that you’re importing @cloudflare/workers-types
and neutralises @types/node
. If you’re running Workers by default, this would most accurately reflect the types you’d have access to. But the beauty of this solution is that if you’re running in Node compatibility mode, you could specify "customConditions": ["workerd", "node"]
and we’d serve up a version with all of the incompatible types removed. This would most accurately reflect Node compat mode, where we want a lot of the proper node types, but Cloudflare-augmented globals would still be present. We could even omit anything that’s not supported in Node compat mode—but I assume Cloudflare would be responsible for maintaining this file.
I’ll hit up the DefinitelyTyped team and see what they think—an opinion from Cloudflare would be greatly appreciated here also :)
First, add this to your tsconfig.json
:
{
"compilerOptions": {
"customConditions": ["workerd", "worker", "browser"],
}
}
Second, patch @types/node/package.json
using your package manager to add:
"exports": {
"workerd": {
"types": "./noop.d.ts"
},
"types": "./index.d.ts"
}
And add @types/node/noop.d.ts
to your patch—it just needs to be empty. Here’s a patchfile for pnpm:
diff --git a/index.workerd.d.ts b/index.workerd.d.ts
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/package.json b/package.json
index e9e15c90021711122a01a324f07f3675e13e1e14..b04e27142c8ef196d8bb3bb53661794fde3569e7 100644
--- a/package.json
+++ b/package.json
@@ -213,6 +213,12 @@
],
"main": "",
"types": "index.d.ts",
+ "exports": {
+ "workerd": {
+ "types": "./index.workerd.d.ts"
+ },
+ "types": "./index.d.ts"
+ },
"typesVersions": {
"<=4.8": {
"*": [
This is fantastic insight, thank you so much for the detailed response!
I'm unfortunately in a project where I do value having the types for node:crypto
, so noop-ing everything is less than ideal, but this seems like a better workaround for folks who don't need any Node types for the nodejs_compat
flag.
I'm unfortunately in a project where I do value having the types for node:crypto, so noop-ing everything is less than ideal, but this seems like a better workaround for folks who don't need any Node types for the nodejs_compat flag.
If you use "types": ["@cloudflare/workers-types", "node/crypto"]
alongside the above workaround, you might be able to get this to work?
I'm unfortunately in a project where I do value having the types for node:crypto, so noop-ing everything is less than ideal, but this seems like a better workaround for folks who don't need any Node types for the nodejs_compat flag.
If you use
"types": ["@cloudflare/workers-types", "node/crypto"]
alongside the above workaround, you might be able to get this to work?
Yes that might be a possible workaround, with loading @types/node/crypto
, @types/node/buffer
etc. explicitly.
Reading Andrew’s comments, it looks like the way forward is for Cloudflare (or the community) to maintain their own fork of @types/node
with the above changes, and encouraging users to override their dependencies. I think it would be enough for that package to just /// <reference types="node" />
for index.node.d.ts
, and have a blank file at index.workerd.d.ts
, in which case it could just have a peer dependency on @types/node
and never have to update its resolutions? But I’m not super well-versed with package magic.
Now that @types/node
v18 has been updated to include fetch types, I suspect this issue will show up a lot more frequently. So it’s worth also adding that there are a few ways to handle having multiple versions of @types/node
in your project that need patching:
pnpm -r upgrade @types/node
(or equivalent) to upgrade to the latest satisfiable versions. In my monorepo this got me down to a patch for v18 and v20@types/node
to the version you’re using and just patch it onceI have gone with (3), which looks like (package.json
):
"pnpm": {
"patchedDependencies": {
"@types/node@20.8.9": "patches/@types__node@20.8.9.patch"
},
"overrides": {
"@types/node": "^20.0.0"
}
},
For me, this means updating the patch whenever @types/node
updates, but usually that just involves renaming the file cause their package.json
isn’t going to change a tonne.
I am trying to get a similar patch, is this the most up to date version?
Should work, are you having any problems with it? I also gave instructions on how to re-create the patch for any @types/node
version if it’s not working…
Thanks @huw for getting back so quickly
Here is what I have so far outlined in this PR: https://github.com/Helicone/helicone/pull/1140/files
However, it is still not finding the Node types
src/lib/secureCache.ts:51:28 - error TS2591: Cannot find name 'Buffer'. Do you need to install type definitions for node? Try `npm i --save-dev @types/node` and then add 'node' to the types field in your tsconfig.
51 const encryptedContent = Buffer.from(encrypted.content, "hex");
~~~~~~
Could you please explain what you’re trying to achieve in a broader sense?
The error you have posted occurs because secureCache.ts
is trying to use Buffer
, which is a Node-only API. This API will not run in Cloudflare Workers unless you enable node_compat
or nodejs_compat
. You will either need to enable one of those flags or remove the dependency on Buffer to get your code to run.
This issue is about something different, which is that sometimes when importing third-party libraries, they will also import types from Node into the global typespace, which can cause code to type-check correctly when it shouldn’t (for example, if you wrote code with Buffer
but you wanted to run it in Workers, this code might type-check fine even if it can’t actually run!). You can use the patch above to improve the correctness of your type-checking, but this will have no bearing on whether the libraries you depend on will actually need Buffer
in the first place.
It sounds like you might trying to do the opposite, which is to import the Node types (remember, you will also need to enable node_compat
or nodejs_compat
in your wrangler.toml
). You can do that by adding types: ["node"]
to your tsconfig.json#compilerOptions
.
If this doesn’t make sense or you’re having more problems, can I ask you to ask this in the Cloudflare Discord? I can’t be 100% sure without more context, but it doesn’t sound like your problem is relevant to this issue.
@huw Happy to chat more on discord but that link seems to be broken.
Can you friend me? #justin75 and we can pick up from there?
This link should work, but just so we're clear I don't have the time to help you there (I don't work for Cloudflare just FYI), I'm just redirecting you to the right place to ask for it :)
Thanks! No worries, I appreciate your time @huw thanks
For anyone else running into this issue here is what I am trying to solve....
We keep getting type conflicts within our edittors and when trying to build our project.
src/lib/providerCalls/call.ts:63:3 - error TS2741: Property 'webSocket' is missing in type 'import("/Users/justin/Documents/repos/promptzero/helicone/worker/node_modules/.pnpm/undici-types@5.26.5/node_modules/undici-types/fetch").Response' but required in type 'import("/Users/justin/Documents/repos/promptzero/helicone/worker/node_modules/.pnpm/@cloudflare+workers-types@4.20231121.0/node_modules/@cloudflare/workers-types/index").Response'.
63 return response;
~~~~~~
node_modules/.pnpm/@cloudflare+workers-types@4.20231121.0/node_modules/@cloudflare/workers-types/index.ts:1073:12
1073 readonly webSocket: WebSocket | null;
~~~~~~~~~
'webSocket' is declared here.
The issue is because there are two version of Response/Request and fetch that were added to node types 20^.
I tried to use @huw's solution here however, I was still getting a ton of different compile and type checking issues.
using pnpm to override the node types version to 18 works for now but this is not a long term solution
"pnpm": {
"overrides": {
"@types/node": "18.15.3"
}
},
or for yarn...
"resolutions": {
"@types/node": "18.15.3"
},
Thank you all for your help!
@huw thanks for https://github.com/cloudflare/workerd/issues/1298#issuecomment-1756830290, that was very helpful.
Just a note for anyone else who runs into this: some libraries (like Next.js) try to find a package's package.json
, and this will fail with this patch. By default, package.json
is considered to be exported, but if there is an explicit exports
entry, it needs to be manually included there: https://github.com/nodejs/node/issues/33460 / https://github.com/uuidjs/uuid/issues/444
So our patch looks like this (as you can't mix ./
and prefixless exports):
diff --git a/noop.d.ts b/noop.d.ts
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/package.json b/package.json
index c902b012556ed4bca4b8038a0f4523ce3cb76287..07f730b9d31475f59e108de66e832122ee57c3c6 100644
--- a/package.json
+++ b/package.json
@@ -226,5 +226,12 @@
},
"typesPublisherContentHash": "e270b1ec6520966a2cb0578fa33386344f78258bc0bd109a3e4b315d052e2b62",
"typeScriptVersion": "4.5",
- "nonNpm": true
+ "nonNpm": true,
+ "exports": {
+ "./workerd": {
+ "./types": "./noop.d.ts"
+ },
+ "./types": "./index.d.ts",
+ "./package.json": "./package.json"
+ }
}
Hitting this while migrating my Workers from Service Workers to ESM (by force, not choice), and in turn having to migrate from Jest to Vitest. This is painful.
The docs already recommend pinning a specific Vitest version, do we also need to pin @types/node
?
Potentially could be fixed if we generate node types on the fly based on node compat flags (aka Types as a Service).
Hello! Just to say that using overwrite or resolutions in package.json
is not working for us, as we use a monorepo (nx
) and they have a bug that requires a nodejs version at least 20.12.2 :( (which conflicts with the v20.8.3 advised here)
So we are basically blocked currently 😅
Related nx issue: https://github.com/nrwl/nx/issues/22998
@Lp-Francois - I think you are saying that Nx requires Node.js 20.12.2 or above to work (at runtime). The workaround suggested is only about locking down the "types" of Node.js to 20.8.3. Are you saying that locking the types is somehow breaking your builds when using Nx?
Hey @petebacondarwin, it gives me No matching export in "../../../node_modules/@cloudflare/workers-types/index.ts" for import "Response"
when starting locally my worker using wrangler.
"resolutions": {
"@types/node": "20.8.3"
}
I tried using 20.8.1, 20.8.3, 20.9.0 for the types and 20.8.1, 20.9.0, 20.12.2 as node version (with a yarn install
in between). I thought the runtime version could be an issue :/
Also, as we have different projects with nx others than cloudflare workers, I am afraid than a global override in a monorepo is not the best option, as it could cause type issues 🤔
FYI, TypeScript 5.6 will pretty much require a newer version of @types/node
to pick up a number of iterator type updates (iterator methods / improved correctness); the workaround of pinning back to an old version is not going to be viable after that.
I struggled with this issue for a whole day and resolved it by using the --experimental-include-runtime
option added to the wrangler types
command in Wrangler v3.66.0 (the doc incorrectly says 3.66.1).
Don't forget to uninstall @cloudflare/workers-types
and add the path of the generated runtime types file to the compilerOptions.types
field in your project's tsconfig.json
file:
{
"compilerOptions": {
"types": ["./.wrangler/types/runtime.d.ts"]
}
}
I struggled with this issue for a whole day and resolved it by using the
--experimental-include-runtime
option added to thewrangler types
command in Wrangler v3.66.0 (the doc incorrectly says 3.66.1).
Sadly this is only really a solution if you're not using nodejs_compat
and any Node.js APIs. The --experimental-include-runtime
option doesn't include these types, so you still have to load @types/node
. Or, if something else you're loading includes these, the problem as described in the original issue still persists.
As per https://github.com/DefinitelyTyped/DefinitelyTyped/pull/66824 which was merged recently and released in
@types/node@20.8.4
, Node.js now defines its ownRequest
,Response
, etc. types viaundici-types
.While this is great in Node environments, it completely messes with the
Request
andResponse
(maybe others?) types defined by@cloudflare/workers-types
.Example code that used to work before:
Now has a type error with:
Expected 0 type arguments, but got 1.
.Or:
Now has a type error with `Property 'cf' does not exist on type 'Request'.
Or:
Now has a type error with:
Why load
@types/node
?This is a fair question, and you generally don't need to load
@types/node
at all, and in fact, I don't directly. But, many dependencies that people use, do in fact require@types/node
, including the likes of@types/pg
, which is loaded by@neondatabase/serverless
.It seems that once this newer version of
@types/node
is loaded into memory at all, it just overrides the globally defined Request/Response/fetch types from@cloudflare/workers-types
.Full reproduction
See the following github repo: https://github.com/Cherry/workers-node-types-issue-repro.
This showcases an example package that loads
@types/node
, and if you comment in/out the import you'll see the type errors now show up.Workaround
My current workaround for this is to pin
@types/node
via npm overrides, with this in mypackage.json
:It's a very naive and short-term workaround, but works for now.