search

OpenFn / kit

The bits & pieces that make OpenFn work. (diagrammer, cli, compiler, runtime, runtime manager, logger, etc.)
9 stars 9 forks source link

Update node versions #617

Open josephjclark opened 6 months ago

josephjclark commented 6 months ago

Ideally we should just work across all vaguely modern node versions.

18.19 did something I think to break the loader vm arg, which causes all sorts of problems.

Akshanshkaushal commented 4 months ago

hi @josephjclark @christad92 To better understand the codebase, I want to solve this issue. Could you please assign me this?

josephjclark commented 4 months ago

Sorry @Akshanshkaushal this is one I want to take on myself

josephjclark commented 2 months ago

So the problem we have is that node 20 changes the way loaders work, which breaks our ava config. This will affect adaptors as well - see https://github.com/OpenFn/adaptors/issues/187

I think the issue technically is that a) loaders now run in worker threads (something to do with not corrupting the global environment), and b) If you want a worker thread itself to us a loader, basically you can't feed the option through. I think this is the active bug on nodejs: https://github.com/nodejs/node/issues/47747

The first problem is that the ts node loader has changed and doesn't work with ava. It works if I use a thing called tsimp, but it builds all the test files to disk. Not essential I suppose but it annoys me. Also the docs say it does everything virtually, so that's kinda confusing.

I suppose one solution here is to just wait this out and still on node 18 for a while. It doesn't affect the runtime, only the build-time.

The second problem is some changes about importing folders, which we can't do any more. I suppose we'll just have re-write the affected code, although that could be a big job

josephjclark commented 2 months ago

Running ava with workerThreads off (forcing it to use child processes) works fine, at the cost of using more system resources (and so being slower in CI).

What I'm confused at with this is that I don't get import errors for importing folders and stuff, which I do get when using other loaders. Is that just a config thing?