OpenFn / kit

The bits & pieces that make OpenFn work. (diagrammer, cli, compiler, runtime, runtime manager, logger, etc.)
10 stars 9 forks source link

worker: if a module fails to load, that's a crash #700

Open josephjclark opened 4 months ago

josephjclark commented 4 months ago

Run log from his Taylorness:

Memory limit: 500mb
Timeout: 10s
Executing a07dd84f-2af5-4010-b181-d949fc858bd1
Starting step Transform data to FHIR standard
[linker] loading module @openfn/language-http
[linker] Loading module @openfn/language-http from /tmp/openfn/worker/repo/node_modules/@openfn/language-http_6.2.2/dist/index.cjs
Resolved adaptor @openfn/language-http to version 6.2.2
Removed configuration from final state
Failed step Transform data to FHIR standard after 4ms
AdaptorError: Cannot find module '@openfn/language-common'
Require stack:
- /private/tmp/openfn/worker/repo/node_modules/@openfn/language-http_6.2.2/dist/index.cjs
{"message":"Cannot find module '@openfn/language-common'\nRequire stack:\n- /private/tmp/openfn/worker/repo/node_modules/@openfn/language-http_6.2.2/dist/index.cjs","name":"AdaptorError"}
Check state.errors.cf797a3b-6933-48fc-97b9-8e037aed16e5 for details.
Edge function (state) {
return Boolean(!state?.errors?.["cf797a3b-6933-48fc-97b9-8e037aed16e5"] ?? true)
} returned false; c08f815d-ee09-402a-a574-482cc25a411a will NOT be executed
Versions:
    ▸ node.js                  18.17.1
    ▸ worker                   1.1.8
    ▸ @openfn/language-http    6.2.2
Run complete with status: fail
AdaptorError: Cannot find module '@openfn/language-common'
Require stack:
- /private/tmp/openfn/worker/repo/node_modules/@openfn/language-http_6.2.2/dist/index.cjs

Somehow common failed to be loaded here. We can probably reproduce that by corrupting the local repo (making it think common is installed from the top package json but actually having an empty folder).

But what's particularly important is that this is a FAIL with an adaptor error. But actually this is a system error and should be a CRASH

Not sure how we'll trap this - either we can trap the error during module loading and before any execution, or we have to regex the error string (less keen on this)