Handle CBD Extract log and errors more gracefully

[pietercolpaert]

I have an LDES that errors due to a too many requests fetcher. We need to do better error handling both here and fix logging in the extract cbd shape library

Example

Maybe dereferencing http://marineregions.org/mrgid/30829?t=1704290215 (not all paths are found (<http://www.w3.org/ns/dcat#centroid>))
node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "Already dereferenced http://marineregions.org/mrgid/30829?t=1704290215 won't dereference again".] {
  code: 'ERR_UNHANDLED_REJECTION'
}

==> do we want for every HTTP request a log on the command line at all time?

[kkostov]

Just experienced this as well. There seem to be async functions without an attached catch handler.

Unfortunately a try/catch in a client application may not be able to suppress the error as the ldes-client performs IO related operations and so the runtime considers such rejections unhandled.

I also noticed that there are some writes to console.error which further complicates the matter of error handling as output to stderr usually means that the operation has failed (which may not be true if a consumer of the library is handling errors... say when making a dashboard).

⠹ building client + server bundles...TypeError: Cannot read properties of undefined (reading 'subject')
    at Client.init (/Users/konstantin/Developer/imec/ldes-registry/node_modules/ldes-client/dist/lib/client.js:185:55)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.start (/Users/konstantin/Developer/imec/ldes-registry/node_modules/ldes-client/dist/lib/client.js:237:17)
⠹ building client + server bundles...Ignoring SHACL shape without a declared sh:targetClass:  NamedNode {
  termType: 'NamedNode',
  value: 'https://private-api.gipod.vlaanderen.be/api/v1/ldes/mobility-hindrances/shape'
}
Nothing in flight, adding start url
⠹ building client + server bundles...Nothing in flight, adding start url
⠴ building client + server bundles...Nothing in flight, adding start url
⠇ building client + server bundles...Closing notifier
(node:41326) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGINT listeners added to [process]. Use emitter.setMaxListeners() to increase limit
⠋ building client + server bundles...Nothing in flight, adding start url
⠧ building client + server bundles...Nothing in flight, adding start url
⠹ building client + server bundles...Max retries
⠹ building client + server bundles...Ignoring SHACL shape without a declared sh:targetClass:  NamedNode { id: 'https://graph.irail.be/sncb/connections/feed/shape' }
Nothing in flight, adding start url
⠏ building client + server bundles...Nothing in flight, adding start url
⠼ building client + server bundles...node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "Already dereferenced http://marineregions.org/mrgid/2545?t=1704805889 won't dereference again".] {
  code: 'ERR_UNHANDLED_REJECTION'
}

Node.js v21.7.1

I also found this rather revealing https://github.com/nodejs/node/issues/43326

[kkostov]

Scratch that, there is a way to globally suppress unhandled promise rejections and it even allows to print the problematic lines we should tackle first:

process.on("unhandledRejection", (reason, promise) => {
  console.error("Unhandled Rejection at:", promise, "reason:", reason);
});

[kkostov]

Some of them appear to originate from comunica packages:

[pietercolpaert]

https://github.com/rdf-connect/ldes-client/pull/24 is working on fixing those!

[ajuvercr]

That process.on("unhandledRejection", ...) is naughty! I will remember, for a friend ofcourse