search

smithy-lang / smithy-typescript

Smithy code generators for TypeScript. (in development)
Apache License 2.0
211 stars 78 forks source link

`MaxListenersExceededWarning` showing on the latest version when using an AbortController #1318

Closed nicholasgriffintn closed 1 day ago

nicholasgriffintn commented 3 weeks ago

First up, sorry for an ignorance as to if this is an issue or not, I'm not super well versed in this library or AbortController overall.

We (bbc/sqs-consumer) recently received an issue around a MaxListenersExceededWarning showing after upgrading to the latest version of AWS-sdk (https://github.com/bbc/sqs-consumer/issues/500).

It looks like this stems from the changes made to AbortController here:

https://github.com/smithy-lang/smithy-typescript/releases/tag/%40smithy/node-http-handler%403.1.0

On investigation, it looks like this library is adding event listeners here (and elsewhere): https://github.com/smithy-lang/smithy-typescript/pull/1308/files#diff-98216b22e6b39a1cd6f2ab013304d4964152b554c4ad8fee4182a961b42285d8R192

And then not removing them.

This seems to be a bit different to implementations I've seen elsewhere, where the listener is added and then removed, such as in undici: https://github.com/nodejs/undici/blob/3fd1fe1a98fe49dcf7629f5d974246dae2692883/lib/core/util.js#L519

What I'm wondering is if this is the cause of the issue, and if it is, could it be fixed here?

We've had a suggestion to change how we use the AbortController to initialise at the start of every call, rather than globally, that needs some investigation as to if that's the fix or not, but I'm also wondering if that is just moving to having an issue elsewhere, down the road.

What's the expectation for libraries using this library here?

kuhe commented 1 week ago

The changes from https://github.com/smithy-lang/smithy-typescript/pull/1332 were released in

https://www.npmjs.com/package/@smithy/node-http-handler/v/3.1.2 https://www.npmjs.com/package/@smithy/fetch-http-handler/v/3.2.1

For AWS SDK and other client consumers, you can remove your lockfile for a fresh install that should bring these in transitively.

nicholasgriffintn commented 1 week ago

Thanks.

I've just ran our Docker setup for 5 minutes (https://github.com/bbc/sqs-consumer-starter/tree/main/examples/docker), and got no errors.

So as long as we don't get any further reports, it seems that this PR resolved the issue.

paulbrimicombe commented 1 week ago

I've checked the new version with various different scenarios and not seen any issues.