Open piranna opened 4 years ago
I think it is not possible to use the Node.js EventTarget
implementation without introducing breaking changes and performance regressions.
ws.dispatchEvent()
every time we call ws.emit()
.'ping'
, 'pong'
, 'unexpected-response'
, and 'upgrade'
. Currently the listeners of these events do not receive an Event
even if they are added via ws.addEventListener()
The ws
implementation relies on the EventEmitter
interface.
If I could go back in time I would have never made the WebSocket
an EventTarget
. Anyway the feature was added before I started contributing to ws
and I think the reason was to have a browser compatibile interface.
Can't you use the EventEmitter
interface in your code? Make Client
inherits from EventTarget
and use ws.{on,once}()
instead of ws.addEventListener()
.
I think it is not possible to use the Node.js
EventTarget
implementation without introducing breaking changes and performance regressions.
Or maybe yes :-) NodeEventTarget extends from EventTarget
and implements the EventEmitter
API on top of it. I read in the development discussion that it was specifically designed for compatibility and migration issues, so if you are concerned about that, probably this would be the correct aproach to begin with.
Can't you use the
EventEmitter
interface in your code? MakeClient
inherits fromEventTarget
and usews.{on,once}()
instead ofws.addEventListener()
.
I didn't consider that, since by inertia I always use the W3C API for compatibility between browser and Node.js, but yes, it's something I can do since in this case it would be a Node.js only code :-)
Or maybe yes :-) NodeEventTarget extends from
EventTarget
and implements theEventEmitter
API on top of it. I read in the development discussion that it was specifically designed for compatibility and migration issues, so if you are concerned about that, probably this would be the correct aproach to begin with.
Ok, but NodeEventTarget
currently only implements a subset of the EventEmitter
API. Some important missing API are emitter.prependListener()
and emitter.listeners()
. It would still be a breaking change and I'm still worried about performance regressions. I didn't follow the EventTarget
implementation closely in Node.js and I'm not sure if it was about NodeEventTarget
but I remember some benchmarks where EventTarget
was an order of magnitude slower than the EventEmitter
.
One important missing API is
emitter.prependListener()
andemitter.listeners()
. It would still be a breaking change
Is it actually being used? We could take a look for what NodeEventTarget
missing features are being used and identify if they can be fixed someway.
I'm still worried about performance regressions. I didn't follow the
EventTarget
implementation closely in Node.js and I'm not sure if it was aboutNodeEventTarget
but I remember some benchmarks whereEventTarget
was an order of magnitude slower than theEventEmitter
.
I somewhat remember something about that, but an order of magnitude is too much... How are ws
benchmarks being done? Maybe we can work in a separate branch and check for them...
Reading Node.js EventTarget
source code, it shows events list is implemented with a private SafeMap
, and that's what NodeEventTarget
uses for the listenerCount()
count method. We would need to change its implementation, but if both emitter.prependListener()
and emitter.listeners()
are needed and can be used an alternative, seems it's possible to implement them there... just not something immediate.
I prefer to keep making WebSocket
inherit from EventEmitter
for now.
Advantages:
WebSocket
class is always an EventEmitter
and the documentation is the same in all supported Node.js versions.Disadvantages:
ws
EventTarget
implementation and the Node.js EventTarget
implementation.That said, I think making WebSocket
inherit from NodeEventTarget
is something worth exploring.
Also there are some node packages with binary blobs that aren't ready, and don't yet work under Node v15. I use one of those closely tied to ws (wrtc)
After taking a more carefully look, probably this can be splitted in two tasks:
Event
classEventTarget
or NodeEventTarget
classFirst one is just a data container, so it's easy to replace, and that would fix the current error since it's doing a validation that the provided object inherit from the Event
class, so it would work and probably second one would not be needed (or would get a lower priority).
target
field of Event
class is read-only, and we are assigning it. I've reviewed Node.js source code and it's only set by private [kHybridDispatch]()
method, so it's not possible to only do first step, we'll need to go directly to use Node.js native EventTarget
or NodeEventTarget
class, not just only the native Event
class :-/
Would very much also want to see EventTarget being used also
I just had this dumb error too:
TypeError [ERR_INVALID_ARG_TYPE]: The "event" argument must be an instance of Event. Received an instance of OpenEvent
at new NodeError (node:internal/errors:363:5)
at EventTarget.dispatchEvent (node:internal/event_target:401:13)
[...]
Can you make a fork or something that uses proper EventTargets?
With the introduction of e173423c180dc1e4e6ee8938d9e4376a7a8b9757 subclassing EventTarget
would be even harder.
Also the pattern used in the issue description would not work even if WebSocket
was a subclass of EventTarget
.
const event = new Event('foo');
const target1 = new EventTarget();
const target2 = new EventTarget();
target1.addEventListener('foo', target2.dispatchEvent.bind(target2));
target2.addEventListener('foo', function (event) {
console.log(event);
});
target1.dispatchEvent(event);
node:internal/event_target:635
process.nextTick(() => { throw err; });
^
Error [ERR_EVENT_RECURSION]: The event "foo" is already being dispatched
at new NodeError (node:internal/errors:370:5)
at EventTarget.dispatchEvent (node:internal/event_target:401:13)
at EventTarget.[nodejs.internal.kHybridDispatch] (node:internal/event_target:455:20)
at EventTarget.dispatchEvent (node:internal/event_target:403:26)
at Object.<anonymous> (/home/luigi/event-target.js:11:9)
at Module._compile (node:internal/modules/cjs/loader:1095:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1124:10)
at Module.load (node:internal/modules/cjs/loader:975:32)
at Function.Module._load (node:internal/modules/cjs/loader:816:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:79:12) {
code: 'ERR_EVENT_RECURSION'
}
The event "foo" is already being dispatched
Does this means, once an event is dispatched, it can't be dispatched again in other object, and a new one needs to be created? It sorts of make sense, since target
should be different...
It can but you need to wait for the previous dispatch to complete.
We could do something like this
but it is very fragile and event.target
would not be a real EventTarget
.
Yeah I actually figured out that I can't redispatch events anyways, not even in the browser. So nvm for my part. But it would still be better if you used something that's instanceof EventTarget and then dispatched things that are instances of native Event because then .addEventListener would be native and have options like once.
The once
option is already supported. See https://github.com/websockets/ws/commit/2e5c01f5b550ae4171d127b0b707ebcec5925cc3.
-class Event { +class WsEvent extends Event {
👍🏻 to this :-)
I would be open to that if Symbol('kTarget')
was exposed.
The once option is already supported. See 2e5c01f.
Ah ok, but if I use .onmessage at the same time it doesn't seem to work IIRC
Ah ok, but if I use .onmessage at the same time it doesn't seem to work IIRC
It works if websocket.addEventListener()
is used after websocket.onmessage
. It doesn't if the order is reversed but
on<eventName>
attributes with the Node.js EventTarget
implementation. There is an helper function to do that but it is not public.Ah ok, but if I use .onmessage at the same time it doesn't seem to work IIRC
It works if
websocket.addEventListener()
is used afterwebsocket.onmessage
. It doesn't if the order is reversed but1. It is fixable. 2. It is an edge case. Why using both? 3. AFAIK it is not possible to create `on<eventName>` attributes with the Node.js `EventTarget` implementation. There is an [helper function](https://github.com/nodejs/node/blob/e2a6399be742f53103474d9fc1e56fadf7f90ccc/lib/internal/event_target.js#L651-L683) to do that but it is not public.
That's exactly my point. It's a big strange buggy mess because you tried making your own events. By using the browser/node provided events everything is compatible and you don't get weird issues. There is a, maybe limited, spec, but it's followed well in all aspects.
2. It is an edge case. Why using both?
Made a "mothersocket" that reconnects if anything goes wrong, "queues" messages and sends them so often until the server acknowledges them - and that still can be used like it's "one websocket". It runs both on browser and node and because the browser has .onmessage, etc I had to implement on-something support. I tried to just put it onto the websocket and debugged the hell out of it because I ran into the weird something added first other ones don't fire bug. But I ended up just adding one event listener and dispatching to the class itself and calling the .on functions from there.
It works if
websocket.addEventListener()
is used afterwebsocket.onmessage
. It doesn't if the order is reversed but
If I understood correctly the spec, using on...
remove all the previous handlers, both on...
and addEventListener
ones. That's why on...
ones are totally discouraged and unrecommended in W3C specs.
It's a big strange buggy mess because you tried making your own events.
I respect your opinion, but I didn't do it. I've just tried to improve it. From https://github.com/websockets/ws/issues/1818#issuecomment-731575731
If I could go back in time I would have never made the
WebSocket
anEventTarget
. Anyway the feature was added before I started contributing tows
and I think the reason was to have a browser compatible interface.
The primary, EventEmitter
based interface is way better for a server-side library in my opinion. I don't think it is possible to inherit from both EventEmitter
and EventTarget
as that means emitting the events twice as written above. So what is in place is a "best-effort" approach to have a browser compatible API. I wasn't there but I'm pretty sure that making it spec-compliant was not even considered. If you have better ideas please share them, I'm all ears.
Also, again in my opinion, the fact that there is no way to create on<eventName>
attributes when using the Node.js EventTarget
implementation proves that it is not quite there yet. We would need to use a copy of defineEventHandler()
or something like that.
If I understood correctly the spec, using
on...
remove all the previous handlers, bothon...
andaddEventListener
ones. That's whyon...
ones are totally discouraged and unrecommended in W3C specs.
If that is the case, then scratch and ~~1. It is fixable
2. It is an edge case. Why using both?
~~, as it works as intended.
If I understood correctly the spec, using on... remove all the previous handlers, both on... and addEventListener ones. That's why on... ones are totally discouraged and unrecommended in W3C specs.
What spec? :eyes:
This is how it works in the browser and how I'm used to it:
I respect your opinion, but I didn't do it. I've just tried to improve it. From #1818 (comment)
Ah shoot, np. Thanks.
Then node is the weird thing and you're just an unfortunate target of my frustration as I'm coming from the frontend.
Why the fuck do they have an EventEmitter when there's also EventTarget? Also they don't even have CustomEvent.
Also, again in my opinion, the fact that there is no way to create on
attributes when using the Node.js EventTarget implementation proves that it is not quite there yet. We would need to use a copy of defineEventHandler() or something like that.
I'd just not support it. But if you want, why not just make it a normal property and every time you dispatch an event you check if the property is a function and then call it? It's like 4 lines and as far as I understand it would cover the functionality
Why the fuck do they have an
EventEmitter
when there's alsoEventTarget
?
The EventEmitter
came first and is more efficient in terms of both performance and memory usage.
I'd just not support it. But if you want, why not just make it a normal property and every time you dispatch an event you check if the property is a function and then call it? It's like 4 lines and as far as I understand it would cover the functionality.
Yes, I guess that is a not spec-compliant way of doing it :)
The EventEmitter came first and is more efficient in terms of both performance and memory usage.
Weird. Why would they do that?
Yes, I guess that is a not spec-compliant way of doing it :)
Just using it as a property? Why is that not spec compliant? But yeah, coupled with
on... ones are totally discouraged and unrecommended in W3C specs.
I'd call for not supporting it.
FWIW https://github.com/websockets/ws/commit/0b21c03a6e69f8e37b2dfe55c4e753575fc09ac7 fixes the ws.on<eventName>
after websocket.addEventListener()
issue discussed above.
Damn, that's awesome. Props to you! One frustration less for users.
I just had a weird bug migrating to Nuxt's v3-bridge: Class extends value [object Module] is not a constructor or null
coming from class WebSocket extends EventEmitter while on Node 16. My current fix is to specify the following in nuxt.config.ts
:
alias: {
ws: 'ws/browser.js',
}
@stackdev37 it is already like that.
function foo() {}
function bar() {}
websocket.onmessage = foo;
websocket.onmessage = bar;
In the above example, the foo
handler is removed when the bar
handler is set.
This is still an issue. In fact, ws
should drop event-target.js
entirely. Both EventTarget
and Event
are provided by Node.js. Polyfilling them leads to all sort of unpredictable behaviors (see https://github.com/nodejs/undici/issues/2663) and fails Node's internal instanceof
check on dispatched events, causing errors where none should be.
MessageEvent
, as well as other events, must be extending the globalEvent
class. Right now, they extend the internal customError
class.
Can someone please share the status of this issue? If I find time to open a pull request, will somebody support me with the review and see this change merged?
I think it's time to reconsider this, all Node.js versions that don't support EventTarget
are largely unmaintained...
@piranna, replacing event-target.js
with Node.js globals should be rather straightforward, given ws
didn't implement any custom functionality. I'd pretty much like to see this change released to have good examples for WebSocket API interception in MSW. I think ws
is great!
Removing the EventEmitter
interface is not an option. Having both the EventEmitter
interface and the native EventTarget
interface is not easy. I would rather remove the EventTarget
interface from ws
.
@lpinca any particular reason the WebSocket
class extends EventEmitter
and not EventTarget
(apart from historical reasons, of course)?
It can benefit greatly from extending EventTarget
instead. In fact, the source code goes at length to adhere to EventTarget
(since that's the interface a WebSocket instance expects) due to extending EventEmitter
:
.on()
listeners to event target-compatible listeners. https://github.com/websockets/ws/blob/5e42cfdc5fa114659908eaad4d9ead7d5051d740/lib/event-target.js#L204EventTarget
prototype methods (addEventListener
and removeEventListener
) onto the WebSocket
class. https://github.com/websockets/ws/blob/5e42cfdc5fa114659908eaad4d9ead7d5051d740/lib/websocket.js#L614-L615this.emit()
instead of this.dispatchEvent()
, which requires step 1 in this list. This leads to additional logic to pass event-specific information, like the code
of the CloseEvent
instead of constructing that event instance directly. https://github.com/websockets/ws/blob/5e42cfdc5fa114659908eaad4d9ead7d5051d740/lib/websocket.js#L265There are likely more, I haven't looked much. This is a fundamental issue since WebSocket
extends EventTarget
by design, so it will always be easier to implement such a class extending EventTarget
.
I'd even argue the convenience of the .on()
/.off()
methods doesn't justify the cost but if one wishes to keep that public API, it's more straightforward to implement those custom methods through addEventListener()
/removeEventListener()
of the event target.
I propose the following:
event-target.js
module entirely. Everything it implements is available natively in Node.js now. WebSocket extends EventEmitter
to keep the change area to a minimum. addEventListener
and removeEventListener
custom mapping from event-target.js
to websocket module directly. This is the only place it's being used (to map .emit()
calls to proper corresponding dispatchEvent()
calls). Release this. Then, discuss anything else.
@lpinca, what do you think about this proposal?
any particular reason the
WebSocket
class extendsEventEmitter
and notEventTarget
(apart from historical reasons, of course)?
Performance, and the ability to pass multiple arguments to the listeners. This allows for example to do
websocket.on('message', function (buffer, isBinary) {
// Let the user decide what to do here.
});
without calling buffer.toString()
every time for text messages. This is useful for streaming and for cases like this
websocket.on('message', function (buffer, isBinary) {
websocket.send(buffer, { binary: isBinary });
});
- Move the
addEventListener
andremoveEventListener
custom mapping fromevent-target.js
to websocket module directly. This is the only place it's being used (to map.emit()
calls to proper correspondingdispatchEvent()
calls).
What is the difference? We can't extend the native Event
class to create the MessageEvent
, OpenEvent
, and CloseEvent
, and even if we could, there would still be incompatibilities, see https://github.com/websockets/ws/issues/1818#issuecomment-880689446.
without calling buffer.toString() every time for text messages
Isn't the transferred data either text or buffer-like? I know this is a naive suggestion but when does typeof data === 'string' ? data : data.toString()
falls short?
We can't extend the native Event class to create the MessageEvent, OpenEvent, and CloseEvent, and even if we could, there would still be incompatibilities, see
I set the target
on the WebSocket events using this bindEvent
function:
export function bindEvent<E extends Event, T>(
target: T,
event: E
): EventWithTarget<E, T> {
Object.defineProperty(event, 'target', {
enumerable: true,
writable: false,
value: target,
})
return event as EventWithTarget<E, T>
}
I have the luxury of not caring about the Node internal that much but for ws
you can set the necessary symbols here to represent the target. I need to look at how Node represents it and whether that's an internal symbol we have no effect on. Then it would indeed be problematic.
I know this is a naive suggestion but when does
typeof data === 'string' ? data : data.toString()
falls short?
That is how it worked and was changed to not call data.toString()
every time. It is unnecessary overhead sometimes. See https://github.com/websockets/ws/commit/e173423c180dc1e4e6ee8938d9e4376a7a8b9757.
I need to look at how Node represents it and whether that's an internal symbol we have no effect on. Then it would indeed be problematic.
It is indeed an internal Symbol
. See https://github.com/nodejs/node/blob/f3fcad280477061ab99803875069757ae117a485/lib/internal/event_target.js#L63 and https://github.com/nodejs/node/blob/f3fcad280477061ab99803875069757ae117a485/lib/internal/event_target.js#L184-L188.
We could override the getters in the subclass but that would prevent event.target
from returning the correct target if the event is dispatched again via eventTarget.dispatchEvent()
. At least an error is thrown now.
I am not 100% sure if my observation is the same, but we started using native EventEmitter.addListener(
It appears part of the events are sent using the EventEmitter, part of them use some other implementation and require on(event, listener) type of bindings.
Would it be possible to send also pong events through EventEmitter API? I noticed 'pong' is not part of native WebSocket events (see https://developer.mozilla.org/en-US/docs/Web/API/WebSocket#events) so I could not use it as an argument there. Regardless, it would be a nice addition to us.
All events are emitted via emitter.emit()
.
Sorry, then it must have be me misreading the typings. On the second look, it indeed seems that 'pong' is listed as part of addListener calls here (https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/ws/index.d.ts).
I was almost certain this would be the case, as our websocket code simply did not work with addListener('pong', ...) but I needed to use on('pong', ...). Perhaps the problem is somewhere else. I will see if I can isolate an easily reproducible sample.
Attached my SocketHandler class for the sake of completeness. It acts as glue code between graphql-ws and fastify. Never mind fastify.WebSocket typing - it is re-exported WebSocket.
class SocketHandler<E extends Record<PropertyKey, unknown> = Record<PropertyKey, never>> {
static isProd = process.env.NODE_ENV === 'production'
static eventNames = ['error', 'ping', 'pong', 'close', 'message'] as const
pingInterval: NodeJS.Timeout | null
emittedErrorHandled: boolean = false
pongWait: ReturnType<typeof setTimeout> | null = null
pongListener: ((...args: any) => void) | null = null
// Runtime bound callbacks
messageCallback: ((data: string) => Promise<void>) | null = null
closed: ((code?: number | undefined, reason?: string | undefined) => Promise<void>) | null = null
constructor(private readonly socket: fastifyWebsocket.WebSocket, private readonly keepAlive: number = 12_000) {
// Bind event handlers. Never mind the type errors - socket is EventEmitter and uses the handleEvent method
for (const event of SocketHandler.eventNames) {
socket.addEventListener(event as any, this as any)
}
// For an unknown reason, we must respond pong this way. Pong simply won't get emitted via
// SocketHandler.handleEvent
this.pongListener = this.handlePong.bind(this)
socket.on('pong', this.pongListener as (...args: any) => void)
this.pingInterval =
keepAlive > 0 && isFinite(keepAlive)
? setInterval(() => {
this.handlePing()
}, keepAlive)
: null
}
async send(data): Promise<void> {
if (this.socket.readyState !== this.socket.OPEN) {
return
}
return new Promise((resolve, reject) => {
this.socket.send(data, (err) => {
err ? reject(err) : resolve()
})
})
}
open(server: Server<Extra & Partial<E>>, request: FastifyRequest) {
this.closed = server.opened(
{
protocol: this.socket.protocol,
send: async (data: string) => this.send(data),
close: async (code, reason) => {
this.close(code, reason)
},
onMessage: (callback) => {
this.messageCallback = callback
}
},
// eslint-disable-next-line @typescript-eslint/consistent-type-assertions
{ socket: this.socket, request } as Extra & Partial<E>
)
}
close(code, reason) {
this.socket.close(code, reason)
}
handleError(err: Error, checkEmitErrors = true) {
if (checkEmitErrors && this.emittedErrorHandled) {
return
}
this.emittedErrorHandled = true
console.error('Internal error emitted on a WebSocket socket. Please check your implementation.', err)
this.socket.close(
CloseCode.InternalServerError,
SocketHandler.isProd
? 'Internal server error'
: SocketHandler.limitCloseReason(err instanceof Error ? err.message : String(err), 'Internal server error')
)
}
async handleMessage(data: WebSocket.Data) {
try {
await this.messageCallback?.(String(data))
} catch (err) {
this.handleError(err, false)
}
}
handlePing() {
// Ping pong on open sockets only
if (this.socket.readyState === this.socket.OPEN) {
if (this.pongWait) {
clearTimeout(this.pongWait)
}
// Terminate the connection after pong wait has passed because the client is idle
this.pongWait = setTimeout(() => {
this.handleTerminate()
}, this.keepAlive)
this.socket.ping()
}
}
handlePong() {
if (this.pongWait) {
clearTimeout(this.pongWait)
this.pongWait = null
}
// Note: WS sends pong automatically, so no need to do it manually
}
handleClose(code, reason) {
if (this.pongWait) {
clearTimeout(this.pongWait)
}
if (this.pingInterval) {
clearInterval(this.pingInterval)
}
if (
!SocketHandler.isProd &&
code === CloseCode.SubprotocolNotAcceptable &&
this.socket.protocol === DEPRECATED_GRAPHQL_WS_PROTOCOL
) {
console.warn(
`Client provided the unsupported and deprecated subprotocol "${this.socket.protocol}" used by subscriptions-transport-ws.` +
'Please see https://www.apollographql.com/docs/apollo-server/data/subscriptions/#switching-from-subscriptions-transport-ws.'
)
}
// Remove all listeners
this.socket.off('pong', this.pongListener as (...args: any) => void)
for (const event of SocketHandler.eventNames) {
this.socket.removeEventListener(event as any, this as any)
}
// eslint-disable-next-line @typescript-eslint/no-floating-promises
this.closed?.(code, String(reason))
}
handleTerminate() {
this.socket.terminate()
}
handleEvent(event: Event) {
switch (event.type) {
case 'error': {
this.handleError((event as ErrorEvent).error)
return
}
case 'message': {
// eslint-disable-next-line @typescript-eslint/no-floating-promises
this.handleMessage((event as MessageEvent).data)
return
}
case 'ping': {
this.handlePing()
return
}
case 'pong': {
this.handlePong()
return
}
case 'close': {
const { code, reason } = event as CloseEvent
this.handleClose(code, reason)
}
}
}
/**
* Forked from: https://github.com/enisdenjo/graphql-ws/blob/c030ed1d5f7e8a552dffbfd46712caf7dfe91a54/src/utils.ts
*/
static limitCloseReason(reason: string, whenTooLong: string) {
return reason.length < 124 ? reason : whenTooLong
}
}
Description
Node.js 15 already provides a native implementation of
EventTarget
, so there's no need to use our own implementation. In fact, using both of them at the same time leads to errors.In my use case, I've created a
Client
class that extends from Node.js nativeEventTarget
class, that internally it's using anws
instance (and doing some other project specific things), and setting as listeners methods from thisClient
class, with the idea of propagate these errors to the user:Problem is, since
ws
is using its own implementation of bothEventTarget
andEvent
classes, when these events gets propagated to the native one, I get the next error:This is due because Node.js native
EventTarget
class is expecting a nativeEvent
class instance, instead of the one provided byws
. According to Node.js docs it should be accepting any object with atype
field, but for some reason is not accepting it.Reproducible in:
Steps to reproduce:
EventTarget
class in its prototype chainws
instance and call toaddEventListener
setting one function that propagate the event to the nativeEventTarget
Expected result:
ws
should check for actual support of bothEvent
andEventTarget
classes in the native platform (in this case, Node.js 15) and use them. In case they are not available, then use its own implementation as a polyfill.Actual result:
ws
is using always its own implementation ofEvent
andEventTarget
classes, since there was none before, so now it conflicts with the new Node.js native available ones.