Support mocking WebSocket APIs

Doesntmeananything commented 4 years ago

Is it possible to use msw to mock server-sent events (SSE) and WebSocket (WS) connections?

My use case is to have a somewhat full client-side mocking story (including REST, SSE, and WS functionality), and since msw is such a joy to use when mocking out REST APIs, I was wondering if it makes sense to use it to mock more specialised server interactions.

Have you thought about this? I admit that I haven't looked much into whether it's possible just to use custom request handlers to add this functionality, emulating SSE and WS behaviour in some way. I wanted to know if you already had something in mind regarding this question. Thanks!

kettanaito commented 4 years ago

Hey, @Doesntmeananything, thanks for bringing this topic up. I'd love to bring SSE and WS support to the users of MSW. I admit I haven't researched the topic yet, but would use this thread for this.

Technically, it comes down to the ability of Service Worker to intercept those requests/events. If the spec supports it, there shouldn't be much changes needed on the MSW side.

Here's some useful resources:

https://github.com/w3c/ServiceWorker/issues/885, EventSource interception in Service Worker.
https://github.com/w3c/ServiceWorker/issues/947, a discussion whether Service Worker should access WebSocket events.

Could you please try to set up a proof of concept, if those events can be intercepted in the worker's fetch event?

You're right about the custom request handler, we can use it to log all intercepted requests:

setupWorker(
  {
    predicate(req) {
      console.log(req)
      // log captured requests, but bypass them
      return false
    },
    resolver: () => null
  }
)

If we confirm it working, I'd be glad to discuss the API for a dedicated request handler for WebSocket/SSE. I can read on their specification meanwhile.

Doesntmeananything commented 4 years ago

Sounds like a plan! At a cursory glance, it does indeed seem quite doable. Let me PoC this, and I'll get back to you with my results as soon as I can.

Doesntmeananything commented 4 years ago

Hi, @kettanaito! I've set up a (very quick and dirty) repository to test these interactions over at https://github.com/Doesntmeananything/msw-sse-ws.

My initial findings are the following:

I am able to intercept SSE connections, however this occurs only when a stream ends. For example, if a stream consists of 3 messages, logging happens only after all 3 messages have been received, including the final end event.
I am not able to intercept WS connections. I tried to make sure that the WS connection is established from the client only after the mock service worker had been initialised, but it didn't seem to help. I want to investigate this further.

I'm a bit concerned about WS events, although I hope that with some additional work it'd possible to intercept them.

kettanaito commented 4 years ago

@Doesntmeananything, thank you for the investigation! I'm excited to hear that SSE can be intercepted! Wonder if there's anything we can do it intercept events as they go.

I'm currently working on a NodeJS support, but can switch to this issue to help you once I'm done. I'm always open to questions or discussions, so please don't hesitate to raise those here.

Also, if you don't mind, we could then move your proof of concept repo under "msw" to serve as an example how to work with SSE/WS. That'd be awesome.

kettanaito commented 4 years ago

I'm trying to get my head around the SSE example. It seems MSW should intercept the hi from client event, so then it can mock the server response to it. I can see the once the WS connection is established, all the messages are inspectable live in DevTools. However, the webSocket.send("hi from client") is not intercepted by the Service Worker. I'm reading through https://github.com/w3c/ServiceWorker/issues/947, trying to figure out if it's technically possible to access WS communication in a service worker.

API-wise, I think there should be at least two types of request handlers: event-based handler, and persistent handler (pulsing back messages to the client, like you have in your example using AsyncIterator).

kettanaito commented 4 years ago

One of the most useful pieces of code I've found in the w3c discussion (https://github.com/w3c/ServiceWorker/issues/947#issuecomment-410816076) was that the Service Worker file can establish a WebSocket connection. It appears that the WS events are not subjected to be intercepted in the fetch event, but one can establish a socket connection an (?) intercept events that way.

If it comes down to the manual WS connection, I'd suggest to do that on the client's side, not in the worker file. There's no technical reason to move this logic to worker, at least as of how I understand such implementation now.

Doesntmeananything commented 4 years ago

Thanks very much for taking the time to look further into this!

Since I've hit the wall in regards to intercepting WS connections, your suggestions come in handy. Will definitely look into this.

To be clear, are you saying that mocking WS connections falls strictly outside of MSW concerns? My investigations lead me to believe this, and I would certainly not want to push for something that doesn't make sense neither on technical nor on conceptual level.

kettanaito commented 4 years ago

Not necessarily. What I was trying to say is that a WebSocket event is not intercepted by the fetch event in a Service Worker. That's a per-spec behavior. However, I've mentioned an example above, that creates a WS connection within the worker file, which I suppose allows to intervene the communication in some way. I haven't tried that approach out, whether it's actually possible to mock the response of an event.

kettanaito commented 4 years ago

I've received a suggestion to look at mock-socket. We may get some inspiration from how it's implemented, and see if a similar approach can be done in MSW.

kettanaito commented 3 years ago

Update: I've started with the WebSocket support and will keep you updated as I progress. For those interested I will post some technical insights into what that support means, what technical challenges I've faced, and what API to expect as the result.

Session 1: It's all about sockets

No service for the worker

Unfortunately, WebSocket events cannot be intercepted in the fetch event of the Service Worker. That is an intentional limitation and there's no way to circumvent it. This means a few things:

WebSocket events won't be visible in the "Network" tab;
WebSocket support does not require the worker and can live outside of setupWorker context.
WebSocket events interception should be done by patching the underlying logic (i.e. a WebSocket class).

Goodbye, handlers!

WebSocket operates with events, not requests, making the concept of request handler in this context redundant. Instead, you should be able to receive and send messages from ws anywhere in your app, including your mock definition.

import { rest, ws, setupWorker } from 'msw'

// Create an interception "server" at the given URL.
const todos = ws.link('wss://api.github.com/todos')

setupWorker(
  rest.put('/todo', (req, res, ctx) => {
    const nextTodos = prevTodos.concat(req.body)

    // Send the data to all WebSocket clients,
    // for example from within a request handler.
    todos.send(nextTodos)
    return res(ctx.json(nextTodos))
  })
)

// Or as a part of arbitrary logic.
setInterval(() => todos.send(Math.random()), 5000)

URL that got away

When constructing a WebSocket instance you must provide a URL that points to a WebSocket server. Providing a URL to a non-existing server yields and exception that you cannot circumvent by patching WebSocket class, as the URL validation lives in its constructor.

I've ended up re-implementing a WebSocket class, effectively making a polyfill out of it. That way it can have its custom constructor that would tolerate non-existing URLs if there is a ws.link interception server declared for that URL.

Persisting WebSocket clients

The entire idea of WebSocket is to sync data between multiple clients in real time. When you dispatch a mocked ws.send event to send some data to all clients, you need to let all the clients know they should receive the data (trigger their message event listener). However, there's no way to know and persist a list of WebSocket clients on runtime, since each page has its own runtime.

Usually a solution to this kind of problems is to lift the state up and maintain a record of clients in the upper context shared with all the clients (pages). However, in JavaScript there isn't that may ways to share and persist data between clients. In case of WebSocket clients one needs to store references to WebSocket instances—basically, object references. I've considered:

localStorage/sessionStorage. Good for sharing textual data, not that suitable for storing objects with prototypes. Objects flushed here effectively lose their references, making them completely different objects.
Web Worker/Service Worker. A great way to have a detached process in a browser that can control and communicate with multiple pages. However, as stated before, worker API cannot intercept WebSocket events, and using it only for the sake of storing some records is utterly redundant (I don't wish to ask people to copy any more worker scripts). Utilizing an existing mock worker may be an option, however, I'm afraid it would put too many logic into it, increasing its maintenance, and violating its purpose of containing a bare minimum logic that you seldom need to update.
BroadcastChannel. Turns out the API that allows workers to communicate with clients exists standalone and it's awesome. You can create a broadcast channel as a part of page's runtime, and as long as another page on the same host creates a channel with the same name they can send data between them.

const channel = new BroadcastChannel('ws-support')

// One client sends a data.
channel.send('some-data')

// All clients can react to it.
channel.addEventListener('message', (event) => {
  event.data // "some-data"
})

I find BroadcastChannel a great choice to mimic the real time data synchronization functionality of WebSocket. I've chosen it to spawn a single channel between all clients and notify them when they should trigger their message event listeners.

Sun-2 commented 3 years ago

@kettanaito

URL that got away When constructing a WebSocket instance you must provide a URL that points to a WebSocket server. Providing a URL to a non-existing server yields and exception that you cannot circumvent by patching WebSocket class, as the URL validation lives in its constructor. I've ended up re-implementing a WebSocket class, effectively making a polyfill out of it. That way it can have its custom constructor that would tolerate non-existing URLs if there is a ws.link interception server declared for that URL.

You could use an ES6 Proxy. It can mess with ctors.

Link.

BlackGlory commented 3 years ago

SSE and WebSockets are different issues. If msw supports response streams (such as ReadableStream), it can support SSE.

kettanaito commented 3 years ago

@BlackGlory, MSW should support ReadableStream as the mocked response body. Would you have some time to try to which extent that's true, and whether SSE would be supported now?

BlackGlory commented 3 years ago

@kettanaito Although ctx.body supports ReadableStream, it does not seem to work.

export const worker = setupWorker(
  rest.get('/sse', (req, res, ctx) => {
    return res(
      ctx.status(200)
    , ctx.set('Content-Type', 'text/event-stream')
    , ctx.body(sse(function* () {
        yield 'message1'
        yield 'message2'
      }))
    )
  })
)

function sse(gfn) {
  let iter
  return new ReadableStream({
    start() {
      iter = gfn()
    }
  , pull(controller) {
      controller.enqueue(`data: ${iter.next().value}\n\n`)
    }
  })
}

Browser:

[MSW] Request handler function for "GET http://localhost:8080/sse" has thrown the following exception:

DOMException: Failed to execute 'postMessage' on 'MessagePort': ReadableStream object could not be cloned.
(see more detailed error stack trace in the mocked response body)

Node.js:

TypeError: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received an instance of ReadableStreamTypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received an instance of ReadableStream

      at ClientRequestOverride.<anonymous> (node_modules/node-request-interceptor/src/interceptors/ClientRequest/ClientRequestOverride.ts:216:34)
      at step (node_modules/node-request-interceptor/lib/interceptors/ClientRequest/ClientRequestOverride.js:33:23)
      at Object.next (node_modules/node-request-interceptor/lib/interceptors/ClientRequest/ClientRequestOverride.js:14:53)
      at fulfilled (node_modules/node-request-interceptor/lib/interceptors/ClientRequest/ClientRequestOverride.js:5:58)

kettanaito commented 3 years ago

Hey, @BlackGlory. Could you please report this as a separate issue? Thanks.

MarkLyck commented 3 years ago

any update on web socket support? 🙏

It would be amazing to get this built into msw.

I'm currently using MSW for all normal requests, but I have yet to find any good solution for mocking my 1 web socket service which is causing me grief.

kettanaito commented 3 years ago

Hey, @MarkLyck. The support is in progress, but we're prioritizing some more pressing bug fixes at the moment.

You can follow the progress here #396. The in-browser support looks good, needs polishing. The NodeJS support is on the roadmap. I have some concerns over NodeJS usage, as WebSocket is not a part of the standard library (to my knowledge). I'd rather not build any client-specific logic around web sockets.

kettanaito commented 3 years ago

Update

The proof of concept for WebSocket support in a browser is functional. As mentioned earlier, it can be achieved by patching the native WebSocket class that's a standard browser API.

The Node.js support remains unknown, as there's no native WebSocket class to operate with. We'd have to reverse-engineer how most common WebSocket libraries function in Node.js (i.e. socket.io) and see what kind of requests they create if any. The end goal is to provide functionality that's library-agnostic and works no matter what WebSocket client you've decided to use. So far there's no confirmation it'll be the case in Node.js.

When to expect this?

While WebSocket support is a great feature to have, we prioritize other features that are objectively more useful to the majority of our users. There's no deadline on WebSocket support, as it's approached in a free time when there are no other pressing issues or features to address.

How to make this happen?

We'd be thankful for the investigation on how WebSocket libraries function in Node.js, specifically:

What kind of requests do they make? Are those instances of ClientRequest or something else?
How can a WebSocket event be intercepted in Node.js? At which level a patch to WebSocket should be introduced?

jbcpollak commented 3 years ago

Hi @kettanaito thanks for the update - do you think you can release support sse in browsers or do you need to wait till there is parity with nodejs? If you'd release browser support, what do you need to get that done?

kettanaito commented 3 years ago

Hey, @jbcpollak. We will release this feature only when it's fully functional in both browser and Node.js.

As I've mentioned above, the initial phase of the browser support is completed, and you can install it in your projects:

$ npm install mswjs/msw#pull/396/head
$ yarn add mswjs/msw#396/head

Although looking at the build status on that pull request, it doesn't pass, so you're unlikely to use it this way, sorry. You can build it locally from that pull request, I verify that npm run build command is okay.

We'd be thankful if you tried it out and provided us with your feedback!

jbcpollak commented 3 years ago

Hi, I gave it a go using the npm link method and added some comments about problems on the PR. Only after getting it working I realized the PR is for web sockets, and I need Server Sent Events support, so I'm stuck for now.

kettanaito commented 2 years ago

SSE in Service Worker

I've found a great article that goes into detail on how to intercept and respond to SSE using Service Worker:

Server-Sent Events Gateway by Service Worker

This looks like a good starting point to explore to enable SSE support.

Note that WebSocket events still can't be intercepted. At least I couldn't achieve that during the development. If that's the case, and SSE and WS should be treated differently, then I'd suggest splitting this issue into two phases: SSE and WS support respectively.

dbehmoaras commented 2 years ago

@kettanaito this would be very helpful indeed. It seems that the current behavior handles a websocket interception (via socket.io) by throwing a warning. Until the new library is released, do you have a suggestion for how to handle and/or disable this warning?

lukas-mertens commented 2 years ago

@kettanaito Thank you for experimenting on this. I haven't used mswjs yet and don't know if this possibly covered by some other feature, but I wanted to add my usecase of websocket-support anyways, as I believe this issue has to be resolved to achieve what I am trying to do:

According to the GraphQL specification, section 6.2.3 graphql supports subscriptions. This allows the server to push updates to query-results to the client. It would be very nice, if it was possible to mock this behavior as well. If this is already possible and I just missed something in the docs, just ignore what I said.

kettanaito commented 2 years ago

Hey, @lukas-mertens. Yes, you're right in thinking that GraphQL subscription mocking will benefit greatly from WebSocket mocking support (hence #285). This pull request focuses on mocking a WebSocket server, which currently functions as expected in the browser. Once we release this feature (if ever), we'll follow up with the GraphQL subscriptions support, letting you push updates from the server.

If you or your team can use a unified way to mock WebSocket and subscriptions across platforms, consider supporting the effort of our team. Any contribution brings features like this closer to reality. Thanks.

mariothedev commented 2 years ago

I was able to successfully mock a SSE!!!

Here's how:

Your mocked endpoint must have the following properties:

export const sseMock = rest.get(
`http://localhost:3000/stream`,
(_, res, ctx) => {
    console.log('mock SSE server hit')
    return res(
        ctx.status(200),
        ctx.set('Connection', 'keep-alive'),
        ctx.set('Content-Type', 'text/event-stream'),
        ctx.body(`data: SUCCESS\n\n`)
    )
}
)

Comment out the following from the mockServiceWorker.js file:

// Bypass server-sent events.
// if (accept.includes('text/event-stream')) {
//     return
// }

And that's it! 👍

After that just insert your mocked response under setupWorker:

export const worker = setupWorker( someOtherMockResponse, sseMock )

SerkanSipahi commented 2 years ago

@mariothedev does your suggestion any negative impact to the usage of msw?

mariothedev commented 2 years ago

@SerkanSipahi - not really. I've been using like this for quite a while now and no issues as of yet.

dbehmoaras commented 2 years ago

I was able to successfully mock a SSE!!!

This is a really neat solution!

kettanaito commented 2 years ago

Update: I'm back on the WebSocket support. Currently, I've implemented the native window.WebSocket interceptor in @mswjs/interceptors and working/testing the interception of SocketIO in particular.

Meanwhile, I'd like to toss around some thoughts on the ws API in MSW. I'd very much like everybody following this feature to contribute their thoughts and ideas around that API.

The `ws` API

First of all, just like @mswjs/data models its API around Prisma because it's a superb querying client, I think the ws API should model itself around SocketIO. SocketIO is de-facto a standard for WebSocket implementations and is widely used.

Usage example

Here's an extensive usage example of a hypothetical ws API in MSW:

import { ws } from 'msw'

export const handlers = [
  ws.link('wss://api.mychat.com', (io) => {
    // Handles when a new socket connects.
    io.on('connection', (socket) => {
      // Handles the "message" events from this socket.
      socket.on('message', handler)

      // Handles a custom event from this socket.
      socket.on('greet', (username) => {
        // Sends data to this socket.
        socket.send('hello to you too', username)

        // Send data to other sockets but this one.
        socket.broadcast.emit('hello everybody else!')
        socket.broadcast.send('sends to everybody')
      })
    })

    // Sends data to all connected sockets.
    io.send('hello')
    io.emit('greet', 'John')

    // Handles when a socket disconnects.
    io.on('disconnect', (socket) => {
      console.log('socket %s disconnected', socket.id)
    })
  })
]

General statements

Mocking a WebSocket API would require you to specify the WebSocket server endpoint via ws.link(). Intercepting any events to any WebSocket servers is not what you want to do anyway, as that pushes the endpoint distinguishing logic on your shoulders.

Capabilities

Here's what I think the ws API should enable our users:

Listen when a new client connects: io.on('connection').
Listen when a client disconnects: io.on('disconnect').
Send a message to all clients: io.send().
Send a message to the currently connected client: socket.send().
(SocketIO only) Emit a custom event to all clients: io.emit().
(SocketIO only) Send a custom event to the currently connected client: socket.emit().
(SocketIO only) Send/emit data to all clients except the currently connected one: socket.broadcast.send()/socket.broadcast.emit().

As SocketIO introduces custom features (such as socket.emit and socket.broadcast), I feel that MSW should enable those features in the mocks. We have to be cautious not to end up re-implementing SocketIO internally though, so I'd expect certain SocketIO features not to be supported at least in the initial ws release.

Question to you

What do you think about the overall ws API look and feel above?
Are there any scenarios that this API couldn't handle? Please share them below.
Is there anything missing from this API to mock your real-world WebSocket applications?
Should you be able to spy on the events sent from the actual running server? To have an introspection of sorts.

james-reed-toshi commented 2 years ago

As a consumer of this api, I think I'd vastly prefer (if possible) named functions rather than on events - so you'd do io.onConnection(socket => ..., io.onDisconnect and socket.onMessage(handler) and so on, with the option to fall back to the foo.on('bar', handler... case for custom events.

Have you thought about how to deal with unhandled events? Maybe both io and socket should have the same options for unhandled requests as the server.listen onUnhandledRequest param via io.onUnhandledEvent and socket.onUnhandledEvent, ie the option to send a little handler in but also to continue and hand back control to MSW if wanted?

kettanaito commented 2 years ago

@james-reed-toshi, named functions would work for known event names. In the window.WebSocket API there are only as many fixed event names: open, message, error, close. You cannot subscribe nor emit anything else. Frameworks like SocketIO may allow for arbitrary event names like socket.emit('greet', 'John') which are coerced into the "message" event as an implementation detail. Handling arbitrary event names with the on[Name] pattern you suggest becomes problematic and loses type-safety.

Are there any particular reason you'd prefer such API? Perhaps you can give some examples? Since neither WebSocket nor socket.io utilize such API format, I wonder where does the source of preference come from in your case?

Have you thought about how to deal with unhandled events?

That's a good question. "Unhandled" in the MSW dictionary refers to requests which do not have a corresponding request handler. If we translate this to WebSocket events, I'd say that unhandled event is the one which doesn't have a mock event listener defined.

Yet the only event that a client can send to a server is the "message" event. So, technically, if you have a "message" event listener, all events will be treated as handled:

socket.on('message', (event) => {})

// Actual code
socket.send('hello') // Dispatches "MessageEvent"

With SocketIO you can send arbitrary events but they get coerced to "message" event under the hood anyway. While I like SocketIO API, we need to make sure we support plain WebSocket API as the first thing. Given the duplex nature of data transfer, you may not necessarily define a "message" listener, as you may be sending events from the server straight away:

io.send('hello to all clients!')

This is a valid usage but if your code sends a "message" event, I'd expect the unhandled callback to get triggered in some shape or form. I'd keep the strategies the same:

warn: unhandled event produces a warning but goes to the actual server.
error: unhandled event produces an error and does not go to the actual server.

What do you think about this?

james-reed-toshi commented 2 years ago

Are there any particular reason you'd prefer such API? Perhaps you can give some examples? Since neither WebSocket nor socket.io utilize such API format, I wonder where does the source of preference come from in your case?

Ease of use for API consumers is all - I don't need to have an in depth knowledge of how sockets/SocketIO works in order to use MSW. Can you not use the same types as sockets/SocketIO and extend them?

If we translate this to WebSocket events, I'd say that unhandled event is the one which doesn't have a mock event listener defined. Yet the only event that a client can send to a server is the "message" event.

Hm, I see. Fair enough then, I'd agree with your opinion on the strategies. What about if someone hasn't handled the io.on events?

kettanaito commented 2 years ago

`WebSocket` and `socket.io` compatibility

In one of its transports, SocketIO utilities native WebSocket and introduces an abstraction on top of it to support custom events among many things. I think MSW should care not what WebSocket framework you're using, and provide an agnostic API to mock events. That doesn't mean that we cannot model that API to resemble a widely used solution like SocketIO, lowering the threshold for developers to adopt WebSocket interception in MSW.

From the functional perspective, I lean towards using the native WebSocket behavior in the ws API. As various frameworks may build abstractions on top of it, the native behavior seems a great common ground to lift off those abstractions and allow developers to mock WebSocket communication regardless of what libraries/frameworks they use to achieve it.

To illustrate, here's an example of real code implemented with SocketIO:

io.on('connection', (socket) => {
  socket.on('message', (text) => {})
  socket.on('greet', (text) => {})
})

You can see that SocketIO provides the text string in the listener to both "message" and arbitrary "greet" event. The native WebSocket API, however, provides you with a MessageEvent instance in the "message" listener:

const ws = new WebSocket('wss://example.com')

ws.addEventListener('message', (event) => {
  event.data // analogous to "text" from SocketIO
})

For the mocks, these are two drastically different data types passed along the same event handling lines. Since window.WebSocket is a standard API, I lean towards using the platform and exposing the MessageEvent in the mocks for both WebSocket and SocketIO (and any other framework for that matter, as all will utilize the message event):

// src/mocks/ws.js

ws.link('wss://example.com', (io) => {
  io.on('connection', (socket) => {
    socket.on('message', event => event.data)
    socket.on('greet', event => event.data)
  })
})

It's an incongruence for those who use SocketIO but I find it worth the cost. Your handlers won't break if you switch from SocketIO to native WebSocket or to any other library, which is one of the values of using MSW.

james-reed-toshi commented 2 years ago

Since window.WebSocket is a standard API, I lean towards using the platform and exposing the MessageEvent[...] It's an incongruence for those who use SocketIO but I find it worth the cost.

Agree :) One of the reasons I like msw so much is that I dont have to care whether I'm using fetch or axios or swr or whatever, it all just works.

kettanaito commented 2 years ago

What about if someone hasn't handled the io.on events?

io in the ws handler's context represents a server. MSW cannot argue about whether a server event is handled or not because for that it'd actually had to occur on the real server to see whether it was handled or not. MSW simply doesn't have access to the internals of an actual server.

A server may handle incoming events with side-effects that do not face the connected client. For example, it may send an analytics call upon the "greet" event and don't communicate back to the client. In cases like this, MSW has no means to find out whether you have that event handled or not.

It's important to keep in mind that the ws API represents a mock WebSocket server with which your actual client will communicate. The same way as rest and graphql represent servers. Asserting server events effectively means testing the server, which is not the purpose of this library.

Although, to be fair, I'd like to have an API to emulate client-sent events so you could test your WebSocket server implementation. It'd be a separate API nonetheless, as it'd flip the domain upside down, giving you a mock socket instance to emulate connection/messaging/etc. Maybe it's something we will consider in the future but it doesn't change all I've said above.

james-reed-toshi commented 2 years ago

io in the ws handler's context represents a server.

Ah I see, ok.

Testing a server via a mock client sounds cool!

That all makes sense then to me - interested in seeing this come live!

kettanaito commented 2 years ago

The scope of WebSocket support has grown tremendously since the last time I've looked into it. It's not really a matter of spying on the window.WebSocket. I've learned much since then, especially about frameworks like socket.io that introduce their own abstraction. We will use custom extensions internally for those abstractions to work. I already have a few passing tests for socket.io interception and it behaves as expected.

I've also learn that there are multiple ways to implement web sockets in your application. The SocketIO itself illustrates them rather descriptively with their transports:

Via window.WebSocket.
Via HTTP polling.
Via XMLHttpRequest.

And all transports must be accounted for if we wish for MSW to intercept WebSocket events reliably regardless of implementation detail. The WebSocket class is too low level to rely upon in this case.

I've split the work into chunks, and now I'm working on the transports that utilize window.WebSocket. While it's going well, I'm concerned about the other two options: polling and XHR. It looks like we'd have to mix interceptors in some way, so the WebSocket interceptor could also intercept regular HTTP calls and check if those are related to the resource in any way. On top of that, we'd much want to intercept those HTTP calls via Service Worker in the browser, the logic for which lives in the msw package directly and is not consumable anywhere outside of MSW.

I haven't assessed yet what changes would be required to support all transports. I've anticipated that we'd have to rework the way we establish interceptors but the concept that a single interceptor can accept requests from various origins (http/xhr/service worker) has never been explored. Until now, all interceptors have been self-contained.

Jhony0311 commented 2 years ago

Looks great, on my end I feel the API looks good and really resembles how I would use a sockets server. I do have a use case that I wanted to ask for. Let's say I have a message that needs to be sent after X amount of time after a REST endpoint is hit, I handle the REST endpoint on my MSW mocks, but how would I generate a WS response for my client based on that REST endpoint being hit first. Seems to me that the current API is not capable of doing that with ease.

kettanaito commented 2 years ago

@Jhony0311, that's a great use case!

Originally, I planned for the ws API to be standalone, leaving outside of your handlers:

import { ws } from 'msw'

export const service = ws.link('wss://my-service.com')

This way you can reference it in any other handler you like, for example, a REST handler:

import { service } from './ws.mocks'

rest.post('/user', (req, res, ctx) => {
  // Emit event from the WebSocket server upon hitting this REST API route.
  service.send(`welcome a new user ${req.body.username}`)
})

My concern with this is that the WebSocket handlers suddenly become their own special thing, while I'd much prefer them to be alongside the rest of your handlers, keeping the entire network behavior collocated.

At the same time, WebSocket handlers fall out of the response resolution loop anyway: they won't be signaled incoming requests because they are event-based. So, from the user experience, I feel it'd be great to colocate all handlers together. Internally, WebSocket handlers are special because they represent a mock server of an entirely different protocol.

I'll try to summarize my points below.

Why I think `ws` should be among other handlers

Collocation.
Predictability.
Response resolution logic becomes in control of when to apply the handler. This is tricky but I'd like for it to behave this way.

Why I think `ws` shouldn't be among other handlers

It's a de-facto special thing and falls out of the default response resolution loop. ws handlers won't have predicate, neither will they have resolver. It's just not a RequestHandler and rightfully so.
It must establish some sort of instance to reference so that you could send WebSocket events from anywhere (i.e. other handlers, including other WebSocket handlers).

joaopcartier commented 2 years ago

I was able to successfully mock a SSE!!!

Here's how:

Your mocked endpoint must have the following properties:
export const sseMock = rest.get(
    `http://localhost:3000/stream`,
    (_, res, ctx) => {
        console.log('mock SSE server hit')
        return res(
            ctx.status(200),
            ctx.set('Connection', 'keep-alive'),
            ctx.set('Content-Type', 'text/event-stream'),
            ctx.body(`data: SUCCESS\n\n`)
        )
    }
)
Comment out the following from the mockServiceWorker.js file:
// Bypass server-sent events.
    // if (accept.includes('text/event-stream')) {
    //     return
    // }
And that's it! 👍

After that just insert your mocked response under setupWorker:

export const worker = setupWorker( someOtherMockResponse, sseMock )

How would you consume this? I am trying with EventSource and it is throwing me a 404 code. If I try to fetch the same endpoint it works

jbcpollak commented 1 year ago

How would you consume this? I am trying with EventSource and it is throwing me a 404 code. If I try to fetch the same endpoint it works

the suggestion by @mariothedev works for me, for a single message. After that, the connection drops and I get an 'error' event on the client side. Still hoping to be able to stream events or at least keep the connection open for a while.

I don't do much other than new EventSource(url)

jbcpollak commented 1 year ago

note if you modify mockServiceWorker.js to comment out the following code as @mariothedev suggested:

// Bypass server-sent events.
    // if (accept.includes('text/event-stream')) {
    //     return
    // }

make sure to remove the msw.workerDirectory setup in package.json, or it will overwrite your change above every time you install dependencies.

In other words, remove this config from package.json if you have it:

"msw": {
    "workerDirectory": "public"
}

this will allow you to manually choose when to update the service worker.

asherccohen commented 1 year ago

Hey guys, thanks for the hard work into supporting more than just rest and graphql.

May I ask if there's something we can start using (even if experimental) to explore the current API and use in our projects?

We'd love to help with testing this feature!

kettanaito commented 1 year ago

Hey, folks. I wanted to give a quick update on the state of the WebSocket support in MSW.

Short, the browser side of this feature is implemented in https://github.com/mswjs/interceptors/pull/236. The Node side remains as there's no WebSocket API in Node, and we need to find a suitable way to implement request interception provisioned by third-party libraries without shipping any library-specific code. This likely means diving into net and other low-level modules and figuring out how much we can utilize those.

I'm not planning on working on WebSocket support in the foreseeable future. My focus goes to the Fetch API support (#1436) because it's something absolutely everybody using the library will benefit from. Help is welcome to make WebSocket happen!

SerkanSipahi commented 1 year ago

@kettanaito it is possible to merge the browser part into main branch so this can be consumed by NPM?

kettanaito commented 1 year ago

@SerkanSipahi, the issue is that only browser-side interception is not a finite feature. I wouldn't merge things that don't make sense on their own into main. With MSW (and with the Interceptors), we're establishing across-environment support, so it's a requirement to guarantee compatibility with both browsers and at least a limited set of Node.js versions. The latter is missing in the current implementation of the WebSocket support so it doesn't belong in the main.

The best I can do is release the browser implementation under a beta flag for Interceptors but it'd still require a bump in MSW, including the release of the ws() API which may not be fully ready at this point.

Honestly, this makes little sense to me, and if you wish to give this a try, consider using GitHub pull requests as dependencies in your project. Add the @mswjs/interceptors resolution on top of MSW and you should get the things installed correctly. But before that, check on the status of the #396 API because I don't remember if it's in working condition at the moment.

Contributing to the ws API is also something I'd much appreciate, as that's a wrapper around the Interceptors and doesn't concern itself much with how to intercept requests, rather how to consume this new interceptor from MSW.

wade-gooch-kr commented 1 year ago

Hello @kettanaito! My team and I have been following this thread, and we saw that the change has been merged in so that msw will support EventSource. I was hoping to inquire when an official release might be happening that includes this update? Thank you in advance!

kettanaito commented 1 year ago

Hey, @wade-gooch-kr. Excited to hear that. I'm in the middle of some test rewrites, I will publish that branch when I have a minute. Meanwhile, you can specify that PR as your dependency in package.json and should be able to try it out.

Stackustack commented 10 months ago

Hey, whats the status on this? It seems like some part of the job was done in this merged PR 🤔 Should this be marked closed / done or is this still WIP?

mswjs / msw