Closed brian-mann closed 4 years ago
Related to:
@brian-mann I would be very willing to help out with this issue. This is a big blocker for use when testing our app.
@ericadamski could you go into more detail about this? What exactly is a big blocker?
@brian-mann making network requests (using a fetch polyfill) and asserting on those requests is important to our integration testing. Currently we are just spying on the window.fetch
and doing some simple assertions on the params request. It would be ideal to provide the same functionality as XHR
s to completely omit the server and mock responses when needed, understanding that we could stub the fetch
as well, but this means stubbing each individual request with a tailored response.
You can do this right now using stubs. Albeit it's certainly not as nice as what we did for XHR stubbing, but it is possible and we have several examples of making this work. This is even in a new getting started video we're releasing later this week.
Yes we realize it can be done, albeit we would rather wait (or work toward) a more XHR style stubbing, it is just super nice.
Yeah it is. Okay just making sure we're on the same page.
The proposal here does a decent job outlining what needs to be done but it will be a significant amount of work.
Likely the next steps just need to be formalizing a plan for the API's themselves. There have been many requests for additional features and we should start there.
That sounds good.
Is it worth creating separate issues for some of the requested features for I know almost nothing about graphQL queries 😜
They mostly already are other issues. We need to factor in all of them and come up with a set of API's that unifies those different use cases and desired experiences.
Right OK, I see what you are saying.
@brian-mann side note, has it been thought of moving away from coffeescript and into es6?
Do you consider this feature in the next release? I reckon it would be highly valuable, for instance when
Yeah blacklisting specific requests is part of this feature roadmap
Nock https://github.com/node-nock/nock mocking syntax is really sweet, some examples
nock('http://www.example.com')
.get('/resource')
.reply(200, 'domain matched');
nock('http://myapp.iriscouch.com')
.post('/users', {
username: 'pgte',
email: 'pedro.teixeira@gmail.com'
})
.reply(201, {
ok: true,
id: '123ABC',
rev: '946B7D1C'
})
A lot more at https://github.com/node-nock/nock#specifying-hostname
Each part (like method, is an function .get, .post, etc
can take either exact match or a callback predicate function.
nock.get('http://myhost.com')
nock.get(url => { ... }) // return true to match
nock.query({foo: 'bar'})
nock.query(query => { ... }) // return true to match
Just wanted to mention that @bahmutov has started work on this. 🎉
I do like nock's fluent API, but one problem is that it would end up adding a lottttt more Cypress methods. I also feel some of their method signatures are odd, or not what I expect, or I constantly have to look them up to remember - what is it that I use again? The surface area is really large. But definitely good stuff to pull from!
It might be an easier sell to mimic what request
does -> since we built cy.request
around the same API style....
Some generic pairing notes not intended to be canon
Client (Non Proxied Request) Browser
- GET https://www.github.com
- DNS resolution (local computer)
- DNS resolve (24.4512.23.23)
- Server gets
- SSL Handshaking
- Receives bytes
- GET http://localhost:98765/
Request Headers
---------------
GET / HTTP/1.1
Host: localhost:98765
--------------------------------
Client (Proxied Request) Browser
- CONNECT github.com:443
- CONNECT localhost:8080
GET http://localhost:8080/foo/bar
Host: localhost
GET https://www.github.com/ HTTP/1.1
Host: github.com
-------------------------------
Cypress creates a http server: localhost:98765
Cypress creates a file server server: localhost: 98764
Cypress creates an SNI proxy server: localhost: 98763
--proxy-server=http://localhost:98765
cy
.route('GET', 'http://localhost:8081', {
onRequest: req =>
onResponse: res =>
}).as('getFoo')
...
...
...
.wait('@getFoo.request')
injecting hosts
1. captureTrafficHosts: "*" by default we do it on '*'
1. captureTrafficHosts: ['foo', 'bar'] by default we do it on '*'
1. know ahead of time what hosts to inject (generate a cert / have access to traffic)
2. when any request come in, capture a HAR
3. expose an API to the user to enable them to control traffic
4. when we encounter user rules, the internal traffic rules instance has to know about them
5. when network request match traffic rules, we have to apply those rules to the network requests
- stubbing the responses
- waiting on the responses (pass through)
- modifying the requests
- delaying the responses
6. if there are callbacks (onRequest / onResponse) we have to decode / buffer the responses (potentially) send data to the driver (websockets) after serializing into objects
- wait for the user to potentially respond (if we support promises)
- socket.pause() the responses so the browser hasn't completed the http request
7. complete the http request
8. notify the driver of a completed request
Found the fetch-mock which has very similar api to cy.route()
so it makes stubbing/mocking of window.fetch
much easier and readable.
@FredyC do you use fetch-mock
in your cypress tests?
@ifeanyi I do, but my case is really simplified to a login scenario only. Rest of a communication goes through the GraphQL which is much more simple to mock.
I did it basically like this.
// it would be nice to have some global onBeforeLoad
onBeforeLoad: win => {
cy.stub(win, 'fetch', fetchMock.sandbox()).as('fetch')
},
// then in any test I can use API of fetch-mock like that
cy.get('@fetch').get('*', { token: 'AAA', userId: 'BBB' })
afterEach(() => {
cy.get('@fetch'}.restore()
})
@FredyC I am doing something similar, I created a fetchMock module so ensure that I re-use the same sandbox. I was able to mock requests in my command file but I am having trouble mocking requests in individual tests. I'm gonna keeping hacking at it till I find a reliable pattern.
// fetchMock.js
module.exports = require('fetch-mock').sandbox();
It's your call, but "re-use the same sandbox" does not make any sense :) With that approach, you can easily stumble into a situation when one test is influenced by other because you forgot to clean up the sandbox. So yea, I think that onBeforeLoad
is still the best spot.
Besides, I think it's the main reason you are struggling with it. You need to stub window.fetch
before a code of the tested site runs.
Besides the missing support for fetch
, it would be nice to support xhr/fetch stubbing requests made in web workers. Unless I'm missing something it doesn't seem straightforward to wrap the workers XmlHttpRequest
implementation?
If you use Apollo and would like to be able to stub calls to your GraphQL layer in Cypress, here is how you can do it until this problem gets resolved:
+import unfetch from 'unfetch'
createHttpLink({
uri: '/graphql',
credentials: 'same-origin',
+ fetch: unfetch
})
And in the support file:
// Cypress doesn’t recognize `window.fetch` calls as XHR requests, which makes
// it impossible to stub them. We delete `fetch` from the window object so the
// `unfetch` polyfill (which uses proper `XMLHttpRequest`) kicks in.
Cypress.on('window:before:load', win => {
delete win.fetch
})
@HugoGiraudel You don't even need to use unfetch
in your HttpLink. As long as you make win.fetch
unavailable in window:before:load
, Apollo will revert to using XHR. I think Apollo uses isomorphic-fetch
, which is built on top of Github's fetch
polyfill.
In other words, you don't need to do the first part of your answer. You just need the delete win.fetch
part.
Hi. Apologies if this has been answered or covered elsewhere, but after extensive searching, I'm struggling to nail down the answer I'm looking for, including within this issue.
In short, we would like to mock a websocket that we use with our Angular 6 UI, using Cypress (which is great, btw!).
I have seen in a couple of places that using cy.stub is the way to go. But are there any examples of doing this for a websocket? Specifically for an Angular 2+ UI? The Cypress examples cover stubbing AngularJS code, but not Angular 2+ (as far as I can tell).
Or is this issue coming up with another solution?
Any pointers to get me going in the right direction will be really appreciated.
I realize this is still in progress, but I wanted to mention the use case of simulating a network delay for specific requests; I have a bug that only manifests when a specific request resolves before another specific request. I would like to be able to simulate that in a test for a regression test; not to serve canned responses but to use real responses, just delay one of them so that it "comes back" after another.
Thanks for the work on this, I'm excited for this feature!
@bahmutov is starting back on this work this week.
full-network-stubbing-687
is in https://github.com/cypress-io/full-network-proxyHi, Sorry for possibly duplicated question, but with this feature will it be possible to work (wait, check response body) with Websocket events? thank you
Do you have an ETA on this?
Hey guys,
is there any information, when this feature is going to be ready?
I have an issue when I use cy.server
and cy.request
as first operation in before()
or it()
blocks. There are sometimes test runs, where the request seems to be not to trigger. It runs into the requestTimeout but I can see the request in the developer tools. When I reload the spec, than it mostly runs without any problemens. But this is impossible for the CI server.
For me, it looks like the listeners listentens to late on my requests. Is this gona fixed with this issue too?
I see that this package has not been mentioned here. It has solved a lot of my team's problems: https://github.com/ijpiantanida/talkback
@Vages do I understand correctly, that talkback is replacement for my "original" / production server? Or is is possible to forward requests from cypress over talkback to my native apache?
Thanks for your reply
Talkback is a proxy which stores the last response for a given URL. Spin up one or more talkback-servers for the APIs you want to mock via cypress/plugins/
. When in the test environment, set your frontend to fetch/get data from these URLs instead of the regular URLs.
I'm using talkback also, working really good!
I mentioned this here https://github.com/cypress-io/cypress/issues/95#issuecomment-433679170 but i wonder why PollyJS wouldnt be a perfect fit, it can do what cypress wants and more and already offers adaptors and other means to hook in. I use it with cypress and its amazing. Would love to see it become part of cypress :). I think it was meant to be :P
Also my 2 cents, Think you should leave the stubbing in the client as this gives you the best flexibility and control, other wise syncing mock settings will be required and adds more complexity. Additionally stubbing in the client renders the fastest response, which is nice for testing purposes. Having stubs that return dynamic content based of your test suite might be tricky as you have to send the response to the server so it can respond to your app which is already in client ???
I feel your mocks should originate from your suite since whats where its executed from, and since this is in the client it feels clunky to push this config to the server and have it respond from there back to your client.
Just use Polly, has the backing and its already developed :P
Could we add a recipe for using https://github.com/ijpiantanida/talkback for HTTP stubbing?
@Vages Are you able to provide some detail as to how to add Talkback to cypress. I'm getting a bit stuck trying.
Start a Talkback server in cypress/plugins/index.js
. It will run automatically when Cypress starts. An example inspired by the Talkback Readme:
// cypress/plugins/index.js
const talkback = require("talkback");
const opts = {
host: "https://api.myapp.com/foo",
port: 5544,
path: "./my-tapes"
};
module.exports = () => {
const server = talkback(opts);
server.start(() => console.log("Talkback Started"));
}
How this works: When a request is sent to localhost:5544
, Talkback will look for a tape (previous request and response) matching the request, looking at the URL, headers, body, and all. If a matching tape is found, Talkback will respond with the stored response. If not, Talkback will forward the request to https://api.myapp.com/foo
, storing the request and response as a tape. If you send a request to localhost:5544/bar
, it will be forwarded to https://api.myapp.com/foo/bar
and so on.
Common pitfall: I have seen people going "Why isn't Talkback storing any tapes?" because they are sending their requests "production style" during testing, e.g. https://api.myapp.com/foo/baz
. Sadly, Talkback isn't magic (like Cypress): When you run your tests, you have to make your app request the localhost URL that the Talkback server is attached to. For example localhost:5544/baz
instead of https://api.myapp.com/foo/baz
. You've probably got a way of changing API base URLs from development to production already, so just make a test configuration that you use when running with Cypress. But it's likely very project specific, so you have to do that homework yourself.
Thanks @Vages. That was a very complete explanation. Thanks for your effort. Clears things up for me.
I've been trying to get XMLHttpRequest mocking going using pollyjs and the basics have been working (after replacing the xhr adapter with one that works with cypress). I am seeing issues in some cases though, a maximum call stack issue keeps happening. Trying to track it down. Don't know if anyone else has had any luck with pollyjs.
@psachs21 i created this till i have a better solution https://github.com/timpur/cypress-polly-xhr-adapter (https://github.com/sinonjs/nise/issues/70#issuecomment-439674811)
Edit: Just add the git repo url to package.json and it should just work (install)
@timpur This is exactly what I was looking for. Figures someone had already solved it. Much appreciated (even as a temporary solution)
For anyone who is interested in auto stub based on existing cypress, I've created a simple example to demonstrate my solution https://github.com/PinkyJie/cypress-auto-stub-example
There has been some progress made on this issue - but it does involve many parts, so we do not have an ETA at the moment.
With the proxy approach, am I correct to say that it will not work for API with Kerberos authentication?
It would be great if the current browser side stubbing option is kept as an alternative.
Is sendBeacon in scope for this? I was testing to ensure Sentry (error tracking) fires errors appropriately, and saw that the calls to sendBeacon weren't intercepted by the route-stub I had created.
@frattaro yea, I was discussing this over on chat with someone the other day, and it seems that it's not possible to use cy.route
on them, which is understandable, considering.
WTBS, you can stub them and use XHR instead.
Pull Request (in progress): https://github.com/cypress-io/cypress/pull/4176
What users want
window.fetch
What we do now
XMLHttpRequest
object which limits us to only stubbing XHR's from the main application frame.What we need to do
cy.server
and instead just usecy.route
blacklistDomains
option forcypress.json
and we may not be able to provide callback functions anymore.Things to consider
onRequest
andonResponse
currently provide the raw levelxhr
object. This would no longer be possible. We'd have to serialize the request / response into basic objects. No method calling would be available.onError
callback could no longer existonerror
events for XHR's and will automatically fail the test if this is called. This would no longer be possible.https
cannot be dynamic. It would have to be set incypress.json
or environment variables. The reason is that for each blacklisted domain that goes over https, we would have to route that traffic to a self signed certificate. Browser freak out if traffic to https gets assigned one certificate, and then within the same session it gets another cert applied. What this means is that before spawning the browser we have to isolate that traffic and know ahead of time not to route it to the actual server. This enables us to send a single cert to the browser.cy.route
but I'm not sure there's much of a use case here of only dynamically blacklisting non https traffic.XHR
and any other kind ofhttp
request. We'd likely have to change the page events to sayREQ
andREQ STUB
instead ofXHR
andXHR STUB
.onResponse
callback functions. It's possible for the internal server to know when an HTTP request has been completed, but it's not possible to know whether the browser has actually finished processing it. This may or may not be a problem.Bonus Points
cy.stub
, or perhaps a 3rd party extension