search

cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.02k stars 3.18k forks source link

Cypress.log type definition and/or documentation #1110

Closed NicholasBoll closed 6 years ago

NicholasBoll commented 6 years ago

The implementation of Cypress.log returns a chainable interface with methods like invoke and snapshot, etc. The type definitions just return void. The documentation doesn't say much.

I'm wondering what should the public interface returned by Cypress.log be? Should they reflect the implementation and the docs also get updated?

Is this a Feature or Bug?

Not sure

jennifer-shehane commented 6 years ago

Related documentation issue: https://github.com/cypress-io/cypress-documentation/issues/110

NicholasBoll commented 6 years ago

Thanks @jennifer-shehane

Playing around with snapshot is very useful. Combined with { log: false } of cy commands gives me a smarter snapshot experience.

For example, I have a command that renders React components (also did it with Angular components) where it will add an isolated component at a mount point for per-component testing which is much faster than refreshing the page.

Cypress.Commands.add('mount', name => {
  const log = Cypress.log({
    message: 'Mounting ' + name,
    name: 'mount',
  })
  cy
    .window({ log: false })
    .then((win: Window & { [key: string]: any }) => {
      win.unmountComponentAtNode(win.document.getElementById('test'))
      win.render(
        win.react.createElement(win.RenderWithStore, {}, win.react.createElement(win[name])),
        win.document.getElementById('test'),
      )
    })
    .then(() => {
      ;(log as any).snapshot().end()
    })
})

The snapshot allows me to log just the mount command with a snapshot before cy.mount and after rather (before blank page, after mounted component). The log as any is to get around the Cypress.log returning void according to the type definitions.

cypress-command-snapshot

brian-mann commented 6 years ago

Yes - this is good. This is what we imagined using Cypress to be able to do pure component based testing. It's a little hacky visiting a blank page - we felt like there needs to be something directly in Cypress to "zero out" the application area.

There's a bunch of improvements I wanted to make to Cypress.log() before officially documenting it. It starts to get complex in the sense that it requires other supporting API's to be fully baked before it all "fits together". If you look at our own commands you'll see the full usage of it.

PS. you can name your snapshots by passing them a name - and also provide the "next" name as well, as whenever a command is resolved we automatically take a snapshot of it at the correct time.

brian-mann commented 6 years ago

Can you also try just requiring your components directly in your spec files? Like in a unit test.

I imagine that you are making all of the components accessible via the page served by cy.visit but wouldn't it be easier just to require them and mount them programmatically?

NicholasBoll commented 6 years ago

This test is written against the documentation of a component library. The docs are just a SPA with an index.html and can be served from any file-base webserver (python -m SimpleHTTPServer, nodes http-server, Jenkins artifacts, etc). There is a special test route that just loads with a blank page and a <div id="test"></div> which the mount command uses. The hack is the documentation exposes globals for the mount command to use. The before of a file visits this route and the beforeEach mounts it. I brought this concept from an Angular component library that used Protractor which did the same thing through Selenium's evalScript. It increased test speed by about 10x (load page once per file rather than once per it block)

I suppose we could actually require the components directly. We'd be bringing in all of React, ReactDOM and component source code with the test file as well as with the page. I'd have to experiment with what that does to the loading of a spec (probably increase slightly). Also not sure how this will work with React. I think it will basically be 2 React libraries in the page?

brian-mann commented 6 years ago

No - what I'm suggesting is this. You visit a blank html page with nothing in it. Zero, nadda. Just your basic <html><body><div id='root'></div></body></html>.

You then require the react component in your spec file, and then mount it / render it onto #root. There will only be one react application - the component will live in your spec file, but render to the "application area".

This gives you the benefit of "poking" at the component with cypress commands, or programatically interacting with the component directly since its a direct in memory reference in your spec file. You're basically doing the same thing albeit in a much more indirect way (by exposing them on window).

brian-mann commented 6 years ago 
const myComponent = require('./lib/my-component')

describe('my component', () => {
  before(() => {
    cy.visit('/blank.html')
  })

  beforeEach(() => {
    cy.get('#root')).then(($root) => {
      // mount the component directly here
      // on root
    })
  })

  it('foo', () => {
     // use cypress commands or
     // the myComponent reference directly
   })
})
NicholasBoll commented 6 years ago

Oh, I see. Still have the exposed window variables for the mount, just pass a reference to the component rather than the name of it (which is pulled off window)

brian-mann commented 6 years ago

I don't believe you need to expose anything in the remote window. All of it can be done directly from your spec file.

The only thing specific to your application in the remote window would be CSS styles. That's it. The rest of it should all go in the spec file itself.

I would like to wrap all of these up into a react specific plugin.

NicholasBoll commented 6 years ago

Interesting. There are some challenges there. I went down a rabbit hole of configuration changes to make TypeScript and Webpack happy. Our documentation examples don't use relative paths, but rather the library's package name so example code can be copy/pasted into an app and "just work".

As I suspected, the spec files just run in the browser and require a bundler for import/require statements to work. So my component requires React which also gets bundled per spec file (unless I expose React on window and have the bundler reference window.React rather than bundling per test).

For now it is more work to get a reference through JS than it is to just expose the component on window.

I'm interested in how you'd solve this with a mount plugin.

NicholasBoll commented 6 years ago

I have to say the ability to customize the log and snapshot experience is awesome! Our previous CodeceptJS tests have very messy output (every command was a log entry) and made it almost impossible to figure out what was going on in a test.

NicholasBoll commented 6 years ago

I got past the resolution issues, but our components uses CSS modules. Basically the entire app is being built per spec file. The only way to share that code is through Webpack's externals which requires those modules to be exposed on window anyway. Angular 1 was a bit easier since I just exposed angular on window and that exposed the whole app. I could then use angular.module to get at whatever I needed. React is a bit different - component references aren't exposed to external processes.

I'm sure it is possible since the React dev tools do it, but I'm not sure how.

I'm interested in how this might work with a React plugin. For now I'll keep our app just using the global components exposed through the documentation.

brian-mann commented 6 years ago

You can access react internals through properties directly on the DOM elements.

Here's how testcafe does it https://github.com/DevExpress/testcafe-react-selectors/tree/master/src/react-16

NicholasBoll commented 6 years ago

From that example I can see how given a DOM element, I can get the component's constructor (the component class): $0.__reactInternalInstance$bea5aytb3j4._currentElement._owner._instance.constructor. I think that would require the component to be rendered already. I could then grab the component reference, unmount and now mount whenever I need. That might not be a bad compromise. Our examples do have individual routes to test them in isolation of the rest of the documentation. At that point I could just manipulate the route to force the App to remount.

If a component isn't rendered, I'm not sure how to access its constructor.

brian-mann commented 6 years ago

Right, it would already have to be rendered. But that is what we will do for the custom cy.component command which instead of querying the DOM, it will search and find the component by name.

The cy.mount stuff won't help us here but I imagine there is some middle ground for how to do this. The direct references would be ideal as this would come very close to being a true unit test while including the power of Cypress.

brian-mann commented 6 years ago

Released in 3.0.0.