Proposal: Support for "first class" Unit Tests

brian-mann commented 7 years ago

To date Cypress has always represented itself as an integration / e2e testing tool.

Nearly all of the cy commands drive the browser in this manner.

Almost There However with automatic ES2015 support we are also tantalizingly close to being able to add first class unit testing support. This would enable users to import their application specific libs and test them in Cypress.

Thinking Ahead And in fact the entire reason we created a cypress/integration folder is so that we could add unit testing support alongside this as cypress/unit. By distinguishing these two types of tests means that Cypress could handle them both differently and optimize against each.

Today's workaround As it stands right now, and as per the cypress-example-recipes you currently just write "unit tests" alongside your integration tests.

But this is not ideal for several reasons:

The problems

Unit tests are just different than integration/e2e tests. Currently we enforce a certain lifecycle to integration tests - meaning between the tests we do things like clear cookies, local storage, and tear stuff down. This has no place in a unit test and is a massive waste of CPU clock cycles and network traffic. In other words, unit tests won't run nearly as fast in this manner.
There is no concept of visiting the application in a unit test. For instance the default view before you cy.visit makes no sense. Perhaps we could repurpose this area as like a "scratch-pad" of sorts for tossing HTML into that does still get torn down between tests.
Most of the cy commands are useless, but others are still really helpful. cy.spy, cy.stub, cy.fixture could all still be used.
Handling things like coverage reports would be possible to automatically since a unit test inlines all of the JS files, whereas an integration/e2e test does not.
Likely sourcemaps could be "on" by default in a unit test.
We could possibly add more commands like cy.render which takes an HTML string and "renders" it to the scratchpad area.

jhanink commented 7 years ago

is there demand for this? I would love to see "first class" unit test support.

lukemadera commented 7 years ago

Seconded! @brian-mann is unit testing coming to Cypress and if so, any ETA? Or should we switch to other tools for now?

brian-mann commented 7 years ago

It is coming, there is no ETA, but you can do this right now. Just import your modules in the spec files and it'll run. It'll just run slower / not be as ideal as other node based tools. But it will work.

We have examples of this (even using Enzyme here).

https://github.com/cypress-io/cypress-example-recipes#contents

lukemadera commented 7 years ago

Thanks for the prompt reply! I'll take a look!

lgandecki commented 6 years ago

Hey guys, could someone elaborate a bit about what this would be useful for? Why not use cypress for ui testing, and tools like mocha/jest/whatever else for unit/integration? Having one test results would be nice (and easier to consume by CIs), but I'm not sure if advantage of that is bigger comparing to using the battle-tested unit testing frameworks. (watch mode of jest is fantastic, for example, and ability to run it with wallabyjs makes it even better)

I hope I'm missing something :)

paulfalgout commented 6 years ago

I personally hope to move my current mocha/chai unit tests to cypress and share the resources more. I am doing little bits of the same work when I would not need to.

lgandecki commented 6 years ago

interesting. I can't think of things that I redoing between e2e tests and unit/integration (I use cucumber with webdriverio/chimp and jest in majority of my projects). By sharing the resources you mean speeding up the test run?

paulfalgout commented 6 years ago

I don't yet run Cypress for e2e tests. I find it most useful for stubbing the server endpoints. I restub a few of those endpoints for some my mocha tests.

lgandecki commented 6 years ago

oh, I see. Very cool. Thanks for sharing!

daKmoR commented 6 years ago

As it's almost a year old now - I'm just wondering has there been any process on unit tests becoming a "first class" citizen?

We may try e2e test with cypress and while we are add it - it would be nice to have also the unit test in one package :p if not it's fine too - just asking if it would make sense to also convert unit test to cypress or keep unit test in a separate system for now?

bahmutov commented 6 years ago

@daKmoR can I ask you what you are trying to unit test that you cannot do already? For example we have unit tests directly in Cypress in the examples https://github.com/cypress-io/cypress-example-recipes/blob/master/cypress/integration/unit_test_application_code_spec.js

It is just a little bare, because the iframe that normally shows your web UI stays empty

daKmoR commented 6 years ago

sure unit test work fine - that is not the problem. However the problems/inconvenience in the original message still remain right? I would be especially interested in

"remove" enforce a certain lifecycle to integration tests
test coverage
actually be able to put them in a separate folder then the integration tests

bahmutov commented 6 years ago

No, sorry, nothing was done in this regard, although with release of preprocessor plugins, you could add code coverage. For separate folder you could put them into a different folder and run with integration folder CLI option https://docs.cypress.io/guides/references/configuration.html#Folders

junyper commented 6 years ago

I would really love to write React/Enzyme tests that render to the DOM and use some of the Cypress commands (especially screenshot). The missing piece for me is the default view/"scratch-pad" area so that I can see what my test is rendering.

cwohlman commented 6 years ago

After playing around with cypress-react-unit-test, I'd really like to see the ability to render my ui tests in the same iframe they're running in & show that iframe when unit tests are running (vs. integration tests). This clears up several side issues I ran into (e.g. react components interacting with the wrong document).

cwohlman commented 6 years ago

Also, many of the points originally raised are still relevant, especially when running in non-dev mode.

bahmutov commented 6 years ago

I don't like the idea of shared environment between the tests and the code / element @cwohlman. Yes, there is danger of doing the wrong thing to the wrong document, but at least there is no easy way to "dirty the water". We probably need a better way to isolate code bundled for mounting from the test code!

egucciar commented 6 years ago

The scratch pad is a really cool idea, @bahmutov , but is something like this readily/easily possible today?

jeffscottward commented 6 years ago

@brian-mann issue was created quite some time ago. whats the status on this? Would love to run our App's redux tests (with latest babel v7 features)

Maybe there is a way to isolate mocha testing with a flag on the CLI? --unit perhaps

brian-mann commented 6 years ago

@jeffscottward people are using Cypress to write unit tests today, and it works just fine. You just configure use the webpack preprocessor or typescript or whatever it is that you do for your own app files and then you can import your components directly in your spec files.

We have example recipes of this (i think) or at the very least @bahmutov has a ton of repos that show examples of this for react, angular, vue, and a bunch of others.

amirrustam commented 6 years ago

@jeffscottward check these out: https://github.com/bahmutov/cypress-vue-unit-test#test-adapters-for-other-frameworks

silbinarywolf commented 5 years ago

I'd like to just point out an issue I hit while playing with importing libraries directly into the browser.

Apologies if this is wrong, but I'm going off my vague memory from a few weeks ago. The "test running" code is running in a seperate to where the element is rendered, any non-test code that uses <code>document.querySelector</code> will not work as expected. To work around this, I simply changed my non-test code... but ideally I shouldn't have to! I want to be able to provide an API similar to <a href="https://flatpickr.js.org/">https://flatpickr.js.org/</a>, so being able to use <code>document.querySelector</code> would be good.</p> <p>I can imagine this would cause issues for people who are trying to install Cypress on a legacy project that utilizes <code>document.querySelector</code> for whatever reason. (Mix of jQuery / VDOM-framework)</p> <p><strong>Code:</strong></p> <pre><code class="language-ts">if (!buttonEl) { throw new Error('Collapo: buttonEl cannot be null'); } const ariaControls = buttonEl.getAttribute('aria-controls') if (!ariaControls) { throw new Error('Collapo: aria-controls missing on button element given.'); } // NOTE: Jake: 2018-11-11 // I'm doing this so that I'm using the correct document context when running // cypress... this is another compelling reason to stop doing everything in Cypress as // they suggest! const document = buttonEl.ownerDocument; if (!document) { throw new Error('Collapo: ownerDocument missing on button.'); } const containerEl = document.getElementById(ariaControls) if (!containerEl) { throw new Error('Collapo: Cannot get element by id: '+'#'+ariaControls+'. Are you missing aria-controls on the button element?'); } if (!styles) { throw new Error('Missing styles param. TODO: Make this optional') } buttonEl.innerHTML = "~~~~~~~~~~~~~collapo initializing~~~~~~~~~~~~~~~" console.warn(styles); buttonEl.addEventListener('click', function(e) { containerEl.classList.add(styles['container--is-active']) })</code></pre> <p><strong>Repo:</strong> <a href="https://github.com/silbinarywolf/collapo/blob/a6de78b1bbf336fa683514e44b5582579ddc272e/src/Collapo.ts#L26">https://github.com/silbinarywolf/collapo/blob/a6de78b1bbf336fa683514e44b5582579ddc272e/src/Collapo.ts#L26</a></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/jennifer-shehane"><img src="https://avatars.githubusercontent.com/u/1271364?v=4" />jennifer-shehane</a> commented <strong> 5 years ago</strong> </div> <div class="markdown-body"> <p>@silbinarywolf Can you open a new issue for this? You should not have to change your application code.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/silbinarywolf"><img src="https://avatars.githubusercontent.com/u/3859574?v=4" />silbinarywolf</a> commented <strong> 5 years ago</strong> </div> <div class="markdown-body"> <p>@jennifer-shehane Sure thing. It's essentially a copy-paste of the above. As stated on the issue, sorry I don't have a more precise / turn-key example, but I currently have other priorities for my down-time :)</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Vandivier"><img src="https://avatars.githubusercontent.com/u/5559355?v=4" />Vandivier</a> commented <strong> 5 years ago</strong> </div> <div class="markdown-body"> <p>I do like the ability to measure coverage which has a separate issue filed, but one issue which doesn't fall under that scope and I think does fall under the scope of first class unit test support is folder structure. I don't merely want to separate /cypress/integration from /cypress/unit, I want to be able to have my spec files in the UI component folder and have cypress run those.</p> <p>That is a convention for UI unit testing and it facilitates moving components between apps. It also removes the existence of monolithic UI app tests. Consider the structure of phonecat demo app for example:</p> <p><a href="https://github.com/cypress-io/cypress-example-phonecat/tree/master/app/core/checkmark">https://github.com/cypress-io/cypress-example-phonecat/tree/master/app/core/checkmark</a></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/silbinarywolf"><img src="https://avatars.githubusercontent.com/u/3859574?v=4" />silbinarywolf</a> commented <strong> 5 years ago</strong> </div> <div class="markdown-body"> <p>I agree with Vandivier.</p> <p>For the E2E/integration tests, it makes sense for Cypress to be seperated by folder structure but for unit/component testing, I want the code/test/styling to be grouped together in vertical slices.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/silbinarywolf"><img src="https://avatars.githubusercontent.com/u/3859574?v=4" />silbinarywolf</a> commented <strong> 5 years ago</strong> </div> <div class="markdown-body"> <p>For people that are currently using Cypress for unit testing, I thought I'd reveal my workaround solution so that <code>document.querySelector</code> will work despite the iframe sandboxing.</p> <p>Basically, I override the spec iframe's <code>document</code> and <code>document.body</code> methods to simply call the function on the app iframe, ie:</p> <pre><code class="language-ts">function overrideSpecIFrameToApplyToAppIFrame() { const specIframe = window.parent.document.querySelector('iframe.spec-iframe'); if (!specIframe) { throw new Error('Cannot find .spec-iframe'); } const specDocument: Document = (specIframe as any).contentDocument; if (!specDocument) { throw new Error('Cannot find contentDocument on specIframe'); } const rootDocument = window.parent.document; const appIframe = rootDocument.querySelector('iframe.aut-iframe'); if (!appIframe) { throw new Error('Cannot find .aut-iframe'); } const appDocument: Document = (appIframe as any).contentDocument; if (!appDocument) { throw new Error('Cannot find contentDocument on appIframe'); } // Override document.body.appendChild on spec iframe so that we can append those // elements to the app iframe. { const bodyAppendChild = function <T extends Node>(this: T): T { return appDocument.body.appendChild.apply(appDocument.body, arguments as any) as T; }; if (specDocument.body.appendChild !== bodyAppendChild) { specDocument.body.appendChild = bodyAppendChild; } } // NOTE: I do this for each method in my implementation below. This is just a small PoC. } before(overrideSpecIFrameToApplyToAppIFrame);</code></pre> <p><strong>Full implementation currently being used for my Aurelia unit testing framework:</strong> <a href="https://github.com/silbinarywolf/cypress-aurelia-unit-test/blob/4d1e7d1b45142d084255113e4881f0369a1fc43c/lib/iframe-fixes.ts">https://github.com/silbinarywolf/cypress-aurelia-unit-test/blob/4d1e7d1b45142d084255113e4881f0369a1fc43c/lib/iframe-fixes.ts</a></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/JimmyLv"><img src="https://avatars.githubusercontent.com/u/4997466?v=4" />JimmyLv</a> commented <strong> 4 years ago</strong> </div> <div class="markdown-body"> <p>actually cypress for unit test is slow, that's true.</p> </div> </div> <div class="page-bar-simple"> </div> <div class="footer"> <ul class="body"> <li>© <script> document.write(new Date().getFullYear()) </script> Githubissues.</li> <li>Githubissues is a development platform for aggregating issues.</li> </ul> </div> <script src="https://cdn.jsdelivr.net/npm/jquery@3.5.1/dist/jquery.min.js"></script> <script src="/githubissues/assets/js.js"></script> <script src="/githubissues/assets/markdown.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/highlight.min.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/languages/go.min.js"></script> <script> hljs.highlightAll(); </script> </body> </html>

cypress-io / cypress

Proposal: Support for "first class" Unit Tests #318