search

hackforla / tdm-calculator

DTLA Hack for LA is partnering with Los Angeles Department of Transportation (LADOT) to develop a Traffic Demand Management (TDM) calculator tool. This tool will help planners at LADOT and real estate developers to meet the Los Angeles’s Mobility Plan goals by 2035.
https://tdm.ladot.lacity.org
GNU General Public License v2.0
48 stars 32 forks source link

Architectural Decision: Testing frameworks #231

Closed KPHowley closed 4 years ago

KPHowley commented 4 years ago

Overview

Research various testing frameworks (Cypress, Selenium, and React) to determine the best testing practices and procedures to implement.

Action Items

Resources/Instructions

Template: https://docs.google.com/document/d/1qvTyjaS0CnmT3umqWg9LJmAcZzHL6tpi5eCQWeh6ovQ/edit?usp=sharing

KPHowley commented 4 years ago

follows up #207

fyliu commented 4 years ago

I added what I have in the page following the template. I also changed the file name to match this issue.

fyliu commented 4 years ago

We discussed cypress and selenium and I felt that both are very similar in usage. The tangible difference is that selenium requires installation of webdriver binaries for target browsers, which John had issues with setting up for the Jobs-for-Hope project. Cypress likes to show the replay feature, which is potentially good in regression testing as a debugging tool. It feels less useful for acceptance testing. We've decided to implement a test case in both to compare next week.

@nclairesays will create cypress test for Barrington @fyliu will create selenium test for Barrington

fyliu commented 4 years ago

Continuing the discussion with what I found while working toward Selenium testing.

https://medium.com/welldone-software/an-overview-of-javascript-testing-in-2019-264e19514d0a

A couple disadvantages of cypress, side-effects of its running as a layer inside the browser rather than along side it:

Selenium itself is built for browser automation and not specifically for testing although it can be used that way. The webdriver it provides is used by other tools that are made for testing, and the most used one seems to be WebDriverIO, which is what I will use.

https://www.cypress.io/how-it-works

This one diagram is a good summary of the difference that developers would see. Cypress is the Angular of the testing world while Selenium is more like React. By this, I mean that Cypress is an all-in-one framework that provides as much as it can and the developer need not look outside the walls. Selenium does one thing which is the WebDriver which controls web browsers. It doesn't provide a testing framework, assertion module, convenience wrapper, or mocks. A developer is free to, or actually, must, choose those things, which can be very positive for projects that already have something in place, or negative, for projects with decision fatigue. Cypress has already made all the decisions.

joelparkerhenderson commented 4 years ago

You may want to take a look at ING Lion for emerging ideas about JavaScript testing. Lion is a compelling new JavaScript project (in my opinion) and has a significant emphasis on testing because the tools are for finance and government. https://github.com/ing-bank/lion

ExperimentsInHonesty commented 4 years ago

continuing testing phase. Next discussion 2/18.

nclairesays commented 4 years ago

Here's information on how Cypress works (and how it's different from Selenium): https://www.cypress.io/how-it-works

Edit: Woops, just noticed Fang posted the same link in a comment above.

I would argue that running cypress inside the browser is not a con. The link lists key differences between the two. Additionally, from what I've read and from discussion with other developers at work, Cypress pretty much resolved a lot of things that developers hated about Selenium. I also read that Cypress will fit best in Javascript-focused development teams, which we are.

Scroll to the bottom half of the link under "Key Differences" for more info. Here's a snippet:

Most testing tools operate by running outside of the browser and executing remote commands across the network. Cypress is the exact opposite. Cypress is executed in the same run loop as your application. Behind Cypress is a Node.js server process. Cypress and the Node.js process constantly communicate, synchronize, and perform tasks on behalf of each other. Having access to both parts (front and back) gives us the ability to respond to your application's events in real time, while at the same time work outside of the browser for tasks that require a higher privilege.

Because Cypress operates within your application, that means it has native access to every single object. Whether it is the window, the document, a DOM element, your application instance, a function, a timer, a service worker, or anything else - you have access to it in Cypress. There is no object serialization, there is no over-the-wire protocol - you have access to everything at your fingertips.

Having ultimate control over your application, the network traffic, and native access to every host object unlocks a new way of testing that has never been possible before. Instead of being 'locked out' of your application and not being able to easily control it - Cypress instead lets you alter any aspect of how your application works. Instead of incredibly slow and expensive tests such as creating the state required for a given situation - you can simply create these states artificially like you do in an unit test.

nclairesays commented 4 years ago

Here's some basic tests for the barrington condos test case using cypress. For anyone who wants to check it out, here's the branch I'm working on. Made 3 commits. Make sure to do yarn install for server and client and run both. Then, to run the tests, cd into '/cypress' and run yarn cypressor npm run cypress and that's it. Easy peezy :) https://github.com/hackforla/tdm-calculator/tree/claire-acceptance-tests

fyliu commented 4 years ago

I made a branch using selenium(WebDriverIO) to create the test case for Barrington. The test data comes from a json file.

  1. Install everything from the project root. No change to the client.

    npm install

  2. Run the test case

    npm run tests:e2e

    https://github.com/hackforla/tdm-calculator/tree/selenium_testing

fyliu commented 4 years ago

I think we're going with cypress for end to end for ease of use. For unit testing, we have storybook that we've been using. We've also been using Jest. I'm not sure if we're still looking at the tools from @testing-library.

ExperimentsInHonesty commented 4 years ago

@nclairesays - john says you were going to do a pull request. Let us know where you on that.

  1. Progress
  2. Blocks
  3. Availability
  4. ETA
ExperimentsInHonesty commented 4 years ago

@entrotech says after we settle on the testing library, we will need to figure out how we structure the tests. @bfojas says there is information for stubbing requests, that he will review after Claire's changes.

nclairesays commented 4 years ago

Sorry guys, didn't see this thread. I just made a pull request with the Barrington condos test case using cypress. @bfojas @entrotech

ExperimentsInHonesty commented 4 years ago

Fang posted in slack: Fang Liu 10:11 AM Another thing with testing libraries. I heard that jest/enzyme can't test react hooks by shallow rendering and react testing library can. I think someone mentioned this at a meeting, maybe Bradley Fojas