Add Testrunner

[nothingismagick]

I started setting up a test-runner. The idea I am working with - that I hope is in-line with the Quasar method - is to allow the developer to quasar add test- which then creates a new test from a template. This new file is then found in the webpack loop of the dev-server, and mocha-runner executes the tests as part of webpack's dev-packaging job.

Here you can see the status: https://github.com/nothingismagick/quasar-cli/tree/test-runner

I have installed the deps and created a passing test-unit (that I have verified by adding a script value to the package.json - now, if you type "npm test" you will see that the tests are immediately run in the command line.

There is still a little bit of work to do, but really not that much to get the basics up and running. It should be possible to do "static" mocha/chai testing very soon. Testing within scope is likely to be a great deal more challenging. It seems that the vue.js team solved it using https://github.com/zinserjan/mocha-webpack - but that brings its own entire webpack service - which isn't really appropriate here.

Here is their solution: https://github.com/vuejs/vue-cli/tree/dev/packages/%40vue/cli-plugin-unit-mocha

[musicformellons]

In line with this issue I also would like testing to be easy to integrate. I got cypress (e2e testing) working as it is easy to setup, however I would really like to have Jest (unit testing) working as well. I had it running with version 0.14, but I do not manage to get things (properly) working with version 0.15 see e.g. this.

[nothingismagick]

@musicformellons - I got some basic stuff working in quasar-cli as a mode, and I am waiting for the team to get back from vacation so that we can discuss its integration. I hope that by the end of April we can push it out. Here is how you can test it out:

git clone git@github.com:nothingismagick/quasar-cli.git
cd quasar-cli
git checkout test-runner
yarn install
yarn link
cd ../
mkdir test
cd test
yarn init
yarn link quasar-cli
quasar init testapp
cd testapp
yarn link quasar-cli
quasar mode --add test

As far as the stack-overflow question goes, I would recommend closing it and asking over at our discord channel: https://discord.gg/4j2Qq4Q

[laurentpayot]

@musicformellons btw you can run your unit tests in Cypress too https://github.com/cypress-io/cypress-example-recipes/blob/master/examples/unit-testing__application-code/cypress/integration/unit_test_application_code_spec.js

[musicformellons]

That's probably easiest currently. Probably not as speedy? I also feel a bit silly since I bought this book which seems (I am halfway) thorough and works with Jest and vue test library.

[musicformellons]

@laurentpayot I am considering using just Cypress. When I import my client api I get errors which indicate that I need to have my cypress spec files compiled with babel, did you manage in setting this up?

[nothingismagick]

Hey @musicformellons & @laurentpayot - this issue is about adding a test mode to quasar-cli. Are you working on just getting it to work in your quasar app, or are you trying to help us get this into the codebase?

[musicformellons]

Hi @nothingismagick , uh, I think a bit of both..., they do not seem contrary to each other...?!

[nothingismagick]

Ok -cool. So that means you are suggesting that we offer the user a variety of testing frameworks to write their code with? I am not sure that is a good idea in the sense of maintainability.

[nothingismagick]

I would really prefer to choose one framework and support it perfectly and in continuity. There are lots of options out there, and I also looked around a bit... but if you two can really convince me that one framework is better to use than another - or come up with some logic to switch them out - then I am totally happy to work with you on it this way.

[laurentpayot]

@musicformellons & @nothingismagick I created a Cypress folder in my root directory. It uses the same .babelrc as my Quasar project. And in my Cypress/plugins/index.js I have the following module section in my webpack options (I'm also using CoffeeScript btw):

      module: {
        rules: [
          {
            test: /\.js$/,
            loader: 'babel-loader',
            exclude: /node_modules/
          },
          {
            test: /\.coffee$/,
            loaders: ['babel-loader', 'coffee-loader'],
            exclude: /node_modules/
          },
        ]
      },
    },

More details in the Cypress webpack plugin docs. Hope it helps. Having all the tests (unit and integration) running in a single Cypress window is really cool.

[musicformellons]

@laurentpayot I tried to replicate your settings (leaving out the coffeescript), but it seems to error on the way quasar imports components in the routes.js (I am using v0.15 project setup). I get (gave just the error for one component, but it repeats for all pages/ components of my app):

./src/router/routes.js
Module not found: Error: Can't resolve 'components/Home' in '/home/usr/y1/src/router'
resolve 'components/Home' in '/home/usr/y1/src/router'
  Parsed request is a module
  using description file: /home/usr/y1/package.json (relative path: ./src/router)
    Field 'browser' doesn't contain a valid alias configuration
    resolve as module
      /home/usr/y1/src/router/node_modules doesn't exist or is not a directory
      /home/usr/y1/src/node_modules doesn't exist or is not a directory
      /home/usr/node_modules doesn't exist or is not a directory
      /home/node_modules doesn't exist or is not a directory
      /node_modules doesn't exist or is not a directory
      looking for modules in /home/usr/y1/node_modules
        using description file: /home/usr/y1/package.json (relative path: ./node_modules)
          Field 'browser' doesn't contain a valid alias configuration
          using description file: /home/usr/y1/package.json (relative path: ./node_modules/components/Home)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              /home/usr/y1/node_modules/components/Home doesn't exist
            .wasm
              Field 'browser' doesn't contain a valid alias configuration
              /home/usr/y1/node_modules/components/Home.wasm doesn't exist
            .mjs
              Field 'browser' doesn't contain a valid alias configuration
              /home/usr/y1/node_modules/components/Home.mjs doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
              /home/usr/y1/node_modules/components/Home.js doesn't exist
            .json
              Field 'browser' doesn't contain a valid alias configuration
              /home/usr/y1/node_modules/components/Home.json doesn't exist
            as directory
              /home/usr/y1/node_modules/components/Home doesn't exist
[/home/usr/y1/src/router/node_modules]
[/home/usr/y1/src/node_modules]
[/home/usr/node_modules]
[/home/node_modules]
[/node_modules]
[/home/usr/y1/node_modules/components/Home]
[/home/usr/y1/node_modules/components/Home.wasm]
[/home/usr/y1/node_modules/components/Home.mjs]
[/home/usr/y1/node_modules/components/Home.js]
[/home/usr/y1/node_modules/components/Home.json]
 @ ./src/router/routes.js 11:15-40 20:15-40 32:15-40
 @ ./src/router/index.js
 @ ./src/api.js
 @ ./tests/e2e/integration/home_page_spec.js

My routes.js look something like this:

export default [
  {
    path: '/',
    component: () => import(`components/Index`),
    children: [
      {
        path: '',
        name: 'home',
        components: {
          default: () => import(`components/Home`)
        }
      },
      {
        path: '/userprofile',
        name: 'userprofile',
        component: () => import(`components/UserProfile`)
      }
     ]
  },
  {
    path: '*',
    component: () => import(`pages/404`)
  }
]

My questions: 1) You mention: "It uses the same .babelrc as my Quasar project." You need to specifically set this, or (as I think) the .babelrc in my project root is picked up by default by the babel-loader? 2) You have similar dynamic import routing (which is quasars default) and no errors on this? 3) Any idea what's wrong maybe?

I could make an issue at cypress (as I did on something similar recently), but they lack the experience with quasar, which seems to trigger -as far as I can see- the errors I encounter. So your experience with a working quasar-cypress setup is very welcome indeed!

[musicformellons]

@laurentpayot BTW I just made an issue here. I was just wondering: which version of webpack preprocessor did you use? They mention current version: "This version is only compatible with webpack 4.x+" Since Quasar-cli uses webpack 3, maybe I need to use @cypress/webpack-preprocessor 1.x ?

[laurentpayot]

@musicformellons

1. I don't know the exact mechanism but it's working with my conf ;)
2. yep I use dynamic import routing too
3. It looks like webpack can't resolve your components. You're using import('components/Home'), it's working in Quasar because components/ alias is already set in Quasar webpack config. In Cypress webpack config you have to set the same aliases as in Quasar. Note the alias section of my complete config below:

const webpack = require('@cypress/webpack-preprocessor')
const path = require('path')

module.exports = (on) => {
  const options = {
    // send in the options from your webpack.config.js, so it works the same
    // as your app's code
    webpackOptions: {
      resolve: {
        extensions: ['.js', '.coffee', '.json'],
        modules: [
          path.resolve(__dirname, '../../src'),
          path.resolve(__dirname, '../../node_modules')
        ],
        alias: {
          src: path.resolve(__dirname, '../../src'),
          components: path.resolve(__dirname, '../../src/components'),
          support: path.resolve(__dirname, '../support')
        }
      },
      module: {
        rules: [
          {
            test: /\.js$/,
            loader: 'babel-loader',
            exclude: /node_modules/
          },
          {
            test: /\.coffee$/,
            loaders: ['babel-loader', 'coffee-loader'],
            exclude: /node_modules/
          },
        ]
      },
    },
    watchOptions: {},
  }

  on('file:preprocessor', webpack(options))
}

Do not forget to close your Cypress issue if the aliases fix your problem…

[musicformellons]

@laurentpayot Seems I am progressing but I still get errors from my .vue files. How does your setting ensure compilation of .vue files as you only configure .js files and .coffee files? You are on version 2 of webpack-preprocessor.

[musicformellons]

I am starting to fear that I need to setup all the nicely hidden compiling stuff by the Quasar-cli to get this working. Including installing all the loaders etc. like quasar used to do before version 0.15, e.g.:

 rules: [
          {
            test: /\.js$/,
            loader: 'babel-loader',
            // include: projectRoot,
            exclude: /node_modules/
          },
          {
            test: /\.vue$/,
            loader: 'vue-loader',
            options: {
              postcss: cssUtils.postcss,
              loaders: merge({js: 'babel-loader'}, cssUtils.styleLoaders({
                sourceMap: useCssSourceMap,
                extract: env.prod
              })),
              transformToRequire: {
                video: 'src',
                source: 'src',
                img: 'src',
                image: 'xlink:href'
              }
            }
          },
          {
            test: /\.json$/,
            loader: 'json-loader'
          },
          {
            test: /\.(png|jpe?g|gif|svg)(\?.*)?$/,
            loader: 'url-loader',
            options: {
              limit: 10000,
              name: 'img/[name].[hash:7].[ext]'
            }
          },
          {
            test: /\.(woff2?|eot|ttf|otf)(\?.*)?$/,
            loader: 'url-loader',
            options: {
              limit: 10000,
              name: 'fonts/[name].[hash:7].[ext]'
            }
          }
        ]

Aaargh, I hope that is not true? That would make cypress (or the new quasar-cli setup) setup less ideal than I was hoping for.

[laurentpayot]

@musicformellons

• end-to-end testing (integration testing) is just listening to localhost:8080 port and simulating user actions. For that you have to build then serve your app (or eventually use quasar dev). Cypress does not compile Vue files.

• Unit testing is just testing algorithms and complex functions in a lib directory for instance. Webpack aliases are just telling Cypress how to import unit test resources within Quasar app directory in case of unit testing.

• Vue testing (props etc.) is something totally different and IMHO is not worth the pain when already using end-to-end tests.

Really Cypress is great for both end-to-end and unit tests. But maybe you're trying to test Vue components, and if that's the case I don't think it's possible with Cypress or any other e2e testing tool…

[laurentpayot]

PS: Note that I'm using webpack 3.

[musicformellons]

@laurentpayot Thanks for explaining, as that indeed makes sense and maybe I am completely wrong in 'what I want/ try'. So I understand now how you can 'avoid' the problem I have right now.

My case is as follows:

• I use websockets (not html REST).
• To test these I can not use the cypress request with post, get, etc. So following their recommendation I need to 'stub' my backend polls (I think that's the equivalent of requests in websocket-land).
• To make my stub I need to import my app's api-module. This api module also imports other stuff (among which quasar's 'notify' component) so that's where the need for all the subsequent transpiling comes from.

Probably I need to try to refactor the api module to a minimum which should be easier to transpile with just a .js loader.

[nothingismagick]

@laurentpayot - i really have to disagree with not needing unit tests. i think that they are really the only way to enforce spec compliance for projects that have big teams. it is a great way to make sure that you get what you expect. coverage is good to know about so you see parts of your code that are never reached. E2E is important to make sure that your atomic functions collaborate.

@musicformellons - there are really a lot of ways to do unit tests, and the fork that this issue is about tries to find a compromise between staying conformant with the quasar-cli and enabling the developer to write their own tests and manage coverage. later we will be using this for our CI patterns, so it is important to get it right in a sustainable way.

have either of you tried the fork of this repo that i wrote? this is how to use it:

git clone git@github.com:nothingismagick/quasar-cli.git
cd quasar-cli
git checkout test-runner
yarn install
yarn link
cd ../
mkdir test
cd test
yarn init
yarn link quasar-cli
quasar init testapp
cd testapp
yarn link quasar-cli
quasar mode --add test

[laurentpayot]

@nothingismagick who said I was wasn't doing unit tests ? I do a lot of unit tests. Simply I'm not unit testing my vue components because:

1. my components are trivial because all the complex logic is in separated imported libraries functions that are extensively unit tested.

2. the way my components interact is already tested via Cypress integration tests

3. unit testing components is a nightmare and you will spend more time setting it and writing tests than writing the whole app.

Of course if I had a team of testers like in my former (big) company I wouldn't mind if they spend time on tests with low added value…

And I do E2E tests too (that's what Cypress was made for) so maybe you misunderstood me.

[nothingismagick]

I must have misunderstood you, I was trying to clarify the broad approach that I think we need to maintain for integrating a testing suite into the quasar-cli - which is why we are discussing it here in an issue instead of on discord.

[laurentpayot]

@musicformellons I'm using Firebase so I had the same issue as you.

To use Firebase node admin functionalities (test users creation on the fly, JWT generation etc.) I had to set a local express server up that I'm calling inside Cypress tests via POST fetch requests.

Cypress is working on a back-end messaging plugin. Well at least they say they it would be great: https://github.com/cypress-io/cypress/issues/684

[laurentpayot]

@nothingismagick no worries :wink:

[musicformellons]

@laurentpayot So you only import js files, no vue files. It looks like the only reason you need the webpack-preprocessor is since you use coffeescript as the default cypress install already supports ES6. Right?

@nothingismagick I am interested in your unit-testing solution, but first need to fix this cypress config and I must say I find the Laurentpayot test philosopy tempting.

[laurentpayot]

@musicformellons and also to add the same aliases as Quasar webpack config, so importing lib files in Cypress for unit testing will work the same as in Quasar.

[musicformellons]

@laurentpayot Cool, yeah! Thanks for helping out!

[nothingismagick]

@musicformellons - I closed this issue, but I wanted to know if I might be able to interest you in collaborating with me on the new @quasarframework/quasartest team. I am now building a monorepo called @quasar/test that will let users integrate the testing harness of their choice into quasar. It is pretty ambitious, and I could really use your input!

[musicformellons]

@nothingismagick Thanks for asking as this does have my interest. As I am seriously time-restricted at the moment I am afraid I can't help right now. When circumstances change I'll let you know!