Component Testing never finds any tests

azerkail commented 3 years ago

Current behavior

Hi!

I believe there seems to be some error with the component testing framework at the moment that is making it completely unusable. I followed all the steps in the documentation to be able to component test Vue components (in my case I am also using TypeScript), and after making several tweaks I was unable to get the runner ever to work because it can never find any test errors (see image below).

I have tried the following changes to fix this behaviour (neither of these have worked, still getting tests not found):

Removed componentFolder from cypress.json to use the default path
Written a JavaScript spec file instead of TypeScript
Written a JavaScript Vue component AND a spec file instead of using TypeScript
In the other company project that I found this problem, we were using path alias and project references so I thought maybe that was what was causing the issue so I stopped using the custom webpack.config.js that you can find in the repo posted below, so the plugins/index.ts just setups on('dev-server:start') with the config from the Vue CLI service, no dice

Nothing has been able to make the test runner pick up the tests within the spec files 😥

Also I have another weird problem where my default runner is Opera GX but the component runner always opens up Edge for some reason 😁

I have also noticed that someone opened the same ticket about React a few hours ago https://github.com/cypress-io/cypress/issues/16807

Desired behavior

The runner does seem to pick up the .spec.ts files I wrote in the glob expression, but it never finds the describe/it inside so it always reports that No test files were found.

Test code to reproduce

I have gone off and made a public repo that you should be able to clone and run up: https://github.com/St1fle/cypress-component-testing-error

Given all the changes I've done it doesn't seem like this is something to do with my specific configuration but rather just a global problem with the framework right now but in case I am wrong and it's something I have you can just clone the repo and run npm run test. You should see one Button.spec.ts case on the left that if you click on brings up the error. I've also left the example JS files that Cypress creates when it runs the first time to showcase that even for those files the component test runner fails to find tests on.

Versions

Cypress version: 7.4 Browser: Edge (although my default browser is Opera GX) Operating System: Windows CI Provider: N/A (happens locally)

Other package versions can be found within the package.json.

jennifer-shehane commented 3 years ago

@St1fle Thanks for reporting the issue. In your example repo, we don't have access to install the @yottaltd/eslint-config and @yottaltd/eslint-plugin dependencies. These need to be removed in order to run npm install successfully.

The issue can be recreated after removing these deps and their references.

jennifer-shehane commented 3 years ago

Maybe this issue is also related? https://github.com/cypress-io/cypress/issues/16807

azerkail commented 3 years ago

Thanks for the quick reply @jennifer-shehane !

That was my mistake I copied the package.json from the company's internal project and forgot to remove the private packages, I've pushed a commit to the repo that removes these two so hopefully is not a problem anymore 👍

Yes I mentioned in the description above that the other ticket (#16807) is probably the same problem as this. When I opened my ticket, I wasn't sure if the problem was because I was using some combination of TypeScript and Vue that was causing it, but after testing it with raw JavaScript and seeing the other person's open ticket, who is using React, I'm fairly confident that it's a widespread issue.

lmiller1990 commented 3 years ago

There is an issue in @cypress/react (has a PR now). There's something odd in your screenshot - it looks like your component tests and e2e tests are showing up. I will pull your repo and explore - not sure if it's a config issue, something in Cypress, or both. You will need some way to differentiate between component tests and e2e tests, though.

lmiller1990 commented 3 years ago

I tried removing src/specs/examples (so the only test remaining is Button.spec.ts). I got a new error:

ERROR in ./src/specs/buttons/Button.spec.ts 9:18
Module parse failed: Unexpected token (9:18)
File was processed with these loaders:
 * ./node_modules/eslint-loader/index.js
You may need an additional loader to handle the result of these loaders.
|   const disabled = true;
|
>   const properties: ButtonProperties = new ButtonProperties({
|     title,
|     disabled,
 @ ./node_modules/@cypress/webpack-dev-server/dist/browser.js (./node_modules/@cypress/webpack-dev-server/dist/loader.js!./node_modules/@cypress/webpack-dev-server/dist/browser.js) 5:16-155
 @ ./node_modules/@cypress/webpack-dev-server/dist/browser.js

I think what should you do is use cypress/webpack-preprocessor for e2e tests, but @cypress/webpack-dev-server for component tests. You can do conditional configuration using config.testingType, see here.

Also, you need to return config from your plugins file. I updated your plugins file:

/* eslint-disable */
// https://docs.cypress.io/guides/guides/plugins-guide.html

const webpack = require('@cypress/webpack-preprocessor');
const vueWebpackConfig = require('@vue/cli-service/webpack.config');
const alloyWebpackConfig = require('./webpack.config');
const { startDevServer } = require('@cypress/webpack-dev-server');

module.exports = (on: any, config: any): void => {
  // Setup custom webpack to support typescript.
  if (config.testingType === 'e2e') {
    on(
      'file:preprocessor',
      webpack({
        webpackOptions: alloyWebpackConfig,
      }),
    );
  }

  if (config.testingType === 'component') {
    // Setup webpack to use the same config as the Vue CLI.
    on('dev-server:start', (options: any) => {
      return startDevServer({
        options,
        webpackConfig: vueWebpackConfig,
      });
    });
  }

  return config
};

After I did that:

I'm a bit unsure now. Is the webpack configuration provided (imported from const vueWebpackConfig = require('@vue/cli-service/webpack.config')) set up for TypeScript? I am not sure what this linting error is regarding. I think there is some files missing from your reproduction?

azerkail commented 3 years ago

Hi @lmiller1990 ! Thanks a lot for taking a look at this 😄 you did found the issue! It was because I was not returning the config variable as part of the webpack.config.js so I am now at the same point as you and I can see that error as well.

Just to clarify, I am not trying to run end to end tests and component testing at the same time, I am just trying to component test Vue components while writing the tests in TypeScript, even in the example repo I shared, but thanks for the tip! It will probably be useful in the near future 😃 (the example files were just there to show that I got the "No test files found" error regardless if the tests were in JavaScript or TypeScript)

So now I am getting the same error as you but I don't know how to proceed from here. So I understand that we don't need the file:preprocessor since that is being handled by the webpack.config.js provided to dev-server:start.

The problem is I can't use the webpack.config.js from the Vue CLI with require('@vue/cli-service/webpack.config') because we're writing our tests in TypeScript, so we are now seeing this error because Webpack is failing to compile Button.spec.ts since it contains TypeScript that it doesn't know what to do with, I assume it's showing the eslint loader in the error message because it's the last loader trying to parse the file.

If I swap the index.ts in plugins to use my custom webpack config, and my config looks like this:

// index.ts
const customWebpackConfig = require('./webpack.config');
const { startDevServer } = require('@cypress/webpack-dev-server');

module.exports = (on: any, config: any): void => {
  on('dev-server:start', (options: any) => {
    return startDevServer({
      options,
      webpackConfig: customWebpackConfig,
    });
  });

  return config;
};

// Custom webpack.config.js
module.exports = {
  resolve: {
    extensions: ['.ts', '.js', '.json'],
  },
  module: {
    rules: [
      {
        test: /\.ts$/,
        exclude: [/node_modules/],
        use: [
          {
            loader: 'ts-loader',
            options: {
              // https://github.com/cypress-io/cypress/issues/2316#issuecomment-412312754
              transpileOnly: true,
            },
          },
        ],
      },
    ],
  },
};

I get this error:

Ok! This makes sense, webpack doesn't know how to compile the Vue Component I am trying to mount with Cypress. So then, I tried installing vue-loader and adding that as part of the custom webpack config:

module.exports = {
  resolve: {
    extensions: ['.vue', '.ts', '.js', '.json'],
  },
  module: {
    rules: [
      {
        test: /\.ts$/,
        exclude: [/node_modules/],
        use: [
          {
            loader: 'ts-loader',
            options: {
              // https://github.com/cypress-io/cypress/issues/2316#issuecomment-412312754
              transpileOnly: true,
            },
          },
        ],
      },
      {
        test: /\.vue$/,
        loader: 'vue-loader',
      },
    ],
  },
};

I get this:

So now it's telling me that vue-loader doesn't understand the native HTML tag button? Which I can't really understand why :/

So in short, now the problem seems to be that there isn't an easy way to provide a webpack.config.js to Cypress so it can understand both TypeScript files and Vue files at the same time (or at least I don't know how to!).

You do know a lot more than me about this so let me know if you found anything! Oh, and I also updated the example repo to reflect this if that helps 😄

elevatebart commented 3 years ago

Hello @St1fle There was a couple of small issues with your setup

Here is a PR that fixes them one after the other

When using the vue-loader to load vue Single File Component, you have to load the VueLoaderPlugin as well
The null coalescing operator ( ?? ) stays untouched by TypeScript if you target anything after ES2020, but the parser in webpack will fail on this item, so in tsconfig.json I set a target of 2019
The propsData property expects an object. If you open the chrome console when the tests are running, you will see Vue complaining that the "properties" prop is missing from the button component. This is fixed by making it a props bag.

I hope this PR helps. The tests run now

azerkail commented 3 years ago

Thank you so much @elevatebart !

I just went through them and can confirm the fixes in that PR, resolves these issues. I was not aware of these nuances so this was perfect for me.

The only thing I could suggest is maybe adding this to the documentation somewhere for people less experienced like me can know what to do if they want to run the same combination of TypeScript + Vue.

I don't know what the protocol is but feel free to close this issue as resolved 👍 Again thanks for the help, everyone was really helpful 😊

elevatebart commented 3 years ago

I will close this issue once @chrisbreiding and I figure out how to avoid having to always return the config object. We consider this a bug and it has been nagging users for the better part of this year.

The rest of the fixes are specific to the vue+typescript setup.

Since we usually use an already working webpack setup, like VueCLI or create-react-app, you should not have to setup your own webpack config but reuse the one provided for you. So one should not have any problem. We will try to add some links to those docs.

Thank you for spending the time reporting the issue.

azerkail commented 3 years ago

I realised I copied the webpack config from our end to end test project which doesn't return the config and that's how I hit this problem 😅

I agree with you guys that the config from Vue or React probably serves most people use cases, unfortunately we're an exception to this because we are running a TypeScript monorepo using project references and path aliases, it contains several micro apps built with Vue that we test with Cypress so I don't think we can avoid a custom config :/

lmiller1990 commented 3 years ago

Since this is solved, I think we should close it. We could make another ticket to follow up on the issue raised by @elevatebart ?

elevatebart commented 3 years ago

Yeah, it would be clearer. I will create the issue and mention it here when i close

cypress-io / cypress