ESLint-6: All rules fail to load when config contains overrides.extends

nick-invision commented 4 years ago

Actual Behavior

When using the ESLint-6 channel, if the .eslintrc.js file contains an overrides w/ an extends property, all rules fail to load. E.g.,

module.exports = {
  ...
  overrides: [
    {
      ...
      extends: [
        "plugin:@typescript-eslint/recommended",
      ],
      ...
    }
  ]
}

This causes inaccurate and unexpected lint results. This message is logged (note the config file has many rules):

No rules are configured. Make sure you have added a config file with rules enabled. 
See our documentation at https://docs.codeclimate.com/docs/eslint for more information.

Expected Behavior

Overrides.extends should not break the config and accurate lint results should be returned.

Steps to reproduce

See attached .codeclimate.yml, .eslintrc.js, and cli_output.log: ESLint6_Bug.zip

efueger commented 4 years ago

Hi @nick-invision -

Thanks for reaching out. Can you upload the .codeclimate.yml and .eslintrc.js again? I think they missed the zip file.

Emily

nick-invision commented 4 years ago

Hi @nick-invision -

Thanks for reaching out. Can you upload the .codeclimate.yml and .eslintrc.js again? I think they missed the zip file.

Emily

Weird, I'm not sure what happened. I was able to download them from that zip. I've reattached (as txt this time). Thanks for the quick response! cli_output.txt codeclimate.txt eslintrc.txt

efueger commented 4 years ago

My fault! No idea why my hidden files settings wasn't enabled. 😬 I tested on my side with those configs and wasn't able to reproduce the error message.

To help troubleshoot, can you run the following command and paste the output?

CODECLIMATE_DEBUG=1 codeclimate analyze

nick-invision commented 4 years ago

I created a sample project and Code Climate job to demo this at a much smaller scale than we're seeing it in our private repo.

Repo with steps to reproduce in README Logs containing the "No rules are configured" message "No rules are configured" message in Code Climate job "Configured rules" in repo

efueger commented 4 years ago

Thanks @nick-invision -

It looks like this channel supports typescript-eslint/parser, but I'm checking with the team to find out more about how to use the recommended rules.

I'll update you here with news as it comes in. 👌

chrisbutler commented 4 years ago

@efueger we are seeing the same issue... but to add some clarity it appears that it breaks when using extends in the root of the ESLint configuration file (as opposed to being inside of overrides as the original post states)

we've been stuck on eslint-5 for over a year now. what's the status on this?

fede-moya commented 3 years ago

Hey there @nick-invision @chrisbutler 👋, hope you're doing fine and apologizes for the long delayed with this. Also, the repo that @nick-invision has setup for reproducing the issue has been supper helpful really, thanks for that.

Being said that, I have just opened two pull requests against that repository with some changes that seem to be working since codeclimate.com builds are showing no warnings on https://codeclimate.com.

The second pr is for checking @chrisbutler latest comment, again I seem to have no issue with that one neither. The problem that you were facing might have been resolve, both by an update on the typescrippt dependency that we have recently done and by setting typescript as one of the plugins.

I have also tried running codeclimate locally using the CLI, and again no issues.

Please, let me know if there is something that I have missunderstood.

First pr with codeclimate build

Second PR with codeclimate build

chrisbutler commented 3 years ago

hi @fede-moya, really appreciate you digging into this and getting back to us

i think the "no issues" result is a little misleading. it appears that the config files still aren't being parsed/used... i can see the following output on the build that you linked:

No rules are configured. Make sure you have added a config file with rules enabled.
See our documentation at https://docs.codeclimate.com/docs/eslint for more information.
No rules are configured. Make sure you have added a config file with rules enabled.
See our documentation at https://docs.codeclimate.com/docs/eslint for more information.
Warning: React version was set to "detect" in eslint-plugin-react settings, but the "react" package is not installed. Assuming latest React version for linting.
WARN: Skipping express/src/components/logo/word.tsx: it appears to be minified
Ignoring the following rules that rely on module resolution:
     * import/extensionsimport/named
     * import/namespace
     * import/no-absolute-path
     * import/no-cycle
     * import/no-duplicates
     * import/no-extraneous-dependencies
     * import/no-named-as-default
     * import/no-named-as-default-member
     * import/no-restricted-paths
     * import/no-unresolved
     * node/no-hide-code-modules
     * node/no-missing-require
Ignoring the following settings that rely on module resolution:
     * undefined
Skipped modules
     * undefined

we are seeing the same message on our builds, e.g. https://codeclimate.com/repos/5873de19c12e9c57e90067ef/builds/19158

so i don't think this is fixed, since there are definitely rules defined that are not being recognized

fede-moya commented 3 years ago

@chrisbutler you are absolutely right, I have just replicated it locally 🤦🏼 . I'll continue to work on this and reply back again with an update.

chrisbutler commented 3 years ago

hey @fede-moya, any update on this?

fede-moya commented 3 years ago

@chrisbutler I'm currently working on this, I have re take the work yesterday

fede-moya commented 3 years ago

@chrisbutler could you please re check that my proposed fix, basically adding plugins: [ "@typescript-eslint" ] actually does not work. Despite seeing the No rules configure in the output of the codeclimate.com build ? We are considering that the output might be missleading or a false negative. Could you please check that ? It would be really helpful.

fede-moya commented 3 years ago

@chrisbutler the underline question in my previous post is if there are errors that show up using channel/eslint-5 but do not appear when using channel/eslint-6. Despite the No rules configured ...

The reason I'm asking this is because locally I'm getting the message saying that no rules are configured even when errors are reported, that's why I wanted to double check this.

chrisbutler commented 3 years ago

@fede-moya what we were seeing is that it falls back to some "default ruleset" if there are no rules configured... so it does still run the checks, and reports 600+ "new issues" because it's using an entirely different configuration in this case (sidenote: we can't view the details on codeclimate.com because there are "too many issues" reported for the test run)

we don't see this on channel/eslint-5 but we do on the more recent channels. does that help?

fede-moya commented 3 years ago

@chrisbutler yes, thanks 👍🏼. I'll keep you updated.

fede-moya commented 3 years ago

@chrisbutler @nick-invision could you please try modifying your .codeclimate.yml configuration to not ignore warnings ? Like this -->

plugins:
  eslint:
    config:
      ignore_warnings: false

I couldn't find the root of the problem yet, but changing this configuration has worked for me.

fede-moya commented 3 years ago

@chrisbutler @nick-invision Folks, had someone tried my last suggestion ? 🕵🏼 https://github.com/codeclimate/codeclimate-eslint/issues/472#issuecomment-726161556

nick-invision commented 3 years ago

I ended up using a different code coverage service so I can't retest this.

fede-moya commented 3 years ago

@nick-invision Well, thanks for replying.

chrisbutler commented 3 years ago

@nick-invision which service did you end up going with?

chrisbutler commented 3 years ago

@fede-moya adding the ignore_warnings: false to the config doesn't seem to have made any difference, i still see the No rules are configured message under the eslint section in this build: https://codeclimate.com/repos/5873de19c12e9c57e90067ef/builds/19560

but as i mentioned above, if i remove the extends: field from my .eslintrc.js file then i don't see that message and it appears to be using the defined rules

so it seems pretty clear that the issue is related to extends, and it happens with the eslint-6 and eslint-7 channels, but not eslint-5

chrisbutler commented 3 years ago

@fede-moya looking at the reproduction supplied by nick-invision, it seems like this might not be caused by extends itself but rather by Codeclimate's handling of the extends plugins

i say this because ESLint itself runs fine with extends: ['plugin:@typescript-eslint/recommended'], but running ESLint via Codeclimate with that plugin is somehow broken... but i just tested extends: ['plugin:react/recommended'] and i'm not seeing the No rules are configured message

FYI, the source for the @typescript-eslint plugins is here: https://github.com/typescript-eslint/typescript-eslint/tree/master/packages/eslint-plugin/src/configs. since those work fine with ESLint itself, i think you folks should be looking at how Codeclimate runs ESLint to figure out why it is only broken on your platform (and not in the ESLint package itself, or whatever platform nick-invision is now using)

fede-moya commented 3 years ago

Okay @chrisbutler , thanks for the context.

chrisbutler commented 3 years ago

@nick-invision sorry to bug you, but would you mind sharing which coverage service you're using now (assuming it supports ESLint with extends and overrides)?

fede-moya commented 3 years ago

Hi @chrisbutler , apologizes for the long wait. Could you check if the issue is still around ? We have updated channels 6 and 7 with a fix for this problem. Could you check and confirm that the solution works on your end ?. Thank you.

chrisbutler commented 3 years ago

@fede-moya thanks for your efforts on this, it looks like everything is working correctly now!

codeclimate / codeclimate-eslint