search

dimaMachina / graphql-eslint

ESLint parser, plugin and set rules for GraphQL (for schema and operations). Easily customizable with custom rules. Integrates with IDEs and modern GraphQL tools.
https://the-guild.dev/graphql/eslint
MIT License
789 stars 104 forks source link

Processor performance incredibly slow in vscode #1858

Open Jugbot opened 12 months ago

Jugbot commented 12 months ago

Issue workflow progress

Progress of the issue based on the Contributor Workflow

Describe the bug

Whenever a change is made to a file in vscode, the graphql config is reloaded which means loading possibly thousands of files, which in my case means it takes ~30sec for linting to update in my IDE.

My solution was to remove the documents part of the parserOptions and any rules that depended on it.

To Reproduce Steps to reproduce the behavior:

There is a setting to run the eslint server in vscode in debug mode. Turn that on:

  "eslint.debug": true,

Open the eslint Output console: ESLint: Show Ouput Console

In a large codebase with lots of document definitions (sorry I can't provide mine), mostly nested inside ts/tsx files, the linter will take a while to load (which is ok)

When a change is made to a file, eslint takes a long time to show with the bottleneck being the config loading

Logs:

``` @graphql-tools/load: normalizePointers ./libs/data/attentive.schema.graphql: 0.017ms @graphql-tools/load: normalizePointers ./relay-compiler-directives.graphql: 0.008ms @graphql-tools/load: normalizePointers ./client-schema-extensions/*.graphql: 0.002ms @graphql-tools/load: normalizePointers: 0.81ms @graphql-tools/load: collectDocumentString ./libs/data/attentive.schema.graphql: 0.005ms @graphql-tools/load: collectSourcesSync ./libs/data/attentive.schema.graphql: 0.027ms @graphql-tools/load: collectDocumentString ./relay-compiler-directives.graphql: 0.002ms @graphql-tools/load: collectSourcesSync ./relay-compiler-directives.graphql: 0.012ms @graphql-tools/load: collectDocumentString ./client-schema-extensions/*.graphql: 0.001ms @graphql-tools/load: collectSourcesSync ./client-schema-extensions/*.graphql: 0.01ms @graphql-tools/load: loadFileSync ./libs/data/attentive.schema.graphql: 44.734ms @graphql-tools/load: collectFallbackSync ./libs/data/attentive.schema.graphql: 44.776ms @graphql-tools/load: loadFileSync ./relay-compiler-directives.graphql: 0.443ms @graphql-tools/load: collectFallbackSync ./relay-compiler-directives.graphql: 0.458ms @graphql-tools/load: loadFileSync ./client-schema-extensions/*.graphql: 2.277ms @graphql-tools/load: collectFallbackSync ./client-schema-extensions/*.graphql: 2.428ms @graphql-tools/load: collectSourcesSync queue: 47.692ms @graphql-tools/load: parseSchema /Users/lpollice/Documents/GitHub/frontend-code/libs/data/attentive.schema.graphql: 0.003ms @graphql-tools/load: parseRawSDL /Users/lpollice/Documents/GitHub/frontend-code/libs/data/attentive.schema.graphql: 0.001ms @graphql-tools/load: useComments /Users/lpollice/Documents/GitHub/frontend-code/libs/data/attentive.schema.graphql: 230.101ms @graphql-tools/load: collectValidSources /Users/lpollice/Documents/GitHub/frontend-code/libs/data/attentive.schema.graphql: 0.004ms @graphql-tools/load: parseSource /Users/lpollice/Documents/GitHub/frontend-code/libs/data/attentive.schema.graphql: 230.334ms @graphql-tools/load: parseSchema /Users/lpollice/Documents/GitHub/frontend-code/relay-compiler-directives.graphql: 0.001ms @graphql-tools/load: parseRawSDL /Users/lpollice/Documents/GitHub/frontend-code/relay-compiler-directives.graphql: 0.001ms @graphql-tools/load: useComments /Users/lpollice/Documents/GitHub/frontend-code/relay-compiler-directives.graphql: 0.137ms @graphql-tools/load: collectValidSources /Users/lpollice/Documents/GitHub/frontend-code/relay-compiler-directives.graphql: 0.002ms @graphql-tools/load: parseSource /Users/lpollice/Documents/GitHub/frontend-code/relay-compiler-directives.graphql: 0.217ms @graphql-tools/load: parseSchema /Users/lpollice/Documents/GitHub/frontend-code/client-schema-extensions/client-ui-state.schema.graphql: 0.001ms @graphql-tools/load: parseRawSDL /Users/lpollice/Documents/GitHub/frontend-code/client-schema-extensions/client-ui-state.schema.graphql: 0.001ms @graphql-tools/load: useComments /Users/lpollice/Documents/GitHub/frontend-code/client-schema-extensions/client-ui-state.schema.graphql: 0.171ms @graphql-tools/load: collectValidSources /Users/lpollice/Documents/GitHub/frontend-code/client-schema-extensions/client-ui-state.schema.graphql: 0.002ms @graphql-tools/load: parseSource /Users/lpollice/Documents/GitHub/frontend-code/client-schema-extensions/client-ui-state.schema.graphql: 0.214ms @graphql-tools/load: prepareResult: 0.009ms @graphql-tools/load: loadTypedefsSync: 279.461ms @graphql-tools/load: normalizePointers ./apps/*/src/**/*.{graphql,ts,tsx}: 0.007ms @graphql-tools/load: normalizePointers ./{libs,mfes}/*/src/**/*.{ts,tsx}: 0.002ms @graphql-tools/load: normalizePointers !__generated__/**/*: 0.004ms @graphql-tools/load: normalizePointers: 0.073ms @graphql-tools/load: collectDocumentString ./apps/*/src/**/*.{graphql,ts,tsx}: 0.246ms @graphql-tools/load: collectSourcesSync ./apps/*/src/**/*.{graphql,ts,tsx}: 0.265ms @graphql-tools/load: collectDocumentString ./{libs,mfes}/*/src/**/*.{ts,tsx}: 0.048ms @graphql-tools/load: collectSourcesSync ./{libs,mfes}/*/src/**/*.{ts,tsx}: 0.061ms @graphql-tools/load: loadFileSync ./apps/*/src/**/*.{graphql,ts,tsx}: 1.843s @graphql-tools/load: collectFallbackSync ./apps/*/src/**/*.{graphql,ts,tsx}: 1.843s @graphql-tools/load: loadFileSync ./{libs,mfes}/*/src/**/*.{ts,tsx}: 27.955s @graphql-tools/load: collectFallbackSync ./{libs,mfes}/*/src/**/*.{ts,tsx}: 27.955s @graphql-tools/load: collectSourcesSync queue: 29.799s ```

Expected behavior

Changes to individual documents shouldn't cause an entire reload

Environment:

hedgepigdaniel commented 5 months ago

A binary search shows that this problem was introduced between 3.12.0 and 3.13.0. Going by the changelog, the only change was https://github.com/dimaMachina/graphql-eslint/pull/1222

Also, in my case the problem can be fixed similarly to https://github.com/dimaMachina/graphql-eslint/issues/1246#issuecomment-1462460474

Whether in the eslint config or in the graphqlconfig, this problem is triggered by the documents configuration value being an array, as opposed to a string/glob.

e.g. instead of 

documents: [
  "queries.ts",
  "queries.graphql",
]

use

documents: "queries.(ts|graphql)"

Apparently this prevents unnecessary busting of some cache.

hedgepigdaniel commented 5 months ago

Perhaps this specific line change is the problem? The documents array is being used as the cache key, rather than some deterministic serialized version of it: https://github.com/dimaMachina/graphql-eslint/pull/1222/files#diff-f67cfde0491aea2b91db965dff109ebcdc96e966044c069329bb5a939964d472L57

deathemperor commented 5 months ago

@hedgepigdaniel tried both 3.12.0 and latest 3.20.1, same performance issue.

I don't use graphql-config so the operations: "./src/**/*.{ts,tsx}" is what causes slowness for me. I guess it tries to extract graphql documents from lot of files which causes the slowness.