search

gatsbyjs / gatsby

The best React-based framework with performance, scalability and security built in.
https://www.gatsbyjs.com
MIT License
55.21k stars 10.33k forks source link

"invariant violation: inconsistent node counters detected" when upgrading to 4.8.0 #34910

Closed luciobenini closed 2 years ago

luciobenini commented 2 years ago

Preliminary Checks

Description

When build in 4.8.0 process fails returning the attached error.

The error comes in queries with elemMatch filter. Everything normal in develop.

Thanks for any help.

Reproduction Link

https://github.com/pittica/site

Steps to Reproduce

  1. yarn install
  2. gatsby build

Expected Result

Expected: everything works fine.

Actual Result

ERROR #85901 GRAPHQL

There was an error in your GraphQL query:

Invariant violation: inconsistent node counters detected

1 | query CategoriesListTemplate($slug: String!, $limit: Int!, $skip: Int!, $locale: GraphCMS_Locale!, $stage: GraphCMS_Stage!) {

2 | posts: allGraphCmsPost( | ^ 3 | limit: $limit 4 | skip: $skip 5 | filter: {categories: {elemMatch: {slug: {eq: $slug}}}, stage: {eq: $stage}, locale: {eq: $locale}} 6 | sort: {fields: date, order: DESC} 7 | ) { 8 | nodes { 9 | id 10 | image { 11 | localFile { 12 | childImageSharp {

File path: D:/GitHub/pittica/site/src/templates/list/categories.jsx Url path: /categories/company Plugin: none

Error: Invariant violation: inconsistent node counters detected

An error occurred during parallel query running. Go here for troubleshooting tips: https://gatsby.dev/pqr-feedback

Error: Worker exited before finishing task

not finished run queries in workers - 2.029s

Environment

System:
    OS: Windows 10 10.0.22000
    CPU: (8) x64 Intel(R) Xeon(R) W-2225 CPU @ 4.10GHz
  Binaries:
    Node: 16.13.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.17 - ~\AppData\Roaming\npm\yarn.CMD
    npm: 8.1.1 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Chrome: 98.0.4758.102
    Edge: Spartan (44.22000.120.0), Chromium (98.0.1108.50)
  npmPackages:
    gatsby: ^4.8.0 => 4.8.0
    gatsby-cli: ^4.8.0 => 4.8.0
    gatsby-plugin-feed: ^4.8.0 => 4.8.0
    gatsby-plugin-gdpr-cookies: ^2.0.8 => 2.0.8
    gatsby-plugin-image: ^2.8.0 => 2.8.0
    gatsby-plugin-manifest: ^4.8.0 => 4.8.0
    gatsby-plugin-offline: ^5.8.0 => 5.8.0
    gatsby-plugin-preconnect: ^1.2.1 => 1.3.0
    gatsby-plugin-react-helmet: ^5.8.0 => 5.8.0
    gatsby-plugin-robots-txt: ^1.7.0 => 1.7.0
    gatsby-plugin-sass: ^5.8.0 => 5.8.0
    gatsby-plugin-sharp: ^4.8.0 => 4.8.0
    gatsby-plugin-sitemap: ^5.8.0 => 5.8.0
    gatsby-source-filesystem: ^4.8.0 => 4.8.0
    gatsby-source-graphcms: ^2.8.0 => 2.8.0
    gatsby-transformer-sharp: ^4.8.0 => 4.8.0
  npmGlobalPackages:
    gatsby-cli: 4.8.0

Config Flags

No response

Undistraction commented 2 years ago

Seeing the same issue here. Seems to be related to use of sort.

Previously, this worked fine (until at least 4.5.0), 

graphql(`
    {
      articles: allMarkdownRemark(
        sort: { fields: fields___publishedDate, order: DESC }
        filter: { 
          fields: { 
            type: { eq: "article" },
            isPublished: { eq: true }
          } 
        }
      ) {
        edges {
          node {
            fields {
              slug
              template
            }
          }
        }
      }
    }
  `)

Now it returns an error:

{
  errors: [
    Error: Invariant violation: inconsistent node counters detected
        at intersectNodesByCounter (/Users/x/node_modules/gatsby/src/datastore/in-memory/indexing.ts:1145:15)
        at applyFastFilters (/Users/x/node_modules/gatsby/src/datastore/in-memory/run-fast-filters.ts:118:30)
        at convertAndApplyFastFilters (/Users/x/node_modules/gatsby/src/datastore/in-memory/run-fast-filters.ts:382:18)
        at runFastFiltersAndSort (/Users/x/node_modules/gatsby/src/datastore/in-memory/run-fast-filters.ts:308:18)
        at Object.runQuery (/Users/x/node_modules/gatsby/src/datastore/lmdb/lmdb-datastore.ts:205:26)
        at LocalNodeModel._query (/Users/x/node_modules/gatsby/src/schema/node-model.js:342:58)
        at runMicrotasks (<anonymous>)
        at processTicksAndRejections (internal/process/task_queues.js:95:5)
        at LocalNodeModel.findAll (/Users/x/node_modules/gatsby/src/schema/node-model.js:379:36)
        at findManyPaginatedResolver (/Users/x/node_modules/gatsby/src/schema/resolvers.ts:94:20)
        at async Promise.all (index 0)
        at GraphQLRunner.query (/Users/x/node_modules/gatsby/src/query/graphql-runner.ts:220:14)
        at wrappedGraphQL (/Users/x/node_modules/gatsby/src/services/create-pages.ts:34:25) {
      locations: [Array],
      path: [Array]
    }
  ],
  data: null
}

If I remove the sort, it works:

graphql(`
    {
      articles: allMarkdownRemark(
        filter: { 
          fields: { 
            type: { eq: "article" },
            isPublished: { eq: true }
          } 
        }
      ) {
        edges {
          node {
            fields {
              slug
              template
            }
          }
        }
      }
    }
  `)
Undistraction commented 2 years ago

@luciobenini do you still get an error if you remove all the sorts from your queries?

LekoArts commented 2 years ago

Hi!

Sorry to hear you're running into an issue. To help us best begin debugging the underlying cause, it is incredibly helpful if you're able to create a minimal reproduction. This is a simplified example of the issue that makes it clear and obvious what the issue is and how we can begin to debug it.

If you're up for it, we'd very much appreciate if you could provide a minimal reproduction and we'll be able to take another look.

Thanks for using Gatsby! 💜

Undistraction commented 2 years ago

@LekoArts he's included a reproduction.

LekoArts commented 2 years ago

It requires environment variables/secrets to run the site that we don't have :/ Plus it's not a minimal reproduction but the full blown site (that we discourage to share because it makes everything way more complicated)

Undistraction commented 2 years ago

@LekoArts Apologies. Should have looked closer.

LekoArts commented 2 years ago

No worries :)

wahidshafique commented 2 years ago

I'm experiencing the same issue and indeed, when I remove all my filter options, the error goes way (or atleast, its an error I can fix on my own). Here is the top of my original problematic query

allProfile: allAirtable(
        filter: { table: { eq: "${
          constants.tableNames.profile
        }" }, data: { Type_Of_Node: {eq: "Question"} } }
        sort: {fields: data___Index, order: ASC}
      )
nonAlgebraic commented 2 years ago

Same here. Removing a sort clause from the query solves the problem. Other queries on the same site which use sort do not cause the error to be thrown, and there are not obvious functional differences between them.

I will spend some time creating a minimal repro as well, but for now just thought I'd add a +1. Thanks to the maintainers and contributors for looking into this!

Roundups commented 2 years ago

I can build locally but receive this error when building on Netlify, not sure if helpful. I can't remove the sort filter as this will be applied to my production site and mess up the UI.

EDIT I did have this error whilst developing locally before working to deploy to Netlify. I was able to resolve it eventually by updating all packages, including all related gatsby plugins to *.8.0. Then removing node_modules and package-lock.json, doing a fresh npm install, running gatsby clean and gatsby develop then finally gatsby build. It took some iterations of these commands for it to work so I can't pinpoint the moment it suddenly did work. I am using node 16.14.0 and npm 8.3.1.

As a work around to successfully deploy to production, I used the netlify-cli, version 9.8.0. I did a gatsby build and then a netlify deploy. In case anyone else finds their way here with the same issue.

luciobenini commented 2 years ago

fields___

Thanks for your reply and the support.

Yes. The error is in 

filter: {
        categories: { elemMatch: { slug: { eq: $slug } } }
        stage: { eq: $stage }
        locale: { eq: $locale }
      }

If I remove categories: { elemMatch: { slug: { eq: $slug } } } works fine. But I can't remove it. Sort isn't a problem.

I can't share the building of graphcms. Other env variables are not critical for building.

Edit

I tried to delete node_modules and yarn.lock. The reinstall packages, then gatsby clean and now categories works... but stage not :| It's better than categories!

gerler commented 2 years ago

4.7 no problems, but with 4.8 had same errors when allFile( filter multiple args sort no problems,

  1. pageQuery query ($id: String!,$imgDir: String!) { pageContentQueried: allFile( filter: {id: {eq: $id}, sourceInstanceName: {eq: "pagesMdx"}} ) at the end rewritten to query ($id: String!){ pageContentQueried: mdx ( id: {eq: $id} )

  2. useStaticQuery allFile(... fail: filter: { extension: {eq: "mp4"}, sourceInstanceName: { eq: "dirStatic" }, relativeDirectory: { eq: "vba-finst" }} ok: filter: { extension: {regex: "/mp4/"}, sourceInstanceName: { eq: "dirStatic" }, relativeDirectory: { eq: "vba-finst" }}

** I apologize, this is tagged build, I got errors already with gatsby develop

imjoshin commented 2 years ago

Hello everyone! Sorry to hear you're seeing issues here. We haven't been able to reproduce it yet, but are looking into it. If someone has a minimal reproduction, that would help us out a ton! 😄

For anyone new coming into this issue, the workaround is to pin to 4.7 while we investigate this further.

roryclaasen commented 2 years ago

Hi all,

I've been experiencing this same issue on 4.8. Only on page builds and never in the query editor (http://localhost:8000/__graphql). Like @Undistraction mentioned earlier, by removing the sort section from the query I was able to solve one instance of the error. However I was still experiencing the same issue on another query with with no sort but a parameter input.

With the (just released) 4.9 update I am now no longer hitting this bug. I'm not sure what exactly was the cause so I'm just going to say it was "magic". I was also in the process of trying to create a minimal repro but couldn't get it to error unfortunately.

wahidshafique commented 2 years ago

@roryclaasen Interesting, but I am still getting the bug despite having upgraded. I really want to believe in magic. I did not manage to recreate it either, the closest I got was finding a public repo that matched my setup, but it happened to work https://codesandbox.io/s/9u6l4h I guess I'd need to understand where the issue is on my own setup. This project is there in case anyone wants to fork/break

gavlyin commented 2 years ago

I am facing the same issue. Seems like all kinds of sort don't work at all. Otherwise it works fine on GraphiQL.

Error: Invariant violation: inconsistent node counters detected

  - indexing.ts:1145 intersectNodesByCounter
    [bubsdev]/[gatsby]/src/datastore/in-memory/indexing.ts:1145:15

  - run-fast-filters.ts:118 applyFastFilters
    [bubsdev]/[gatsby]/src/datastore/in-memory/run-fast-filters.ts:118:30

  - run-fast-filters.ts:382 convertAndApplyFastFilters
    [bubsdev]/[gatsby]/src/datastore/in-memory/run-fast-filters.ts:382:18

  - run-fast-filters.ts:308 runFastFiltersAndSort
    [bubsdev]/[gatsby]/src/datastore/in-memory/run-fast-filters.ts:308:18

  - lmdb-datastore.ts:205 Object.runQuery
    [bubsdev]/[gatsby]/src/datastore/lmdb/lmdb-datastore.ts:205:26

  - node-model.js:342 LocalNodeModel._query
    [bubsdev]/[gatsby]/src/schema/node-model.js:342:58

  - task_queues.js:95 processTicksAndRejections
    internal/process/task_queues.js:95:5

  - node-model.js:379 LocalNodeModel.findAll
    [bubsdev]/[gatsby]/src/schema/node-model.js:379:36

  - resolvers.ts:94 findManyPaginatedResolver
    [bubsdev]/[gatsby]/src/schema/resolvers.ts:94:20

  - async Promise.all

  - graphql-runner.ts:220 GraphQLRunner.query
    [bubsdev]/[gatsby]/src/query/graphql-runner.ts:220:14

  - query-runner.ts:135 queryRunner
    [bubsdev]/[gatsby]/src/query/query-runner.ts:135:14
pieh commented 2 years ago

Finally produced a sample site that can reproduce this problem (via series of queries in gatsby-node) in https://github.com/pieh/i34910.

What seems so far it's not related to single particular queries and rather to series of queries (execution order is important) where gatsby internal filter caches might lead to those errors. In a way I could reproduce is that there is a query that have some filter and then second query have same filter as previous - one more filter (that wasn't seen before) and sort is aded on second query.

For now I opened https://github.com/gatsbyjs/gatsby/pull/35020 with failing test replicating reported error, actual fix for it is TBD.

LekoArts commented 2 years ago

Should be fixed in gatsby@4.9.1

MikeBroders commented 2 years ago

hey, yarn still gives 4.9.0 is there a way to force latest version?

thx for your effort

gavlyin commented 2 years ago

hey, yarn still gives 4.9.0 is there a way to force latest version?

thx for your effort

@MikeBroders Have you tried this?

yarn add gatsby@latest

Or you can manually update your package.json file.

...
    "gatsby": "^4.9.1",
...

And run yarn install.

MikeBroders commented 2 years ago

hmm... no luck

yarn is just showing me this for selection on yarn install with "gatsby": "^4.9.1" in my packagejson


  4.10.0-next.1 
  4.10.0-next.0 
  4.10.0-alpha-segfault-handler.12 
  4.10.0-alpha-ledger.8 
  4.10.0-alpha-ledger.7 
  4.9.0 
  4.9.0-next.1 
  4.9.0-next.0 ```

update

4.9.1 now available for yarn
artt commented 2 years ago

Seems like 4.9.1 gives another problem of sorting not working. Still investigating, but thought it's worth noting for those who hasn't updated yet.

Add'l info

I clean built on 4.9.1 and got a problem. Either

gatsby develop on 4.9.1 presented no problem.

I don't have time right now to create a reproduction though. Just in case this is helpful.

MikeBroders commented 2 years ago

update

4.9.1 fixes the particular issue with same sort on differnt queries and different filters in one graphql in my case

deleted node_modules

did yarn install with "gatsby": "^4.9.1" in my package.json

michaelampr commented 2 years ago

Ran into this issue yesterday. Gatsby version: 4.13.1. 4 incremental builds failed until we ran npm run clean and did a full build again.

Occurred in:

ERROR #85901  GRAPHQL

There was an error in your GraphQL query:

Invariant violation: inconsistent node counters detected

   1 | query {
>  2 |   allPost(
     |   ^
   3 |     limit: 3
   4 |     sort: {fields: published_on, order: DESC}
   5 |     filter: {featured_url: {ne: null}, el_id: {eq: null}}
   6 |   ) {
mihaisavezi commented 1 year ago

Happened on gatsby@4.24.2. running gatsby clean fixes the issue.