search

gatsbyjs / gatsby

The best React-based framework with performance, scalability and security built in.
https://www.gatsbyjs.com
MIT License
55.27k stars 10.3k forks source link

Jest fails with `Couldn't find temp query result` error #35018

Closed ascorbic closed 2 years ago

ascorbic commented 2 years ago

Preliminary Checks

Description

As part of a site's unit tests, it may need to compile the site. Previously this was simple to do by running the cli via child_process or execa. This works until gatsby@4.9.0-next.0, but since gatsby@4.9.0-next.1 this fails. The site builds correctly if build manually from the CLI.

Reproduction Link

https://github.com/ascorbic/gatsby-jest-bug

Steps to Reproduce

  1. Clone the repo
  2. Run yarn install
  3. Run yarn test

Expected Result

Site builds and test passes

Actual Result

Site build fails with the following error:

success Rewriting compilation hashes - 0.001s

     ERROR

    UNHANDLED REJECTION Couldn't find temp query result for "/404/".

      Error: Couldn't find temp query result for "/404/".

      - page-data.ts:104 readPageQueryResult
        [jest-demo]/[gatsby]/src/utils/page-data.ts:104:11

      - page-data.ts:121 writePageData
        [jest-demo]/[gatsby]/src/utils/page-data.ts:121:18

      - page-data.ts:232
        [jest-demo]/[gatsby]/src/utils/page-data.ts:232:24

Environment

System:
    OS: macOS 11.6.4
    CPU: (8) arm64 Apple M1
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.11.0 - /var/folders/k2/ffw3_41j7h17rzsmx25wcwl80000gn/T/yarn--1646216980464-0.9843919045068601/node
    Yarn: 1.22.17 - /var/folders/k2/ffw3_41j7h17rzsmx25wcwl80000gn/T/yarn--1646216980464-0.9843919045068601/yarn
    npm: 8.3.0 - ~/.nvm/versions/node/v16.11.0/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 98.0.4758.102
    Safari: 14.1.2
  npmPackages:
    gatsby: ^4.9.0 => 4.9.0

Config Flags

No response

JeffersonCarvalh0 commented 2 years ago

I'm also experiencing this problem with gatsby@4.8.0

Nowaker commented 2 years ago

Same for me.

es-createPages: 39.095s
success createPages - 89.772s
success createPagesStatefully - 0.051s
info Total nodes: 6030, SitePage nodes: 1642 (use --verbose for breakdown)
verbose 99.876148055 Number of node types: 10. Nodes per type: Directory: 750, File: 1801, JekyllConfigBundled: 1, JekyllData: 647, JekyllDataBundled: 1, JekyllHtml: 1154, Site: 1, SiteBuildMetadata: 1, SitePage: 1642, SitePlugin: 32
verbose 99.876242955 Checking for deleted pages
verbose 99.877802396 Deleted 0 pages
verbose 99.878584229 Found 1642 changed pages
success Checking for changed pages - 0.001s
success onPreExtractQueries - 0.000s
success extract queries from components - 1.687s
success write out redirect data - 0.004s
success onPostBootstrap - 0.001s
info bootstrap finished - 101.578s
success write out requires - 0.015s
success Building production JavaScript and CSS bundles - 5.234s
success Building HTML renderer - 9.245s
success Execute page configs - 0.019s
success Caching Webpack compilations - 0.001s
success run queries in workers - 2.095s - 1642/1642 783.85/s
success Merge worker state - 0.007s
success Rewriting compilation hashes - 0.001s
error UNHANDLED REJECTION Couldn't find temp query result for "/es/thank-you/index.html".
  Error: Couldn't find temp query result for "/es/thank-you/index.html".

  - page-data.ts:104 readPageQueryResult
    [_gatsby]/[gatsby]/src/utils/page-data.ts:104:11

  - runMicrotasks

  - task_queues:61 runNextTicks
    node:internal/process/task_queues:61:5

  - timers:437 processImmediate
    node:internal/timers:437:9

  - page-data.ts:121 writePageData
    [_gatsby]/[gatsby]/src/utils/page-data.ts:121:18

  - page-data.ts:232 
    [_gatsby]/[gatsby]/src/utils/page-data.ts:232:24

not finished Writing page-data.json files to public directory - 4.315s

Env: Linux (CI), Gatsby 4.7.2.

Note: it started happening recently. I haven't seen this error before. No changes were done to the site in between. CI just started encountering this problem out of the blue moon. Trying yarn install --frozen-lockfile now to see if that fixes the problem. In which case a diff between by locked dep resolution and most recent dep resolution will be the issue. Otherwise... who knows.

Nowaker commented 2 years ago

yarn install --frozen-lockfile helped.

marciolima-praycom commented 2 years ago

I'm experiencing the same issue with gatsby: ^4.11.0.

Running yarn install --frozen-lockfile doesn't help.. 😔

ZeldOcarina commented 2 years ago

Same issue here! :( I am in 4.14.0

Oakwhisper commented 2 years ago

Creating a new blank gatsby project builds locally on MacOS Monterrey (ARM & x64) but fails on jenkins server build (Debian 11). Error is:

success Merge worker state - 0.003s
success Rewriting compilation hashes - 0.062s
error UNHANDLED REJECTION Couldn't find temp query result for "/404/".


  Error: Couldn't find temp query result for "/404/".
  
  - page-data.ts:104 readPageQueryResult
    [gatsby-test-project]/[gatsby]/src/utils/page-data.ts:104:11
  
  - page-data.ts:121 writePageData
    [gatsby-test-project]/[gatsby]/src/utils/page-data.ts:121:18
  
  - page-data.ts:232 
    [gatsby-test-project]/[gatsby]/src/utils/page-data.ts:232:24
  

not finished Writing page-data.json files to public directory - 0.408s

Dependencies for brand new gatsby project: 

  "dependencies": {
    "@emotion/react": "^11.9.0",
    "@emotion/styled": "^11.8.1",
    "gatsby": "^4.14.1",
    "gatsby-plugin-emotion": "^7.14.0",
    "gatsby-plugin-manifest": "^4.14.0",
    "react": "^17.0.1",
    "react-dom": "^17.0.1"
  }
enriqueArb commented 2 years ago

I had this issue too with v4.12.1. For me, the solution was to run gastby clean. After that, build ran correctly.

Hope it helps someone.

belfz commented 2 years ago

Same on gatsby v4.15.2. Neither gatsby clean nor yarn install --frozen-lockfile helped. The error also feels very random, each time mentioning other page within the Couldn't find temp query result for (...) error message.

Precisely I'm getting

 ERROR 

UNHANDLED REJECTION Couldn't find temp query result for "/learn-more/en-us".

  Error: Couldn't find temp query result for "/learn-more/en-us".

  - page-data.ts:104 readPageQueryResult
    [www-2022]/[gatsby]/src/utils/page-data.ts:104:11

  - task_queues:60 runNextTicks
    node:internal/process/task_queues:60:5

  - timers:442 processImmediate
    node:internal/timers:442:9

  - page-data.ts:121 writePageData
    [www-2022]/[gatsby]/src/utils/page-data.ts:121:18

  - page-data.ts:232 
    [www-2022]/[gatsby]/src/utils/page-data.ts:232:24

upon running gatsby build.

learn-more is created using createPage API, precisely:

actions.createPage({
        path: `/learn-more/en-us`,
        component: path.resolve(`src/non-cms-pages/learn-more.tsx`),
        context: {
          lang,
        },
      });
belfz commented 2 years ago

@LekoArts here's a URL to the faulty project (minimal reproduction): https://github.com/belfz/gatsby-bug-reproduce

LekoArts commented 2 years ago

@belfz Your reproduction shows the problem because you're using the createPage action in the wrong lifecycle hook. It's only allowed in createPages while you use it in onCreateWebpackConfig.

@ascorbic The PR https://github.com/gatsbyjs/gatsby/pull/35968 should fix this, wasn't easy to spot this 😅

ascorbic commented 2 years ago

Ooh, good find! Sneaky one

Nowaker commented 2 years ago

I wouldn't consider it resolved. My use case doesn't involve executing createPage in onCreateWebpackConfig.

LekoArts commented 2 years ago

If you can provide a minimal reproduction we'd be able to look at it

belfz commented 2 years ago

big thank you @LekoArts, indeed that was it - I didn't spot that I was using a wrong lifecycle hook 😅

beamercola commented 2 years ago

@Nowaker Agreed - I'm getting this intermittently on netlify deploys and I'm not doing anything funny with createPage. Did you resolve your problem?

Nowaker commented 2 years ago

@beamercola Not really. Going back to some older lockfile did resolve most of it, but not all. It happens every once in a while.

quanglam2807 commented 2 years ago

We're facing the same issue when running build on Netlify. It only happens sometimes.

zolon4 commented 2 years ago

I'm getting the same issue on netlify when running build. Sometimes clearing the netlify cache fixes but not always.

enormousrodent commented 2 years ago

I am also getting netlify deploy issues during build. I have to keep re-building then it will eventually work! really annoying and every time its a different page it complains about!

jlevy-io commented 2 years ago

Same here, this has been an issue on Netlify since August 1st

(@netlify/build 27.8.1 is when we first started seeing it happen)

natapg commented 2 years ago

same here

marciolima-praycom commented 2 years ago

In my case I have a monorepo with 3 Gatsby apps and I was trying to build them on Circle CI using Lerna or Yarn Workspaces to call the clean & build scripts in all of them at once (either sequentially or in parallel). I was seeing that issue happening randomly for different pages.

I was able to fix it by creating a build.sh file that calls yarn clean && yarn build on each app separately. The Circle CI job now calls yarn build on my monorepo, and the build script calls the build.sh file like this:

"scripts": {
  "build": "yarn exec ./build.sh"
},

Not sure why, but after we moved to this approach we never saw that error happening again.

natapg commented 2 years ago

I run yarn clean && yarn buildon every run due to another problem, and I still started having this issue a few days ago

mitchossi commented 2 years ago

Same problem here,gatsby clean didn't help...

NeversSync commented 2 years ago

Having the same issue as well, very intermittent and frustrating

coreyward commented 2 years ago

Another +1 here. Sometimes also seeing PQR related errors—could be related, or a separate issue.

ZeldOcarina commented 2 years ago

Another +1 here. Sometimes also seeing PQR related errors—could be related, or a separate issue.

Hey what is "pqr"?

coreyward commented 2 years ago

Parallel query running, which was enabled for everyone in v4. I get an error pointing to https://gatsby.dev/pqr-feedback

FcjubtcWAAE8bxv

There's also this issue in an upstream dependency that is related to that failure: https://github.com/kriszyp/lmdb-js/issues/153#issue-1194142427

ZeldOcarina commented 2 years ago

Oh yeah it's such a shame that it cannot be deactivated anymore.. It often causes troubles and the only opt-out way is to downgrade to v3 which is not optimal..

coreyward commented 2 years ago

@ZeldOcarina +1000000

@LekoArts Is calling createPage in onCreatePage still okay? I'm using it like the docs show here.

coreyward commented 2 years ago

I just ran into this in gatsby develop mode too. Using 4.21.0.

LekoArts commented 2 years ago

I won't reopen this as the original issue (with the reproduction) was fixed, if you still encounter this, open a new issue with a minimal reproduction. Ideally it has to show locally as we can't give support for Netlify-only builds (reach out to Netlify's support for that).

Nowaker commented 2 years ago

It had nothing to do for me with Netlify. Happening to me in build. To others in develop. This is a random bug in Gatsby - and it happens so ramdomly that there's no reproduction, let alone a minimal one.

jasonlav commented 2 years ago

I experienced this issue when building pages using nested methods. Removing the nesting appears to have resolved the issue.

Example of what caused the issue 

export.createPages = async (args) => {
    await createPageTypeA(args);
};

async function createPageTypeA(args) {
    await createPageInUniversalSharedMethod(args);
}

async function createPageInUniversalSharedMethod({ createPage }) {
   ....
   createPage(...);
    ...
}

Example of what resolved the issue 

export.createPages = async (args) => {
    await createPageInUniversalSharedMethod(args);
};

async function createPageInUniversalSharedMethod({ createPage }) {
   ...
   createPage(...);
   ...
}
coreyward commented 2 years ago

@jasonlav Interesting. Any chance it's a fluke that your issue is fixed?

@LekoArts @graysonhicks Is Gatsby doing any kind of static analysis on gatsby-node that might cause this (i.e., is this worth chasing as a lead)?

jasonlav commented 2 years ago

Interesting. Any chance it's a fluke that your issue is fixed?

Absolutely could be. However, I have not encountered the issue since that change.

cesarvarela commented 2 years ago

it has nothing to do with Jest IMO, we started getting this error after upgrading gatsby

joebentaylor1995 commented 2 years ago

Same issue, not using jest or even creating pages yet

aaadamgo commented 2 years ago

This is still a problem on @4.24.1

I've remove node modules, gatsby clean, etc and try to build locally and get the error UNHANDLED REJECTION Couldn't find temp query result for "/404/".

I was having issues on @4.12.1 but then upgraded to the above to see if a fix was in place.

My 404 page has nothing on it.

Tried removing the 404, it then pointed to another page, same error, different page.

Sorry this is a bit of a 'live' update as I'm working through it.

I've stepped backwards through the gatsby versions, I went to 4.8.1 and the error still happens. I just tried jumping to @4.2, and it gave me a better error (turned out to be a graphql query on a completely separate page). I fixed that and now it builds perfectly fine locally.

Now circling back to @4.24.1

Error'd again. So it's between 4.2 and 4.8.

@4.6 seems to get stuck on the building pages process. I gave it 10 minutes (I only have 4 pages).

Tried all versions between @4.2 and @4.6, and only @4.2 will build the pages.

graysonhicks commented 2 years ago

Could someone please post a minimal reproduction that can be run locally?

ZeldOcarina commented 2 years ago

Unfortunately this minimal reproduction is a huge stop on handling major bugs. It's often really not possible. What about entering the existing repo? I think that is soo much easier on both parts (when possible of course)

coreyward commented 2 years ago

@graysonhicks I can't reproduce this consistently with our full app and we've been suffering from it for weeks. If the error message was able to narrow down the source it might be viable to provide a minimal reproduction, but as it stands, I don't think it's plausible outside of cases where it arrives through obvious user error.

For the record, I tried a solution similar to the one posted by @jasonlav above and I am still running into this.

Nowaker commented 2 years ago

Minimal reproduction example is a pipe dream when a problem is related to concurrency and manifests itself only in non-toy projects.

ZeldOcarina commented 2 years ago

Yeah and just so you know, this exact problem drove me to using Jenkins and dedicated servers and even local builds instead of Gatsby Cloud as it was always failing there.. and working on my local mac..

cesarvarela commented 2 years ago

Here is the error in all its glory; it happens at "random" and usually is fixed by rerunning the build with an empty cache:

https://app.netlify.com/sites/aiid/deploys/633bd0132ef6160009949592

(I know, I know, netlify, but this is the example I have at hand)

The repo: https://github.com/responsible-ai-collaborative/aiid

My spider sense tells me that it has to do with concurrency/size, and a minimal repro case is impossible.

edit: we are also considering moving to Jenkins :/

graysonhicks commented 2 years ago

@coreyward @ZeldOcarina @cesarvarela Thanks, I know it's frustrating, it's just near impossible for us to address without a reproduction locally. Do any of you have links to a build where this is happening on Gatsby Cloud? I have more tools available on that end, but I don't see any links to builds on Gatsby Cloud, or any links to reproductions that happen locally.

ZeldOcarina commented 2 years ago

I do! https://www.gatsbyjs.com/dashboard/e968afff-0cff-4103-8d02-5dbbe05c6a6f/sites/26587904-b4fa-402f-b570-2cb1bbabca80/builds/61d86c44-1010-4154-a00e-f82d681b9823/details#rawLogs If you check inside that project there are so many of these builds failing while locally these are working!

graysonhicks commented 2 years ago

Thanks @ZeldOcarina! I'll open up a ticket on the Gatsby Cloud side and take a look. Do you mind sending an email to support@gatsbyjs.com and tag me in that email and we can track it that way and report back on this thread.

graysonhicks commented 2 years ago

Actually @ZeldOcarina I don't see the Couldn't find temp query result error in any of those builds...

graysonhicks commented 2 years ago

@ZeldOcarina For the builds you sent, they are OOM'ing. Often, it OOMs and exits before the error can be written to the logs. Without looking at the code for a while, I can't say why exactly (at a glance I think adding width option to your image queries would help), but this doc is useful for improving memory consumption.

image

But I still don't see an instance of the temp query result error and I don't think memory usage is related. When I bumped that site to 8GB of memory (from 4GB) it builds successfully.