Building static HTML for pageserror: Error: ENOENT: no such file or directory

hackhat commented 5 years ago

Description

I'm getting a lot of similar errors to this:

 success Building production JavaScript and CSS bundles — 8.258 s
 ⠐ Building static HTML for pageserror ../public/29/path---internal-80-aeac-04-012-a-4-feb-9-d-99-d- 
   6053-a-12-be-5-a-r-006-91-c-794-olfyUiQvtLIAh9ozYkUiFQEB1M
 { Error: ENOENT: no such file or directory, open' X:\xxx\xxx\public\static\d\29\path---internal-80-aeac-04-012-a-4-feb-9-d-99-d-6053-a-12-be-5-a-r-006-91-c-794-olfyUiQvtLIAh9ozYkUiFQEB1M.json'

and then hangs with " Building static HTML for pages" for long time.

Steps to reproduce

Don't know... Just deleted the public folder and build it.

Environment

Run gatsby info --clipboard in your project directory and paste the output here. Not working? You may need to update your global gatsby-cli - npm install -g gatsby-cli

System: OS: Windows 10 CPU: x64 Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz Binaries: Yarn: 1.9.4 - C:\Program Files (x86)\Yarn\bin\yarn.CMD npm: 5.5.1 - C:\Program Files\nodejs\npm.CMD Browsers: Edge: 42.17134.1.0 npmPackages: gatsby: ^2.0.67 => 2.0.67 gatsby-image: ^2.0.25 => 2.0.25 gatsby-plugin-catch-links: ^2.0.9 => 2.0.9 gatsby-plugin-emotion: ^3.0.1 => 3.0.1 gatsby-plugin-google-analytics: ^2.0.8 => 2.0.8 gatsby-plugin-manifest: ^2.0.12 => 2.0.12 gatsby-plugin-offline: ^2.0.19 => 2.0.19 gatsby-plugin-react-helmet: ^3.0.0 => 3.0.0 gatsby-plugin-sharp: ^2.0.15 => 2.0.15 gatsby-plugin-sitemap: ^2.0.3 => 2.0.3 gatsby-plugin-typescript: ^2.0.3 => 2.0.3 gatsby-plugin-typography: ^2.2.2 => 2.2.2 gatsby-remark-component: ^1.1.3 => 1.1.3 gatsby-remark-images: ^3.0.1 => 3.0.1 gatsby-source-filesystem: ^2.0.12 => 2.0.12 gatsby-transformer-remark: ^2.1.15 => 2.1.15 gatsby-transformer-sharp: ^2.1.9 => 2.1.9

hackhat commented 5 years ago

Ok, so I had this issue: https://github.com/gatsbyjs/gatsby/issues/10458 I run gatsby develop a few times and now it works. Then I build and it works.. what is going on? I need to run gatsby develop before building?

hackhat commented 5 years ago

I deleted my public folder and build again, same error again.

I deleted cache folder and build and now it worked.

DSchau commented 5 years ago

Hello!

Sorry about the issues! Could you provide a reproduction or something we could take a look at? Thanks!

hackhat commented 5 years ago

I can't... is just in my project and I don't have a clue about what is affecting this. It's incredibly flaky.

DSchau commented 5 years ago

If it's something that happens every X times, e.g. every ~5 times or something, and you can reproduce it occasionally that's still more to go on that we have now.

It's pretty challenging to know how to begin debugging this. At first glance, it could be some type of Windows pathing issue, but that's a shot in the dark at this point.

Thanks!

hackhat commented 5 years ago

If I run gatsby start and then run gatsby build it fails. If I delete the cache folder and run gatsby build it works.

DSchau commented 5 years ago

That's enough for us to go on! Is it possible to share the repository (or create a minified example without any private info) so we could take a look ourselves?

hackhat commented 5 years ago

I can't right now, is pretty big and entangled... What can we do?

pieh commented 5 years ago

We do output query results to public directory directly, but we keep track of them in .cache - so if you delete public without deleting .cache - cache won't know that query results are no longer there and that's why we it complains about missing file that cache expects to be there

hackhat commented 5 years ago

@pieh Looks like it's correct. So probably we should do something to avoid this scenario. Maybe put an error that in order to build without public folder you need to delete the .cache folder as well.

But I found it strange for the .cache to look into the public folder... but anyway, that's something else.

pieh commented 5 years ago

But I found it strange for the .cache to look into the public folder... but anyway, that's something else.

So this is kind of to eliminate some of the "unnecessary" work - like if we output query results to .cache we would need to copy them later on to public anyway. But for sure it comes with issues like that. We definitely could/should check if query results are available and rerun queries if they are not

jonniebigodes commented 5 years ago

@hackhat this a bit of a edge case, normally people don't even touch these folders, or if they do they delete both of them, either by using a script, or using an auxiliary package to handle that situation. And controlling the scenario you mentioned could be a bit tricky, as both folders are automatically generated, if not present they are generated and their content filled by gatsby and if they are gatsby goes about it's business, starts incrementally making the checks needed and updating the contents incrementally. And if everything is working properly you get an ok, or like it happened to you it throws an error. I believe that this should be a case to be handled by the actual developer, not the tool itself. Before issuing production build and/or issuing the development command you should do some "house cleaning" (pardon the bad pun) beforehand and always getting the correct/actual data/contents you have. And i might be completely off on this, and feel free to correct me, but it's a defensive strategy i adopted that's been working for me since i've discovered gatsby in the past year.

sidharthachatterjee commented 5 years ago

We can maybe add a gatsby clean to the cli that deletes both the folders but generally, @jonniebigodes makes a good point that these folders ought not to be touched since they are owned by gatsby .

Apologies for the not very helpful error messages. We are working towards improving this.

hackhat commented 5 years ago

@sidharthachatterjee Ok, cool. Thanks.

@pieh I'm not sure how would that work when you build for dev vs build for prod. Looks like the prod build uses dev assets. I don't feel very safe about it, but looks like has no problems for now.

@jonniebigodes I think the code should do its best to avoid developer mistakes. For example rememering to delete .cache before you build is an antipattern. Why not delete the .cache folder when you run the build automatically? You will do one mistake at one point.

gatsbot[bot] commented 5 years ago

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

gatsbot[bot] commented 5 years ago

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

Thanks again for being part of the Gatsby community!

lionelbarrow commented 5 years ago

I just ran into this as well.

For future people: the tl; dr is that when making a fresh build, delete both public and .cache.

DSchau commented 5 years ago

@lionelbarrow could you share some context as to why you deleted the public folder to begin with?

We're trying to track down caching issues in #11747, so any insight you have would be invaluable--particularly if it's reproducible! Thanks!

sidharthachatterjee commented 5 years ago

Also side note: You can also use gatsby clean which will delete both for you. Released in gatsby-cli@2.4.11

andrecalvo commented 5 years ago

Not sure if it's related but I found this issue ^ while googling a solution...

I am running npm run serve after gatsby clean however I get this error when trying to view localhost:9000.

Error: ENOENT: no such file or directory, stat '/xxx/xxx/gatsby-site/public/404.html'

the-main-thing commented 5 years ago

Not sure if it's related but I found this issue ^ while googling a solution...

I am running npm run serve after gatsby clean however I get this error when trying to view localhost:9000.

Error: ENOENT: no such file or directory, stat '/xxx/xxx/gatsby-site/public/404.html'

Same here

the-main-thing commented 5 years ago

But gatsby develop works well.

the-main-thing commented 5 years ago

After executing npm run build gatsby serve started to working properly. What I did:

npm install - warnings
gatsby serve - error
delete public and .cache
gatsby serve - error
gatsby develop - ok
npm run build - ok
gatsby serve - ok

danielpuehringer commented 5 years ago

If I run gatsby start and then run gatsby build it fails. If I delete the cache folder and run gatsby build it works.

Hi, I had the same Error. It occured after successfully building the project. 5min later when I was trying to execute gatsby develop I ran into the same error message "Error: ENOENT: no such file or directory, open 'C:...\public\static\d\655\path---index-6a9-M35H41xhcRuRW1Ovpq3cs1Xmo.json'"

So gatsby develop fails with this exception and gatsby build does not finish at all because "Building static HTML for pages" does not stop.

Occurring Error: Error: ENOENT: no such file or directory, open 'C:\...\public\static\d\655\path---index-6a9-M35H41xhcRuRW1Ovpq3cs1Xmo.json' Solution: by deleting all the files in the .cache folder I was able to fix those errors :)

smakosh commented 5 years ago

Having the same issue here even after deleting both the public and .cache folders, I even removed the node_modules and re-installed them

alexmcode commented 5 years ago

Hi. Having the same error. Also I get this output every time I run gatsby serve:

error Building static HTML failed

See our docs page on debugging HTML builds for help https://gatsby.dev/debug-html

  85 | ]);
  86 |
> 87 | proxyRequestMethods(Index, '_index', IDBIndex, [
     | ^
  88 |   'get',
  89 |   'getKey',
  90 |   'getAll',

  WebpackError: ReferenceError: IDBIndex is not defined

alexpreiss commented 5 years ago

Hi. Having the same error. Also I get this output every time I run gatsby serve:

error Building static HTML failed

See our docs page on debugging HTML builds for help https://gatsby.dev/debug-html

  85 | ]);
  86 |
> 87 | proxyRequestMethods(Index, '_index', IDBIndex, [
     | ^
  88 |   'get',
  89 |   'getKey',
  90 |   'getAll',

  WebpackError: ReferenceError: IDBIndex is not defined

Are you trying to use Firebase?

barbalex commented 5 years ago

For a frontline story of how the public folder can grow rather large and how that can bring down a project surprisingly see this issue in the zeit community: https://spectrum.chat/zeit/general/help-i-just-hit-a-deployment-limit-during-a-critical-deployment~6c7c0fb2-4822-4a54-9a20-fd9abb3d1f21

barbalex commented 5 years ago

Well, maybe you have to log in for that community. Here the long story short:

for a large update I had to synchronize SQL-changes and app deployment
I built the app, applied the sql changes then deployed on zeit/now as hundreds of times before
unfortunately the deployment hit a 10'000 file limit
my app was broken, it was 23.00 at night after a hard days work and I had to get things working again
I thought: maybe the public folder contains unneeded stuff. Emptied it. Tried to build and hit the errors mentioned in this issue
I found this issue, deleted public folder and .cache folder, rebuilt - and found only 600 files in the public folder

Minus 9'400 files!

After this deployment worked again :-)

my app was broken, it was 23.00 at night after a hard days work

Reading this after writing it makes me realize just how bad I had timed this :-|

Brydom commented 5 years ago

I was running into this issue as well trying to use Firebase. Ended up following this guide which 'lazy' imports Firebase on componentDidMount.

There's also this starter template if anyone else was trying to do this: https://github.com/muhajirdev/gatsby-starter-firebase

kostimarko commented 5 years ago

Hi. Having the same error. Also I get this output every time I run gatsby serve:

error Building static HTML failed

See our docs page on debugging HTML builds for help https://gatsby.dev/debug-html

  85 | ]);
  86 |
> 87 | proxyRequestMethods(Index, '_index', IDBIndex, [
     | ^
  88 |   'get',
  89 |   'getKey',
  90 |   'getAll',

  WebpackError: ReferenceError: IDBIndex is not defined

Are you trying to use Firebase?

I'm getting the same issue and I'm using firebase

epgui commented 5 years ago

Our production build will always fail exactly this way:

 ~/Developer/dooly/marketing   gp-boards-landing-page ● ⍟3  yarn run deploy                                                      8.16.0 ⬢
yarn run v1.17.0
$ gatsby build && gulp deploy
success open and validate gatsby-configs - 0.020 s
success load plugins - 1.558 s
success onPreInit - 0.007 s
success delete html and css files from previous builds - 0.072 s
success initialize cache - 0.008 s
success copy gatsby files - 0.083 s
success onPreBootstrap - 0.018 s
Starting to fetch data from Contentful
Fetching default locale
default locale is : en-US
contentTypes fetched 21
Updated entries  0
Deleted entries  0
Updated assets  0
Deleted assets  0
Fetch Contentful data: 383.381ms
success source and transform nodes - 0.976 s
success building schema - 0.728 s
success createPages - 0.089 s
success createPagesStatefully - 0.053 s
success onPreExtractQueries - 0.011 s
success update schema - 0.037 s
success extract queries from components - 0.217 s
success write out requires - 0.006 s
success write out redirect data - 0.002 s
success Build manifest and related icons - 0.094 s
success onPostBootstrap - 0.103 s
⠀
info bootstrap finished - 8.207 s
⠀
success run static queries - 0.002 s
success Building production JavaScript and CSS bundles - 14.427 s
error ENOENT: no such file or directory, open '/Users/gp/Developer/dooly/marketing/public/page-data/404.html/page-data.json'

  Error: ENOENT: no such file or directory, open '/Users/gp/Developer/dooly/marketing/public/page-data/404.html/page-data.json'

⠙ Rewriting compilation hashes
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Unless I delete .cache before I run the same production build command. yarn deploy just runs gatsby build && gulp deploy. If .cache is deleted before the production build runs, there are never any issues.

Environment

  System:
    OS: macOS 10.14.5
    CPU: (4) x64 Intel(R) Core(TM) i5-7267U CPU @ 3.10GHz
    Shell: 5.3 - /bin/zsh
  Binaries:
    Node: 8.16.0 - ~/.nvm/versions/node/v8.16.0/bin/node
    Yarn: 1.17.0 - /usr/local/bin/yarn
    npm: 6.4.1 - ~/.nvm/versions/node/v8.16.0/bin/npm
  Languages:
    Python: 2.7.10 - /usr/bin/python
  Browsers:
    Chrome: 75.0.3770.100
    Safari: 12.1.1
  npmPackages:
    gatsby: ^2.11 => 2.11.0
    gatsby-image: ^2.0.25 => 2.1.4
    gatsby-plugin-manifest: ^2.0.13 => 2.1.1
    gatsby-plugin-react-helmet: ^3.1 => 3.1.0
    gatsby-plugin-remove-serviceworker: ^1.0.0 => 1.0.0
    gatsby-plugin-sharp: ^2.0.17 => 2.1.7
    gatsby-plugin-sitemap: ^2.0.3 => 2.1.0
    gatsby-plugin-styled-components: ^3.1.0 => 3.1.0
    gatsby-source-contentful: ^2.0.45 => 2.0.73
    gatsby-source-filesystem: ^2.0.12 => 2.0.41
    gatsby-transformer-remark: ^2.1.12 => 2.4.0
    gatsby-transformer-sharp: ^2.1.10 => 2.1.21

PolGuixe commented 5 years ago

We have encountered this issue on a CICD environment where we cache /public and /.cache in order to speed up builds...

If we eliminate the cache then builds will be slow again... 😢

josephkandi commented 5 years ago

I am having a similar issue using Firebase 6.5.0, WebpackError: ReferenceError: IDBIndex is not defined.

When i change the firebase version to 5.9.0 the build works fine without issues. Anyone solved this when using firebase?

kostimarko commented 5 years ago

I fixed this error by switching to http calls rather than trying to call the cloud function from firebase.

josephkandi commented 5 years ago

I fixed this error by switching to http calls rather than trying to call the cloud function from firebase.

Am using firebase auth and firestore in my case. I would want to continue using it this way but find a fix that would work on the latest version

antonioallen commented 5 years ago

@josephkandi I am having the same issue as well. I'm going to try rolling back to 5.9.0 and see what happens.

fabianrios commented 5 years ago

I had the same problem it should be standard to run gastby clean before build to avoid this error "build": "npm run clean && gatsby build",

josephkandi commented 5 years ago

@fabianrios

I had the same problem it should be standard to run gastby clean before build to avoid this error "build": "npm run clean && gatsby build",

Was your issue related to the firebase package?

fabianrios commented 5 years ago

I don't think so but it got solved as well just by clean before building the same way so many other people have solve it, so the out of the box gatsby build command should include clean and that way a common error could be avoided.

AAverin commented 5 years ago

Because this seems to be the thread to discuss IDBIndex is not defined error when building an app with Firebase inside, just want to link a thread to firebase sdk with explanation why this happens. https://github.com/firebase/firebase-js-sdk/issues/1797

I am not so good with webpack configuration, maybe somebody could come up with a gatsby workaround for the problem? All solutions on the mentioned ticket are not for gatsby

AAverin commented 5 years ago

https://github.com/firebase/firebase-js-sdk/issues/1455#issuecomment-455712500 This comment offers a workaround via regex, maybe we can do something like this with a null-loader?

AAverin commented 5 years ago

Solved using lazy firebase init explained here https://github.com/gatsbyjs/gatsby/issues/8612

odinho commented 4 years ago

Oh my god. I debugged this for a long time. I thought it was the URL global missing. But that was a red herring.

We delete the public folder in our yarn build so it'll always be fresh, and also so that Hugo + Gatsby and some other scripts can interact fully without eachother.

It's not unreasonable to think this would be fine.

roffelsaurus commented 4 years ago

I was having CI build issues. I was caching /.cache/ but not ./public/ and that caused this error to happen. Keeping both or deleting both will fix this issue as it is appear. I have a small worry though.

I see alot of people here talking about "always cleaning both" but that is definitely a no-go as build time is a critical factor.

So my only option is to keep both, but I fear that will lead to remnants of old builds being deployed with new statics.

The design decision of keeping cached stuff in the build output folder is a weird one. File copy time and disk space is insignificant to most people compared to query speed.

At the very least the error message in this code path should be changed to something like "./.cache was expecting file /page-data.json to be present in public/ folder. If this was deleted, please delete both .cache/ and public/ or run gatsby clean to avoid this issue"

redconfetti commented 4 years ago

I had the same issue. Thank you. I had deleted the public folder to troubleshoot an issue with a left over asset displaying, but ran into this error after. After deleting the .cache folder, my issue was resolved.

lifeeric commented 4 years ago

Solved by removing React.lazy

https://stackoverflow.com/questions/62978046/building-static-html-for-pageserror-error-enoent-no-such-file-or-directory/63066839#63066839

gatsbyjs / gatsby