gatsby-plugin-sharp tracedSVG crashes build

tylermenezes commented 6 years ago

Description

Using tracedSVG on a file ending in .jpeg causes gatsby-plugin-sharp to report the following vague error:

error UNHANDLED EXCEPTION

  TypeError: Cannot read property 'bitmap' of undefined

  - Potrace.js:1000 Potrace._processLoadedImage
    [www]/[potrace]/lib/Potrace.js:1000:35

  - Potrace.js:1046 Jimp.<anonymous>
    [www]/[potrace]/lib/Potrace.js:1046:14

  - index.js:85 Jimp.throwError
    [www]/[jimp]/index.js:85:44

  - index.js:201 ReadFileContext.callback
    [www]/[jimp]/index.js:201:44

Of note, Twitter always uses .jpeg for its image suffix, so any images saved from Twitter will crash tracedSVG.

Steps to reproduce

Upload a .jpeg file to Contentful
Load the image in GraphQL using tracedSVG

Expected result

Image is traced into an SVG, or
Gatsby tells me which error could not be traced

Actual result

Vague error message.

Environment


  [4mSystem:[0m
    OS: Linux 4.13 Ubuntu 17.10 (Artful Aardvark)
    CPU: x64 Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz
    Shell: 5.2 - /usr/bin/zsh
  [4mBinaries:[0m
    Node: 8.11.2 - /usr/bin/node
    npm: 6.4.1 - /usr/bin/npm
  [4mBrowsers:[0m
    Chrome: 67.0.3396.62
    Firefox: 61.0.1
  [4mnpmPackages:[0m
    gatsby: ^2.0.0-rc.28 => 2.0.0-rc.28 
    gatsby-image: ^2.0.0-rc.4 => 2.0.0-rc.4 
    gatsby-plugin-catch-links: ^2.0.2-rc.1 => 2.0.2-rc.1 
    gatsby-plugin-manifest: ^2.0.2-rc.1 => 2.0.2-rc.1 
    gatsby-plugin-react-helmet: ^3.0.0-rc.1 => 3.0.0-rc.1 
    gatsby-plugin-react-next: next => 2.0.1-13 
    gatsby-plugin-remove-trailing-slashes: ^1.0.9 => 1.0.9 
    gatsby-plugin-sass: ^2.0.0-rc.2 => 2.0.0-rc.2 
    gatsby-plugin-sharp: ^2.0.0-rc.7 => 2.0.0-rc.7 
    gatsby-plugin-sitemap: ^2.0.0-rc.2 => 2.0.0-rc.2 
    gatsby-source-contentful: ^2.0.1-rc.9 => 2.0.1-rc.9 
    gatsby-source-filesystem: ^2.0.1-rc.6 => 2.0.1-rc.6 
    gatsby-transformer-sharp: ^2.1.1-rc.3 => 2.1.1-rc.3 
    gatsby-transformer-yaml: ^2.1.1-rc.6 => 2.1.1-rc.6 
  [4mnpmGlobalPackages:[0m
    gatsby-cli: 2.4.1

kakadiadarpan commented 6 years ago

@tylermenezes can you provide a reproduction repo for this? That would make it much easier to diagnose this issue.

JanneVincit commented 5 years ago

I can confirm the same error message in our project. The difference is that I haven't been able to fully isolate the issue on my part yet. Though this issue is so far the only one, that might give me a hint on the issue. The error on our project comes randomly, not on every build. I removed the tracedSVG fragment, from the code and so far it seems to work more reliably, but in my case as I'm not able to reproduce it reliable, I won't make any conclusions yet.

fk commented 5 years ago

Hmmmm, I just changed the extension of one of the using-gatsby-image example images to jpeg and everything seems to work fine. :/

@JanneVincit Are you also experiencing this in the context of Contenful like @tylermenezes?

tylermenezes commented 5 years ago

Renaming the file from .jpeg -> .jpg in Contentful was the solution which solved my problem, but the error message is extremely generic, and when I looked into the code while trying to troubleshoot, it seems like any error will result in this message. A better solution is probably to improve the error message quality, but IIRC that requires a fix from both Gatsby and Potrace.

fk commented 5 years ago

@tylermenezes Totally agree that error messages need to be improved here! Didn't want to take away from that with my comment — just wanted to make sure that this is only happening in the context of Contentful.

JanneVincit commented 5 years ago

@fk Sorry for not being more explicit. Yes it happens in the context of contentful. We are actually not in production with this yet (though the code is ready, but this doesn't make me trust the implementation, so haven't gone to production yet).

Our issue is probably something else than the jpeg-issue @tylermenezes was experiencing. But as tyle pointed out, the error is quite generic and could be caused by something else too.

dotlouis commented 5 years ago

Still having the issue here. Happens on Netlify but not on local. Hard to pinpoint the problem.

EDIT: 4 Feb. 2018: Starting to have the issue on local builds also. Intermittently for no apparent reasons.

damontes commented 5 years ago

@dotlouis Hi, i'm having the same problem when deploying to Netlify, have you find a solution ?

tylermenezes commented 5 years ago

@dotlouis @Damex989 Because the error message is super generic, it's hard to tell what "the problem" is. If you're having a problem on Netlify but not on local builds, it's likely a different error entirely.

What has changed since the last successful build? Not just code-wise, but also in the CMS? If you can help narrow the problem down I might be able to help figure out what's causing your particular error.

ghost commented 5 years ago

@tylermenezes Thank you for the response, it's going to sound weird but sometimes it builds correctly sometimes not, i'm using Contentful - Gatsby - Netlify, and i think it's the same error you mentioned at the beginning

error UNHANDLED EXCEPTION

TypeError: Cannot read property 'bitmap' of undefined

- Potrace.js:1000 Potrace._processLoadedImage
 [repo]/[potrace]/lib/Potrace.js:1000:35

 - Potrace.js:1046 Jimp.<anonymous>
 [repo]/[potrace]/lib/Potrace.js:1046:14

 - index.js:85 Jimp.throwError
 [repo]/[jimp]/index.js:85:44

- index.js:201 ReadFileContext.callback
[repo]/[jimp]/index.js:201:44

tylermenezes commented 5 years ago

It's definitely the same error, but as I mentioned, that error message is very generic, and it's probably not the same cause. If it were the exact problem I had, with .jpeg files on Contentful, it would fail 100% of the time in all environments.

That error basically just means "could not read the source file for some reason" -- could be that the file extension/mime-type is wrong, something in the file header is messed up, the file didn't download properly because of network issues, ...

Is the failing build only happening on Netlify, or locally as well? If it's only on Netlify, you'd need to figure out what's changing in the environment between successful and failing builds. If you noticed when you started having this problem, you may also be able to figure it out yourself.

DSchau commented 5 years ago

Some debugging by @fk led to some useful insights.

1) potrace depends on a fairly old version of jimp (https://github.com/tooolbox/node-potrace/blob/master/package.json) 1) jimp has received a few minor releases, and it's possible simply bumping the dep resolves this issue

So what I'd propose we do here is (in order of quick wins, but not necessarily fixing the underlying issue):

PR potrace with a bumped version of jimp to see if the errors reduce or disappear
- We'll also want to validate we haven't broken anything underying in potrace with this change
PR potrace swapping jimp for sharp
Fork potrace and make some of the underlying fixes ourselves (if PRs aren't accepted or encouraged)

What will really help in this issue is if there's anyone who is able to reproduce, even intermittently. I've attempted to reproduce with this repo and this Netlify build but no luck as of yet.

alisonjoseph commented 5 years ago

I had been getting this same error intermittently on Netlify builds only.

TypeError: Cannot read property 'bitmap' of undefined

Removing tracedSVG from my graphQL queries "fixed" it for me for now. (I don't have any files ending in jpeg however, only png and jpg)

kepi0809 commented 5 years ago

the title is misleading because it isn't the jpeg extension that causes the errors (maybe file size?)

On my blog-posts page I had many jpeg files and it works without any problems but it crashes on the post pages…

tried to filter with graphql the jpg and/or jpeg files but it still crashed (I didn't have jpg files on my blog-posts page so I thought it could be the problem but it's not)

If anybody know anything new according to this problem we are all ears 😄

ertrzyiks commented 5 years ago

I spotted this exact issue on CI job, an example failure is here https://circleci.com/gh/ertrzyiks/yummy/51 The problem is gone after I stopped caching .cache and public folders between runs. Each run is clean now and doing everything from scratch.

Still, it's quite far from repro steps, I wasn't able to narrow it down yet. I can try to debug it if it's reproducible after enabling the cache back.

kepi0809 commented 5 years ago

I spotted this exact issue on CI job, an example failure is here https://circleci.com/gh/ertrzyiks/yummy/51 The problem is gone after I stopped caching .cache and public folders between runs. Each run is clean now and doing everything from scratch.

Still, it's quite far from repro steps, I wasn't able to narrow it down yet. I can try to debug it if it's reproducible after enabling the cache back.

Yeah this is unfortunately not a production ready solution... :/ On a blog with over 300 Posts the build time would radically increase :(

quiaro commented 5 years ago

Same as @DanielAlejandroMontesRodriguez, I'm using Gatsby + Contentful and deploying to Netlify. Locally, the builds are running fine all the time, but on Netlify I kept getting the same error that @tylermenezes originally reported.

I followed @alisonjoseph's advice and replaced all my ...GatsbyContentfulFluid_tracedSVG fragments with ...GatsbyContentfulFluid, and Netlify's build finally came through. So in my case, the extensions didn't seem to make a difference since I have both .jpeg and .jpg images in Contentful.

jessepinho commented 5 years ago

Anecdotally, I've been seeing this issue arise when using a larger maxWidth with a fluid image (per @kepi0809 's point). But that's also not consistent, so I can't be sure that's the issue.

wardpeet commented 5 years ago

Seems like a duplicate of https://github.com/gatsbyjs/gatsby/issues/12552?

pieh commented 5 years ago

I opened https://github.com/gatsbyjs/gatsby/pull/12927 PR that hopefully fixes that. Because of random nature of this errors I can't be 100% sure on this

gatsbot[bot] commented 5 years ago

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

pieh commented 5 years ago

Update on this that I forgot to mention before: We published gatsby-plugin-sharp@2.0.33 with potential fixes for this problem - we are not sure if this will fix all instance of it so we keep to issue open, but please update and report back if you still are hitting this issue

vcarl commented 5 years ago

I just stumbled upon this issue while looking for a resolution. Intermittently getting it with gatsby-plugin-sharp@^2.0.35 and gatsby-source-contentful@^2.0.53 (not sure if the latter is relevant). Everything seems consistent with other comments, I can work around the issue by replacing GatsbyContentfulFluid_tracedSVG with GatsbyContentfulFluid.

idesignpixels commented 5 years ago

As above I'm still getting this, not sure what is meant with the replacement solution? I managed to build locally by reducing the max width from 1000 to 800 but netlify won't build.

3:58:27 PM: success extract queries from components — 0.200 s
3:58:29 PM: error UNHANDLED EXCEPTION
3:58:29 PM: 
3:58:29 PM:   TypeError: Cannot read property 'bitmap' of undefined
3:58:29 PM:   
3:58:29 PM:   - Potrace.js:1000 Potrace._processLoadedImage
3:58:29 PM: failed during stage 'building site': Build script returned non-zero exit code: 1
3:58:29 PM:     [repo]/[potrace]/lib/Potrace.js:1000:35
3:58:29 PM:   
3:58:29 PM:   - Potrace.js:1046 Jimp.<anonymous>
3:58:29 PM:     [repo]/[potrace]/lib/Potrace.js:1046:14
3:58:29 PM:   
3:58:29 PM:   - index.js:85 Jimp.throwError
3:58:30 PM: Shutting down logging, 36 messages pending

gallery {
    fluid (
        maxWidth: 800
        maxHeight: 600
        resizingBehavior: FILL
    ) {
        srcSet
        aspectRatio
        src
        sizes
        tracedSVG
    }
   title
   description
}

gatsbot[bot] commented 5 years ago

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community!

smakosh commented 5 years ago

We're having the same issue with Contentful, it happens on png images as well


Errors:
8:04:54 PM:   vipspng: libpng read error
8:04:54 PM:   vips2png: unable to write "/tmp/f0008e9f8878b125b79e0e5adc5ed345-laf.png-5af68fa9e3d0585311eb055946514ab5..png"
8:04:54 PM:   GraphQL request:18:9
8:04:54 PM:   17 |         srcWebp
8:04:54 PM:   18 |         tracedSVG
8:04:54 PM:      |         ^
8:04:54 PM:   19 |       }
``

tylermenezes commented 5 years ago

You should open a brand new issue, as this seems unrelated. The error message for this is "Cannot read property 'bitmap' of undefined".

brotzky commented 4 years ago

We've had the same issue. When using travedSVG on all latest of sharp, gatsby, contentful, etc there are 2 errors:

Netlify bitmap error https://github.com/gatsbyjs/gatsby/issues/8301#issuecomment-450877065
Gatsby tracedSVG error https://github.com/gatsbyjs/gatsby/issues/8301#issuecomment-527601896

For now removing all the tracedSVG instances fixes the build but does dampen the site experience by quite a lot.

I will also edit the title of this issue as it is not specific to .jpeg extensions.

ezekielchow commented 4 years ago

I'm facing this issue too. Using Gatsby + Netlify + Contentful. It happens at random though. On local, the error thrown is

VipsJpeg: Premature end of JPEG file
VipsJpeg: out of order read at line 464

On netlify, I get

TypeError: Cannot read property 'bitmap' of undefined

Potrace.js:1000 Potrace._processLoadedImage
[repo]/[potrace]/lib/Potrace.js:1000:35

Potrace.js:1046 Jimp.<anonymous>
[repo]/[potrace]/lib/Potrace.js:1046:14

pvdz commented 4 years ago

Going to close this issue. There have been many changes to Sharp and friends since this was opened and I feel this issue is no longer applicable to the current version of Gatsby.

If you're still seeing this issue or something that looks like something that was reported here, please open a new, specific, issue about it. And please look to confirm whether there isn't an open issue about it already.

Thanks for all your inputs

gatsbyjs / gatsby