updating gatsby-plugin-sharp from 2.8.0 to 2.9.0 breaks build

barbalex commented 3 years ago

Description

error pngload: non-recoverable state                                                   
pngload: non-recoverable state                                                         
vips2png: unable to write to target target                                             

  Error: pngload: non-recoverable state                                                
  pngload: non-recoverable state                                                       
  vips2png: unable to write to target target                                           

not finished run page queries - 1.474s                                                 
not finished Generating image thumbnails - 1.299s                                      
error Command failed with exit code 1.

I can up- and downgrade repeatedly. v2.8.0 always works, v2.9.0 always errors.

Steps to reproduce

This is a complex app. I have no idea if this happens generally.

clone https://github.com/barbalex/apf2
check that gatsby-plugin-sharp is set to v2.8.0
yarn && yarn dev > should succeed
set gatsby-plugin-sharp to v2.9.0 > should error

Expected result

Build should succeed

Actual result

see above

Environment

Run gatsby info --clipboard in your project directory and paste the output here.

$ yarn gatsby info --clipboard
yarn run v1.22.5
$ C:\Users\alexa\apf2\node_modules\.bin\gatsby info --clipboard

  System:
    OS: Windows 10 10.0.19042
    CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
  Binaries:
    Node: 15.0.1 - ~\AppData\Local\Temp\yarn--1605869962190-0.9665755114141126\node.CMD
    Yarn: 1.22.5 - ~\AppData\Local\Temp\yarn--1605869962190-0.9665755114141126\yarn.CMD
    npm: 7.0.3 - C:\Program Files\nodejs\npm.CMD
  Languages:
    Python: 2.7.17
  Browsers:
    Chrome: 86.0.4240.198
    Edge: Spartan (44.19041.423.0), Chromium (86.0.622.69)
  npmPackages:
    gatsby: 2.27.0 => 2.27.0
    gatsby-cli: 2.14.0 => 2.14.0
    gatsby-image: 2.6.0 => 2.6.0
    gatsby-plugin-create-client-paths: 2.5.0 => 2.5.0
    gatsby-plugin-eslint: 2.0.8 => 2.0.8
    gatsby-plugin-manifest: 2.7.0 => 2.7.0
    gatsby-plugin-offline: 3.5.0 => 3.5.0
    gatsby-plugin-react-helmet: 3.5.0 => 3.5.0
    gatsby-plugin-sharp: 2.8.0 => 2.8.0
    gatsby-plugin-styled-components: 3.5.0 => 3.5.0
    gatsby-plugin-typography: 2.7.0 => 2.7.0
    gatsby-remark-autolink-headers: 2.6.0 => 2.6.0
    gatsby-remark-copy-linked-files: 2.5.0 => 2.5.0
    gatsby-remark-embed-video: 3.0.10 => 3.0.10
    gatsby-remark-emojis: 0.4.3 => 0.4.3
    gatsby-remark-external-links: 0.0.4 => 0.0.4
    gatsby-remark-images: 3.6.0 => 3.6.0
    gatsby-remark-images-medium-zoom: 1.7.0 => 1.7.0
    gatsby-remark-numbered-footnotes: 1.0.1 => 1.0.1
    gatsby-remark-prismjs: 3.8.0 => 3.8.0
    gatsby-source-filesystem: 2.6.0 => 2.6.0
    gatsby-transformer-remark: 2.11.0 => 2.11.0
    gatsby-transformer-sharp: 2.7.0 => 2.7.0

Done in 1.99s.

pieh commented 3 years ago

Thanks for the report, I did run it locally and confirmed I see same result locally.

Only thing to keep in mind that repo as it as right now have mix of incompatible sharp versions:

➜  i28203 git:(master) yarn why sharp
yarn why v1.22.10
[1/4] 🤔  Why do we have the module "sharp"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "sharp@0.26.3"
info Has been hoisted to "sharp"
info Reasons this module exists
   - Hoisted from "gatsby-plugin-manifest#sharp"
   - Hoisted from "gatsby-transformer-sharp#sharp"
info Disk size without dependencies: "25.43MB"
info Disk size with unique dependencies: "26.81MB"
info Disk size with transitive dependencies: "29.48MB"
info Number of shared dependencies: 54
=> Found "gatsby-plugin-sharp#sharp@0.25.4"
info This module exists because "gatsby-plugin-sharp" depends on it.
info Disk size without dependencies: "29.48MB"
info Disk size with unique dependencies: "31.02MB"
info Disk size with transitive dependencies: "33.94MB"
info Number of shared dependencies: 58

Which result in

success open and validate gatsby-configs - 0.036s
⠁ dyld: lazy symbol binding failed: Symbol not found: _g_once_impl
  Referenced from: /Users/misiek/test/i28203/node_modules/sharp/build/Release/sharp.node
  Expected in: /Users/misiek/test/i28203/node_modules/gatsby-plugin-sharp/node_modules/sharp/build/Release/../../vendor/lib/libvips.42.dylib

dyld: Symbol not found: _g_once_impl
  Referenced from: /Users/misiek/test/i28203/node_modules/sharp/build/Release/sharp.node
  Expected in: /Users/misiek/test/i28203/node_modules/gatsby-plugin-sharp/node_modules/sharp/build/Release/../../vendor/lib/libvips.42.dylib

/bin/sh: line 1: 45613 Killed: 9               gatsby build
error Command failed with exit code 137.

Crash almost immediately (at least on MacOS) - using yarn resolutions in package.json to set sharp version to either 0.25.4 (it works) or 0.26.3 (it crashes) was my way around this.

It feels like issue is in sharp package - given that toggling between sharp version and not even touching gatsby-plugin-sharp version cause the failure / successful builds.

Some initial details - here is example image data and arguments we pass to sharp where it fails (I'm not saying it's faulty image - it's just a way to be able to do minimal reproduction we could report in sharp repository):

ERROR {
  file: {
    id: '3e170652-32f4-59b6-ad85-4d7a4e836841',
    children: [ '1c940b52-b428-5f8c-92b8-b237dd0ccd5e' ],
    parent: null,
    internal: {
      contentDigest: 'db413ad7ad34df66d973d41a2829073b',
      type: 'File',
      mediaType: 'image/png',
      description: 'File "src/benutzerDoku/ekPlanen/ekplanen_global.png"',
      counter: 147,
      owner: 'gatsby-source-filesystem'
    },
    sourceInstanceName: 'benutzerDoku-pages',
    relativePath: 'ekPlanen/ekplanen_global.png',
    extension: 'png',
    prettySize: '13.7 kB',
    modifiedTime: '2020-11-20T11:54:51.702Z',
    accessTime: '2020-11-20T12:16:31.549Z',
    changeTime: '2020-11-20T11:54:51.702Z',
    birthTime: '2020-11-20T11:54:51.702Z',
    root: '/',
    dir: '/Users/misiek/test/i28203/src/benutzerDoku/ekPlanen',
    base: 'ekplanen_global.png',
    ext: '.png',
    name: 'ekplanen_global',
    absolutePath: '/Users/misiek/test/i28203/src/benutzerDoku/ekPlanen/ekplanen_global.png',
    relativeDirectory: 'ekPlanen',
    dev: 16777221,
    mode: 33188,
    nlink: 1,
    uid: 501,
    rdev: 0,
    blksize: 4096,
    ino: 205212343,
    size: 13714,
    blocks: 32,
    atimeMs: 1605874591548.6438,
    mtimeMs: 1605873291702.1418,
    ctimeMs: 1605873291702.1418,
    birthtimeMs: 1605873291702.044,
    atime: '2020-11-20T12:16:31.549Z',
    mtime: '2020-11-20T11:54:51.702Z',
    ctime: '2020-11-20T11:54:51.702Z',
    birthtime: '2020-11-20T11:54:51.702Z'
  },
  args: {
    plugins: [],
    maxWidth: 1500,
    wrapperStyle: 'margin-left: 0;',
    linkImagesToOriginal: false,
    showCaptions: false,
    markdownCaptions: false,
    sizeByPixelDensity: false,
    backgroundColor: 'white',
    quality: 50,
    withWebp: false,
    tracedSVG: false,
    loading: 'lazy',
    disableBgImageOnAlpha: false,
    disableBgImage: false,
    pathPrefix: ''
  }
}

Those are arguments we pass to fluid function in https://github.com/gatsbyjs/gatsby/blob/fe18a18812276c19c2bef654f08039f1037113f7/packages/gatsby-remark-images/src/index.js#L151-L156

ascorbic commented 3 years ago

@barbalex It appears to be caused by corrupt images, though sharp should clearly not be crashing. The images in question are the "ekplanen" pngs. I have reported the issue to sharp, and we will be reverting our version until they have a fix. Thanks for the report.

pieh commented 3 years ago

Just 2 cents about crashing part - it's not crashing the process, but it is throwing (we can catch it). We do fail the build with it tho (which we should of course when something fails). It didn't use to throw before (in sharp@0.25.X line) - it's not clear to me if it was silent no-op or not - I didn't get that far in my investigation

lovell commented 3 years ago

Please see https://github.com/lovell/sharp/issues/2448#issuecomment-731279993

barbalex commented 3 years ago

I wonder what "corrupt" exactly means. The image can be viewed locally and is shown in the app just fine.

lovell commented 3 years ago

I wonder what "corrupt" exactly means.

This image contains an ICC (colour) profile with an invalid checksum.

$ pngcheck ekplanen_ap.png 
ekplanen_ap.png  CRC error in chunk iCCP (computed 66723ff5, expected 62c14516)

The venerable libpng, as used by Chrome, ignores this error.

As of v0.26.0, sharp has started to provide the more modern libspng, hence the need to now explicitly confirm you are willing to accept/trust this image via failOnError: false.

vladar commented 3 years ago

Also, gatsby-plugin-sharp supports failOnError option:

https://www.gatsbyjs.com/plugins/gatsby-plugin-sharp/#allow-build-to-continue-on-image-processing-error

Was introduced in #27345

randy408 commented 3 years ago

This will be addressed in the next release of libspng, the default configuration is supposed to match the reference implementation: https://github.com/randy408/libspng/issues/14#issuecomment-731580332

ascorbic commented 3 years ago

Thanks @randy408

ascorbic commented 3 years ago

I've created an internal ticket to catch these errors and log a more friendly message along the lines of "Could not process the image 'foo.png' because it is corrupt"

barbalex commented 3 years ago

@vladar In gatsby-config.js I changed 'gatsby-plugin-sharp' to:

{ resolve: 'gatsby-plugin-sharp', options: { failOnError: false } }

But I still get the same error when using "gatsby-plugin-sharp": "2.10.0".

barbalex commented 3 years ago

I tried this using "gatsby-plugin-sharp": "2.11.1". Result: still get the exact same error. But it does not block the build any more. So I feel that this has been solved. Thanks guys!

barbalex commented 3 years ago

Uups. Suddenly the following error breaks the build again:

alexa@PCA C:\Users\alexa\apf2                                                                                                                                  
$ yarn build                                                                                                                                                   
yarn run v1.22.5                                                                                                                                               
$ gatsby clean && gatsby build                                                                                                                                 
info Deleting .cache, public, C:\Users\alexa\apf2\node_modules\.cache\babel-loader, C:\Users\alexa\apf2\node_modules\.cache\terser-webpack-plugin              
info Successfully deleted directories                                                                                                                          

success open and validate gatsby-configs - 0.104s                                                                                                              
success load plugins - 1.035s                                                                                                                                  
success onPreInit - 0.038s                                                                                                                                     
success delete html and css files from previous builds - 0.003s                                                                                                
success initialize cache - 0.005s                                                                                                                              
success copy gatsby files - 0.425s                                                                                                                             
success onPreBootstrap - 0.021s                                                                                                                                
success createSchemaCustomization - 0.046s                                                                                                                     
success Checking for changed pages - 0.002s                                                                                                                    
success source and transform nodes - 0.259s                                                                                                                    
success building schema - 0.363s                                                                                                                               
info Total nodes: 262, SitePage nodes: 38 (use --verbose for breakdown)                                                                                        
success createPages - 0.054s                                                                                                                                   
success Checking for changed pages - 0.001s                                                                                                                    
success createPagesStatefully - 0.076s                                                                                                                         
success update schema - 0.017s                                                                                                                                 
success onPreExtractQueries - 0.001s                                                                                                                           
success extract queries from components - 2.593s                                                                                                               
success write out redirect data - 0.002s                                                                                                                       
success Build manifest and related icons - 0.754s                                                                                                              
success onPostBootstrap - 0.762s                                                                                                                               
info bootstrap finished - 8.410s                                                                                                                               
success run static queries - 0.024s - 1/1 42.07/s                                                                                                              
warn code block or inline code language not specified in markdown. applying generic code block                                                                 

 ERROR #gatsby-transformer-remark_gatsby-plugin-sharp-20000                                                                                                    

Failed to process image C:/Users/alexa/apf2/src/benutzerDoku/ekPlanen/ekplanen_global.png.                                                                     
It is probably corrupt, so please try replacing it.  If it still fails, please open an issue with the image attached.                                          

  Error: pngload: non-recoverable state                                                                                                                        
  vips2png: unable to write to target target                                                                                                                   

not finished run page queries - 1.246s                                                                                                                         
not finished Generating image thumbnails - 1.137s                                                                                                              

error Command failed with exit code 1.                                                                                                                         
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Here the result from gatsby info --clipboard:

$ yarn gatsby info --clipboard                                                                              
yarn run v1.22.5                                                                                            
$ C:\Users\alexa\apf2\node_modules\.bin\gatsby info --clipboard                                             

  System:                                                                                                   
    OS: Windows 10 10.0.19042                                                                               
    CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz                                                  
  Binaries:                                                                                                 
    Node: 15.0.1 - ~\AppData\Local\Temp\yarn--1608657460259-0.46723258413605295\node.CMD                    
    Yarn: 1.22.5 - ~\AppData\Local\Temp\yarn--1608657460259-0.46723258413605295\yarn.CMD                    
    npm: 7.0.3 - C:\Program Files\nodejs\npm.CMD                                                            
  Languages:                                                                                                
    Python: 2.7.17                                                                                          
  Browsers:                                                                                                 
    Chrome: 87.0.4280.88                                                                                    
    Edge: Spartan (44.19041.423.0), Chromium (87.0.664.66)                                                  
  npmPackages:                                                                                              
    gatsby: 2.29.1 => 2.29.1                                                                                
    gatsby-cli: 2.16.1 => 2.16.1                                                                            
    gatsby-image: 2.8.0 => 2.8.0                                                                            
    gatsby-plugin-create-client-paths: 2.7.0 => 2.7.0                                                       
    gatsby-plugin-eslint: 2.0.8 => 2.0.8                                                                    
    gatsby-plugin-manifest: 2.9.0 => 2.9.0                                                                  
    gatsby-plugin-offline: 3.7.0 => 3.7.0                                                                   
    gatsby-plugin-react-helmet: 3.7.0 => 3.7.0                                                              
    gatsby-plugin-sharp: 2.11.1 => 2.11.1                                                                   
    gatsby-plugin-styled-components: 3.7.0 => 3.7.0                                                         
    gatsby-plugin-typography: 2.9.0 => 2.9.0                                                                
    gatsby-remark-autolink-headers: 2.8.0 => 2.8.0                                                          
    gatsby-remark-copy-linked-files: 2.7.0 => 2.7.0                                                         
    gatsby-remark-embed-video: 3.0.10 => 3.0.10                                                             
    gatsby-remark-emojis: 0.4.3 => 0.4.3                                                                    
    gatsby-remark-external-links: 0.0.4 => 0.0.4                                                            
    gatsby-remark-images: 3.8.0 => 3.8.0                                                                    
    gatsby-remark-images-medium-zoom: 1.7.0 => 1.7.0                                                        
    gatsby-remark-numbered-footnotes: 1.0.1 => 1.0.1                                                        
    gatsby-remark-prismjs: 3.10.0 => 3.10.0                                                                 
    gatsby-source-filesystem: 2.8.0 => 2.8.0                                                                
    gatsby-transformer-remark: 2.13.0 => 2.13.0                                                             
    gatsby-transformer-sharp: 2.9.0 => 2.9.0

and here an excerp from my gatsby-config:

{ resolve: 'gatsby-plugin-sharp', options: { failOnError: false } }

barbalex commented 3 years ago

So that is just the first image that breaks the build. I would probably have to replace lots of images to get my build to work again 😢

mlenser commented 3 years ago

I found the issue to be multiple versions of sharp. If I upgraded

gatsby-plugin-manifest (to 2.10.0)
gatsby-plugin-sharp (to 2.12.0)
gatsby-transformer-sharp (to 2.10.0)

Then when running yarn why sharp I only had sharp@0.27.0 and things seem to work as expected.

yowainwright commented 3 years ago

Thanks, @mlenser! I was dealing with the same issue, and thanks to you, my gatsby site is working within only a few minutes of searching the web.

barbalex commented 3 years ago

This issue still exists with "gatsby-plugin-sharp": "2.13.0":

$ yarn build
yarn run v1.22.5
$ gatsby clean && gatsby build
info Deleting .cache, public, C:\Users\alexa\apf2\node_modules\.cache\babel-loader, C:\Users\alexa\apf2\node_modules\.cache\terser-webpack-plugin
info Successfully deleted directories

success open and validate gatsby-configs - 0.073s
success load plugins - 0.953s
success onPreInit - 0.041s
success delete html and css files from previous builds - 0.003s
success initialize cache - 0.005s
success copy gatsby files - 0.448s
success onPreBootstrap - 0.018s
success createSchemaCustomization - 0.037s
success Checking for changed pages - 0.002s
success source and transform nodes - 0.256s
success building schema - 0.370s
info Total nodes: 262, SitePage nodes: 38 (use --verbose for breakdown)
success createPages - 0.051s
success Checking for changed pages - 0.001s
success createPagesStatefully - 0.074s
success update schema - 0.017s
success onPreExtractQueries - 0.001s
success extract queries from components - 2.582s
success write out redirect data - 0.002s
success Build manifest and related icons - 0.742s
success onPostBootstrap - 0.750s
info bootstrap finished - 8.276s
success run static queries - 0.021s - 1/1 47.70/s
warn code block or inline code language not specified in markdown. applying generic code block

 ERROR #gatsby-transformer-remark_gatsby-plugin-sharp-20000

Failed to process image C:/Users/alexa/apf2/src/benutzerDoku/ekPlanen/ekplanen_ap.png.
It is probably corrupt, so please try replacing it.  If it still fails, please open an issue with the image attached.

  Error: pngload: non-recoverable state
  pngload: non-recoverable state
  vips2png: unable to write to target target

not finished run page queries - 1.264s
not finished Generating image thumbnails - 1.152s

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

My gatsby-config.js contains:

{ resolve: 'gatsby-plugin-sharp', options: { failOnError: false } },

And gatsby info --clipboard gives:

$ gatsby info --clipboard

  System:
    OS: Windows 10 10.0.19042
    CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
  Binaries:
    Node: 15.6.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.5 - C:\Program Files (x86)\Yarn\bin\yarn.CMD
    npm: 7.4.0 - C:\Program Files\nodejs\npm.CMD
  Languages:
    Python: 2.7.17
  Browsers:
    Chrome: 87.0.4280.141
    Edge: Spartan (44.19041.423.0), Chromium (87.0.664.75)
  npmPackages:
    gatsby: 2.31.0 => 2.31.0
    gatsby-cli: 2.18.0 => 2.18.0
    gatsby-image: 2.10.0 => 2.10.0
    gatsby-plugin-create-client-paths: 2.9.0 => 2.9.0
    gatsby-plugin-eslint: 2.0.8 => 2.0.8
    gatsby-plugin-manifest: 2.11.0 => 2.11.0
    gatsby-plugin-offline: 3.9.0 => 3.9.0
    gatsby-plugin-react-helmet: 3.9.0 => 3.9.0
    gatsby-plugin-sharp: 2.13.0 => 2.13.0
    gatsby-plugin-styled-components: 3.9.0 => 3.9.0
    gatsby-plugin-typography: 2.11.0 => 2.11.0
    gatsby-remark-autolink-headers: 2.10.0 => 2.10.0
    gatsby-remark-copy-linked-files: 2.9.0 => 2.9.0
    gatsby-remark-embed-video: 3.0.10 => 3.0.10
    gatsby-remark-emojis: 0.4.3 => 0.4.3
    gatsby-remark-external-links: 0.0.4 => 0.0.4
    gatsby-remark-images: 3.10.0 => 3.10.0
    gatsby-remark-images-medium-zoom: 1.7.0 => 1.7.0
    gatsby-remark-numbered-footnotes: 1.0.1 => 1.0.1
    gatsby-remark-prismjs: 3.12.0 => 3.12.0
    gatsby-source-filesystem: 2.10.0 => 2.10.0
    gatsby-transformer-remark: 2.15.0 => 2.15.0
    gatsby-transformer-sharp: 2.11.0 => 2.11.0
  npmGlobalPackages:
    gatsby: 2.30.1

mlenser commented 3 years ago

@barbalex check what I wrote above in https://github.com/gatsbyjs/gatsby/issues/28203#issuecomment-756685001. It is likely that you have multiple versions of sharp that you could identify with yarn why sharp

barbalex commented 3 years ago

@mlenser Thanks for helping.

$ yarn why sharp
yarn why v1.22.5
[1/4] Why do we have the module "sharp"...?
[2/4] Initialising dependency graph...
[3/4] Finding dependency...
[4/4] Calculating file sizes...
=> Found "sharp@0.27.0"
info Reasons this module exists
   - "gatsby-plugin-manifest" depends on it
   - Hoisted from "gatsby-plugin-manifest#sharp"
   - Hoisted from "gatsby-plugin-sharp#sharp"
   - Hoisted from "gatsby-transformer-sharp#sharp"
info Disk size without dependencies: "68.21MB"
info Disk size with unique dependencies: "69.51MB"
info Disk size with transitive dependencies: "71.95MB"
info Number of shared dependencies: 47
Done in 0.88s.

Unfortunately this does not seem to be the cause in my project.

ascorbic commented 3 years ago

Hi @barbalex , I jsut want to confirm what behaviour you're hoping for? The best we can do it to not make the whole build fail if there is a corrupt image. The image would still be missing. Actually processing a corrupt image would require upstream changes, so is out of our control.

barbalex commented 3 years ago

When updating gatsby-plugin-sharp from v2.8.0 to 2.9.0, I would expect:

my app-build not to completely break. That is something I would only expect from a major version change
me not needing to have to recreate potentially hundreds or thousands of images that so far have worked perfectly in order to be able to keep gatsby up to date. The thing is: This is simply the first image that causes the build to fail. Maybe all do that

The best we can do it to not make the whole build fail if there is a corrupt image

But that is exactly what is happening. I understand that at least one of the images used on my project are technically "corrupted". But that should not stop the project from building.

The image would still be missing. Actually processing a corrupt image would require upstream changes, so is out of our control

Actually, the weird thing is that I have managed to build gatsby with one of the recent gatsby-plugin-sharp versions. And when that worked, the images kept working as well. But then suddenly the build started failing again. In the meantime I had changed nothing at all.

So the image does show when using gatsby-plugin-sharp version 2.8.0 and also sometimes when building with a higher version.

barbalex commented 3 years ago

What would you do if you have a Site where users can upload images. So you now have thousands of them, without a lot of control over the pre-processing of the images the users upload. Suddenly, when updating one of dozens of dependencies of gatsby, your site build breaks and can never be built again.

What now?

barbalex commented 3 years ago

@ascorbic To really answer your question explicitly: I realize you can't influence the fact that so far working images are now technically corrupt. But I expect the gatsby build to still work (while probably telling me about the corrupt images as it does quite nicely).

barbalex commented 3 years ago

I was hoping that https://github.com/gatsbyjs/gatsby/pull/28614 or https://github.com/gatsbyjs/gatsby/pull/28614 had solved this issue

ascorbic commented 3 years ago

@barbalex I agree, so I'll have a look at this. I would expect that a corrupt image would warn but continue building.

ochicf commented 3 years ago

I'm also having this issue: updated all packages to use sharp@0.27.0, configured gatsby-plugin-sharp with failOnError: false, but build still stops when encountering a "corrupted" image.

Info on my system (note: I've tested with node 12 aswell):

➜ ./node_modules/.bin/gatsby info            

  System:
    OS: macOS 10.15.7
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 10.23.0 - ~/.nvm/versions/node/v10.23.0/bin/node
    Yarn: 1.22.10 - /usr/local/bin/yarn
    npm: 6.14.8 - ~/.nvm/versions/node/v10.23.0/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 88.0.4324.96
    Safari: 14.0.2
  npmPackages:
    gatsby: 2.31.1 => 2.31.1
    gatsby-background-image: ^1.1.2 => 1.1.2
    gatsby-image: 2.2.43 => 2.2.43
    gatsby-plugin-eslint: 2.0.8 => 2.0.8
    gatsby-plugin-extract-schema: 0.0.5 => 0.0.5
    gatsby-plugin-flow: 1.1.20 => 1.1.20
    gatsby-plugin-google-analytics: 2.1.37 => 2.1.37
    gatsby-plugin-google-tagmanager: 2.1.20 => 2.1.20
    gatsby-plugin-hotjar: 1.0.1 => 1.0.1
    gatsby-plugin-layout: 1.1.23 => 1.1.23
    gatsby-plugin-manifest: 2.11.0 => 2.11.0
    gatsby-plugin-material-ui: 2.1.6 => 2.1.6
    gatsby-plugin-netlify: 2.1.34 => 2.1.34
    gatsby-plugin-offline: 3.0.40 => 3.0.40
    gatsby-plugin-react-helmet: 3.1.23 => 3.1.23
    gatsby-plugin-remove-serviceworker: 1.0.0 => 1.0.0
    gatsby-plugin-robots-txt: 1.5.0 => 1.5.0
    gatsby-plugin-sharp: 2.13.2 => 2.13.2
    gatsby-plugin-sitemap: 2.2.29 => 2.2.29
    gatsby-plugin-transition-link: ^1.20.5 => 1.20.5
    gatsby-source-filesystem: 2.1.55 => 2.1.55
    gatsby-source-prismic: 3.2.1 => 3.2.1
    gatsby-transformer-sharp: 2.11.0 => 2.11.0

And checking sharp version:

➜ y why sharp                  
yarn why v1.22.10
[1/4] 🤔  Why do we have the module "sharp"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "sharp@0.27.0"
info Reasons this module exists
   - "gatsby-transformer-sharp" depends on it
   - Hoisted from "gatsby-transformer-sharp#sharp"
   - Hoisted from "gatsby-plugin-sharp#sharp"
   - Hoisted from "gatsby-plugin-manifest#sharp"
info Disk size without dependencies: "35.72MB"
info Disk size with unique dependencies: "37.55MB"
info Disk size with transitive dependencies: "40.56MB"
info Number of shared dependencies: 50
✨  Done in 1.25s.

domvo commented 3 years ago

I found that the failOnError option is not passed to sharp() for the creation of Base64 images. I opened a PR to fix this issue: https://github.com/gatsbyjs/gatsby/pull/29254

barbalex commented 3 years ago

I tried again, hoping #29254 would have solved the issue:

The result is:

$ gatsby clean && gatsby build
info Deleting .cache, public, /Users/alexandergabriel/apf2/node_modules/.cache/babel-loader, /Users/alexandergabriel/apf2/node_modules/.cache/terser-webpack-plugin
info Successfully deleted directories

success open and validate gatsby-configs - 0.086s
success load plugins - 1.464s
success onPreInit - 0.045s
success delete html and css files from previous builds - 0.003s
success initialize cache - 0.006s
success copy gatsby files - 0.069s
success onPreBootstrap - 0.027s
success createSchemaCustomization - 0.071s
success Checking for changed pages - 0.001s
success source and transform nodes - 0.287s
success building schema - 0.286s
info Total nodes: 264, SitePage nodes: 38 (use --verbose for breakdown)
success createPages - 0.048s
success Checking for changed pages - 0.001s
success createPagesStatefully - 0.050s
success update schema - 0.019s
success onPreExtractQueries - 0.003s
success extract queries from components - 3.009s
success write out redirect data - 0.006s
success Build manifest and related icons - 1.195s
success onPostBootstrap - 1.199s
info bootstrap finished - 10.266s
success run static queries - 0.011s - 1/1 90.80/s

 ERROR

[object Object] Processing /Users/alexandergabriel/apf2/src/technischeDoku/pwa/installieren.png failed

Original error:

clonedPipeline.resize(...).png(...).webp(...).tiff(...).avif is not a function

  WorkerError: Processing /Users/alexandergabriel/apf2/src/technischeDoku/pwa/installieren.png failed
  Original error:
  clonedPipeline.resize(...).png(...).webp(...).tiff(...).avif is not a function

  - jobs-manager.ts:318 enqueueJob
    [apf2]/[gatsby]/src/utils/jobs-manager.ts:318:21

not finished run page queries - 0.174s
not finished Generating image thumbnails - 0.080s

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

This looks like a different issue to me.

yarn gatsby info --clipboard gives:

alexandergabriel@Alexanders-MacBook-Pro apf2 % yarn gatsby info --clipboard
yarn run v1.22.10
$ /Users/alexandergabriel/apf2/node_modules/.bin/gatsby info --clipboard

  System:
    OS: macOS 11.1
    CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 14.13.1 - /var/folders/68/j97sh7cd4fl5jcl9fwj4w0lh0000gn/T/yarn--1613141138427-0.43653091589364545/node
    Yarn: 1.22.10 - /var/folders/68/j97sh7cd4fl5jcl9fwj4w0lh0000gn/T/yarn--1613141138427-0.43653091589364545/yarn
    npm: 6.14.8 - /usr/local/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 88.0.4324.150
    Firefox: 84.0.1
    Safari: 14.0.2
  npmPackages:
    gatsby: 2.32.3 => 2.32.3
    gatsby-cli: 2.19.1 => 2.19.1
    gatsby-image: 2.11.0 => 2.11.0
    gatsby-plugin-create-client-paths: 2.10.0 => 2.10.0
    gatsby-plugin-eslint: 2.0.8 => 2.0.8
    gatsby-plugin-manifest: 2.12.0 => 2.12.0
    gatsby-plugin-offline: 3.10.0 => 3.10.0
    gatsby-plugin-react-helmet: 3.10.0 => 3.10.0
    gatsby-plugin-sharp: 2.14.1 => 2.14.1
    gatsby-plugin-styled-components: 3.10.0 => 3.10.0
    gatsby-plugin-typography: 2.12.0 => 2.12.0
    gatsby-remark-autolink-headers: 2.11.0 => 2.11.0
    gatsby-remark-copy-linked-files: 2.10.0 => 2.10.0
    gatsby-remark-embed-video: 3.0.12 => 3.0.12
    gatsby-remark-emojis: 0.4.3 => 0.4.3
    gatsby-remark-external-links: 0.0.4 => 0.0.4
    gatsby-remark-images: 3.11.0 => 3.11.0
    gatsby-remark-images-medium-zoom: 1.7.0 => 1.7.0
    gatsby-remark-numbered-footnotes: 1.0.1 => 1.0.1
    gatsby-remark-prismjs: 3.13.0 => 3.13.0
    gatsby-source-filesystem: 2.11.0 => 2.11.0
    gatsby-transformer-remark: 2.16.0 => 2.16.0
    gatsby-transformer-sharp: 2.12.0 => 2.12.0

barbalex commented 3 years ago

So I reverted to the versions that last worked on my windows dev machine (am now on a macOS Notebook).

I first encountered the error mentioned by @pieh (https://github.com/gatsbyjs/gatsby/issues/28203#issuecomment-731138595). So had to use resolutions to prevent the build from crashing due to conflicting versions of sharp. This never happened on windows.

The result is still frustrating:

$ gatsby clean && gatsby build
info Deleting .cache, public, /Users/alexandergabriel/apf2/node_modules/.cache/babel-loader, /Users/alexandergabriel/apf2/node_modules/.cache/terser-webpack-plugin
info Successfully deleted directories

success open and validate gatsby-configs - 0.078s
success load plugins - 2.894s
success onPreInit - 0.044s
success delete html and css files from previous builds - 0.003s
success initialize cache - 0.006s
success copy gatsby files - 0.077s
success onPreBootstrap - 0.015s
success createSchemaCustomization - 0.048s
success Checking for changed pages - 0.001s
success source and transform nodes - 0.250s
success building schema - 0.298s
info Total nodes: 264, SitePage nodes: 38 (use --verbose for breakdown)
success createPages - 0.049s
success Checking for changed pages - 0.001s
success createPagesStatefully - 0.064s
success update schema - 0.021s
success onPreExtractQueries - 0.003s
success extract queries from components - 2.927s
success write out redirect data - 0.002s
success Build manifest and related icons - 1.311s
success onPostBootstrap - 1.314s
info bootstrap finished - 10.586s
success run static queries - 0.011s - 1/1 89.13/s
warn code block or inline code language not specified in markdown. applying generic code block
success run page queries - 1.926s - 44/44 22.84/s
success write out requires - 0.023s
success Building production JavaScript and CSS bundles - 115.676s
success Rewriting compilation hashes - 0.010s
success Building HTML renderer - 35.537s
failed Building static HTML for pages - 1.085s

 ERROR #95313

Building static HTML failed

See our docs page for more info on this error: https://gatsby.dev/debug-html

   7 |  const update = buffer => {
   8 |      const inputEncoding = typeof buffer === 'string' ? 'utf8' : undefined;
>  9 |      hash.update(buffer, inputEncoding);
     | ^
  10 |  };
  11 |
  12 |  if (Array.isArray(data)) {

  WebpackError: TypeError [ERR_INVALID_ARG_TYPE]: The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received type number (NaN)

  - index.js:9
    node_modules/md5-hex/index.js:9:1

  - index.js:17
    node_modules/md5-hex/index.js:17:1

  - uploadcareSignature.js:7
    src/utils/uploadcareSignature.js:7:33

  - Uploader.js:1
    src/components/shared/Uploader.js:1:1

  - index.js:1
    src/components/shared/Files/index.js:1:1

  - index.js:1
    src/components/Projekte/Daten/Ap/index.js:1:1

  - index.js:1
    src/components/Projekte/Daten/index.js:1:1

  - ProjektContainer.js:1
    src/components/Projekte/ProjektContainer.js:1:1

  - index.js:1
    src/components/Projekte/index.js:1:1

not finished Generating image thumbnails - 154.936s

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Another new version of build error.

$ /Users/alexandergabriel/apf2/node_modules/.bin/gatsby info --clipboard

  System:
    OS: macOS 11.1
    CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 14.13.1 - /var/folders/68/j97sh7cd4fl5jcl9fwj4w0lh0000gn/T/yarn--1613142336089-0.41249956488019435/node
    Yarn: 1.22.10 - /var/folders/68/j97sh7cd4fl5jcl9fwj4w0lh0000gn/T/yarn--1613142336089-0.41249956488019435/yarn
    npm: 6.14.8 - /usr/local/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 88.0.4324.150
    Firefox: 84.0.1
    Safari: 14.0.2
  npmPackages:
    gatsby: 2.32.3 => 2.32.3
    gatsby-cli: 2.19.1 => 2.19.1
    gatsby-image: 2.11.0 => 2.11.0
    gatsby-plugin-create-client-paths: 2.10.0 => 2.10.0
    gatsby-plugin-eslint: 2.0.8 => 2.0.8
    gatsby-plugin-manifest: 2.12.0 => 2.12.0
    gatsby-plugin-offline: 3.10.0 => 3.10.0
    gatsby-plugin-react-helmet: 3.10.0 => 3.10.0
    gatsby-plugin-sharp: 2.8.0 => 2.8.0
    gatsby-plugin-styled-components: 3.10.0 => 3.10.0
    gatsby-plugin-typography: 2.12.0 => 2.12.0
    gatsby-remark-autolink-headers: 2.11.0 => 2.11.0
    gatsby-remark-copy-linked-files: 2.10.0 => 2.10.0
    gatsby-remark-embed-video: 3.0.12 => 3.0.12
    gatsby-remark-emojis: 0.4.3 => 0.4.3
    gatsby-remark-external-links: 0.0.4 => 0.0.4
    gatsby-remark-images: 3.11.0 => 3.11.0
    gatsby-remark-images-medium-zoom: 1.7.0 => 1.7.0
    gatsby-remark-numbered-footnotes: 1.0.1 => 1.0.1
    gatsby-remark-prismjs: 3.13.0 => 3.13.0
    gatsby-source-filesystem: 2.11.0 => 2.11.0
    gatsby-transformer-remark: 2.16.0 => 2.16.0
    gatsby-transformer-sharp: 2.12.0 => 2.12.0

I seriously do not know how to get this project to build on macOS. An hour ago it still built on windows. But who knows what will happen next time.

barbalex commented 3 years ago

On the windows pc I now tried updating to the newest versions (gatsby 3):

$ gatsby info --clipboard                                                  

  System:                                                                  
    OS: Windows 10 10.0.19042                                              
    CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz                 
  Binaries:                                                                
    Node: 15.6.0 - C:\Program Files\nodejs\node.EXE                        
    Yarn: 1.22.5 - C:\Program Files (x86)\Yarn\bin\yarn.CMD                
    npm: 7.5.2 - C:\Program Files\nodejs\npm.CMD                           
  Languages:                                                               
    Python: 2.7.17 - C:\Python27\python.EXE                                
  Browsers:                                                                
    Chrome: 89.0.4389.72                                                   
    Edge: Spartan (44.19041.423.0), Chromium (88.0.705.81)                 
  npmPackages:                                                             
    gatsby: 3.0.1 => 3.0.1                                                 
    gatsby-cli: 3.0.0 => 3.0.0                                             
    gatsby-image: 3.0.0 => 3.0.0                                           
    gatsby-plugin-create-client-paths: 3.0.0 => 3.0.0                      
    gatsby-plugin-eslint: 3.0.0 => 3.0.0                                   
    gatsby-plugin-manifest: 3.0.0 => 3.0.0                                 
    gatsby-plugin-offline: 4.0.0 => 4.0.0                                  
    gatsby-plugin-react-helmet: 4.0.0 => 4.0.0                             
    gatsby-plugin-sharp: 3.0.0 => 3.0.0                                    
    gatsby-plugin-styled-components: 4.0.0 => 4.0.0                        
    gatsby-plugin-typography: 3.0.0 => 3.0.0                               
    gatsby-remark-autolink-headers: 3.0.0 => 3.0.0                         
    gatsby-remark-copy-linked-files: 3.0.0 => 3.0.0                        
    gatsby-remark-embed-video: 3.1.0 => 3.1.0                              
    gatsby-remark-emojis: 0.4.3 => 0.4.3                                   
    gatsby-remark-external-links: 0.0.4 => 0.0.4                           
    gatsby-remark-images: 4.0.0 => 4.0.0                                   
    gatsby-remark-images-medium-zoom: 1.7.0 => 1.7.0                       
    gatsby-remark-numbered-footnotes: 1.0.1 => 1.0.1                       
    gatsby-remark-prismjs: 4.0.0 => 4.0.0                                  
    gatsby-source-filesystem: 3.0.0 => 3.0.0                               
    gatsby-transformer-remark: 3.0.0 => 3.0.0                              
    gatsby-transformer-sharp: 3.0.0 => 3.0.0                               
  npmGlobalPackages:                                                       
    gatsby: 2.30.1

But am still getting the error on build:

$ yarn build
yarn run v1.22.5
$ gatsby clean && gatsby build
info Deleting .cache, public, C:\Users\alexa\apf2\node_modules\.cache\babel-loader, C:\Users\alexa\apf2\node_modules\.cache\terser-webpack-plugin
info Successfully deleted directories

success open and validate gatsby-configs - 0.080s
success load plugins - 0.949s
success onPreInit - 0.041s
success initialize cache - 0.005s
success copy gatsby files - 0.608s
success onPreBootstrap - 0.020s
success createSchemaCustomization - 0.041s
success Checking for changed pages - 0.002s
success source and transform nodes - 0.247s
success building schema - 0.434s
info Total nodes: 267, SitePage nodes: 39 (use --verbose for breakdown)
success createPages - 0.063s
success Checking for changed pages - 0.001s
success createPagesStatefully - 0.075s
success update schema - 0.017s
success onPreExtractQueries - 0.001s
success extract queries from components - 3.010s
success write out redirect data - 0.004s
success Build manifest and related icons - 0.930s
success onPostBootstrap - 0.939s
info bootstrap finished - 9.143s
success run static queries - 0.020s - 1/1 51.12/s

 ERROR

[object Object] Processing C:/Users/alexa/apf2/src/technischeDoku/pwa/installieren.png failed

Original error:

clonedPipeline.resize(...).png(...).webp(...).tiff(...).avif is not a function

  WorkerError: Processing C:/Users/alexa/apf2/src/technischeDoku/pwa/installieren.png failed
  Original error:
  clonedPipeline.resize(...).png(...).webp(...).tiff(...).avif is not a function

  - jobs-manager.ts:323 enqueueJob
    [apf2]/[gatsby]/src/utils/jobs-manager.ts:323:21

not finished run page queries - 0.210s

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

ascorbic commented 3 years ago

This error means that your version of sharp doesn't support AVIF. Do you have a manually-installed old version of sharp, or a globally-installed libvips?

ascorbic commented 3 years ago

Reading back through the thread, are you still using yarn resolutions to pin the version of sharp?

barbalex commented 3 years ago

yes

"resolutions": {
  "sharp": "0.25.4"
}

ascorbic commented 3 years ago

That's the reason. It doesn't support AVIF until, I think 0.27. What happens if you remove it now?

barbalex commented 3 years ago

I removed the resolution on sharp v0.25.4:

sharp is used in version 0.27.2
build works fine on windows
build errors on macOS as described by pieh https://github.com/gatsbyjs/gatsby/issues/28203#issuecomment-731138595

Then I updated gatsby-plugin-sharp from 2.8.0 to 3.0.0.

Seems to work on windows 😄 But the build errors exactly as described here: https://github.com/gatsbyjs/gatsby/issues/28203#issuecomment-778251555 on macOS 😢

github-actions[bot] commented 3 years ago

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

tudor07 commented 3 years ago

Having the same issue

EDIT: I added this in package.json. This fixed the problem for me:

"resolutions": {
    "sharp": "0.28.1"
  },

LazerJesus commented 3 years ago

same issue on m1 mac, adding resolutions did not resolve.

princefishthrower commented 3 years ago

hitting this now as well on mac, but I'm v3 and all packages are v3 @tudor07 - were you on mac? any other changes you remember making to package.json?

princefishthrower commented 3 years ago

@FinnFrotscher - just found a stopgap fix, i noticed gatsby-plugin-sharp was working on another v3 gatsby site I had, but not the newest version, but 3.1.2. Reinstalling with:

npm install gatsby-plugin-sharp@3.1.2

made dev and build work again!

dgorczyca commented 3 years ago

I have just encountered the same problem (on Mac OS Cataline - pre M1), combination of few things above worked for me, it might be that only one of those three things was required, nevertheless at this point both gatsby develop and `gatsby build

switching to gatsby-plugin-sharp@3.1.2
adding to package.json
```
"sharp": "0.28.1"
},
```
Used yarn instead of npm

Added this configuration in gatsby-config.js

{
  resolve: "gatsby-plugin-sharp",
  options: { failOnError: false }
},

randy408 commented 3 years ago

Since v0.6.3 all corrupted ancillary chunks are silently discarded, this eliminates 99% of compatibility issues with existing images. Once the spng dependency is updated no workarounds should be necessary.

pragmaticpat commented 3 years ago

Thanks for the fix @randy408 - closing this one. If the issue persists, please open a new issue to keep it fresh with current context.

pacocom commented 3 years ago

npm i sharp@latest NOT solve the problem. At this moment latest version of sharp is "sharp": "0.29.1",

CharlieGreenman commented 2 years ago

For those in this thread, make sure you have the following packages all on the same version:

gatsby-plugin-manifest
gatsby-plugin-sharp
gatsby-transformer-sharp

My issue was that I had gatsby-transformer-sharp + gatsby-plugin-sharp at the latest, but I didn't update gatsby-plugin-manifest. After updating gatsby-plugin-manifest i no longer had the build error. I hope this helps someone ❤️

pacocom commented 2 years ago

In my case I don't need gatsby-plugin-manifest but I will try it.

gatsbyjs / gatsby