search

patrickhulce / fontmin-webpack

Minifies icon fonts to just the used glyphs.
MIT License
139 stars 19 forks source link

fonteditor-core reports "Error: check sum out of bound" #35

Closed zonque closed 3 years ago

zonque commented 3 years ago

I'm trying to use this plugin in a vuejs application with a minimal webpack config:

    new FontminPlugin({
      autodetect: true,
      glyphs: [],
    }),

When the webpack rule is triggered however, I'm getting the following error:

 ERROR  Error: check sum out of bound
Error: check sum out of bound
    at checkSumArrayBuffer (/opt/src/Frontend/node_modules/fonteditor-core/ttf/util/checkSum.js:13:15)
    at checkSum (/opt/src/Frontend/node_modules/fonteditor-core/ttf/util/checkSum.js:80:16)
    at /opt/src/Frontend/node_modules/fonteditor-core/ttf/ttfwriter.js:185:26
    at Array.forEach (<anonymous>)
    at TTFWriter.write (/opt/src/Frontend/node_modules/fonteditor-core/ttf/ttfwriter.js:174:24)
    at TTFWriter.write (/opt/src/Frontend/node_modules/fonteditor-core/ttf/ttfwriter.js:231:24)
    at minifyTtf (/opt/src/Frontend/node_modules/fontmin/plugins/glyph.js:97:29)
    at Through2._transform (/opt/src/Frontend/node_modules/fontmin/plugins/glyph.js:153:27)
    at Through2.Transform._read (/opt/src/Frontend/node_modules/fontmin/node_modules/readable-stream/lib/_stream_transform.js:159:10)
    at Through2.Transform._write (/opt/src/Frontend/node_modules/fontmin/node_modules/readable-stream/lib/_stream_transform.js:147:83)
    at doWrite (/opt/src/Frontend/node_modules/fontmin/node_modules/readable-stream/lib/_stream_writable.js:313:64)
    at writeOrBuffer (/opt/src/Frontend/node_modules/fontmin/node_modules/readable-stream/lib/_stream_writable.js:302:5)
    at Through2.Writable.write (/opt/src/Frontend/node_modules/fontmin/node_modules/readable-stream/lib/_stream_writable.js:241:11)
    at PassThrough.ondata (/opt/src/Frontend/node_modules/readable-stream/lib/_stream_readable.js:619:20)
    at PassThrough.emit (events.js:315:20)
    at PassThrough.Readable.read (/opt/src/Frontend/node_modules/readable-stream/lib/_stream_readable.js:469:26)

Note that I'm using fontmin in version 2.7.0 as I'm dependent on webpack 4 due to other modules in the project.

Is there anything I'm missing? I couldn't find reports on similar issues.

zonque commented 3 years ago

The file I'm trying to reduce in size is the material design icon font, ftr.

patrickhulce commented 3 years ago

Thanks for filing @zonque!

I'm guessing this is one of three things:

  1. The TTF file is corrupt or invalid somehow.
  2. We accidentally identified a non-TTF file from the webpack resources as the font file and passed that to fonteditor-core.
  3. There's a bug in fonteditor-core.

If you have a minimal example you might be able to set DEBUG=fontmin-webpack to see what font files fontmin-webpack is analyzing to narrow down the cause?

zonque commented 3 years ago

Here's that output:

⠦  Building for production...  fontmin-webpack found 5961 glyphs in CSS +0ms
  fontmin-webpack analyzing font group: materialdesignicons.min +1ms
 ERROR  Error: check sum out of bound

My intention is to reduce the font files to only the few items my project actually needs. For that, I also tried to set

      new FontminPlugin({
        autodetect: false,
        glyphs: [ '\uF0493', ],
      }),

(to only get the 'cog' icon from that font, but it failed in the same way.

Note that I'm embedding the material icon font by using the @mdi/font module, and I'm importing @mdi/font/css/materialdesignicons.min.css in the Vue main component, which injects tons of glyphs in the CSS. fontmin seems to recognize those correctly.

zonque commented 3 years ago

FWIW, I've pushed the tree that shows that effect here: https://github.com/MadPresso/Frontend/tree/fontmin You can just run npm run build, the webpack config is in vue.config.js.

happysalada commented 3 years ago

Have the same error with webpack 5 and fontmin 3.0.0 Here is the full stacktrace for the error

remote: #26 12.83 Error: check sum out of bound
remote: #26 12.83     at checkSumArrayBuffer (/opt/app/node_modules/fonteditor-core/ttf/util/checkSum.js:13:15)
remote: #26 12.83     at checkSum (/opt/app/node_modules/fonteditor-core/ttf/util/checkSum.js:80:16)
remote: #26 12.83     at /opt/app/node_modules/fonteditor-core/ttf/ttfwriter.js:185:26
remote: #26 12.83     at Array.forEach (<anonymous>)
remote: #26 12.83     at TTFWriter.write (/opt/app/node_modules/fonteditor-core/ttf/ttfwriter.js:174:24)
remote: #26 12.83     at TTFWriter.write (/opt/app/node_modules/fonteditor-core/ttf/ttfwriter.js:231:24)
remote: #26 12.83     at minifyTtf (/opt/app/node_modules/fontmin/plugins/glyph.js:97:29)
remote: #26 12.83     at Through2._transform (/opt/app/node_modules/fontmin/plugins/glyph.js:153:27)
remote: #26 12.83     at Through2.Transform._read (/opt/app/node_modules/fontmin/node_modules/readable-stream/lib/_stream_transform.js:159:10)
remote: #26 12.83     at Through2.Transform._write (/opt/app/node_modules/fontmin/node_modules/readable-stream/lib/_stream_transform.js:147:83)
remote: #26 12.83 caused by plugins in Compilation.hooks.processAssets
remote: #26 12.83 Error: check sum out of bound
remote: #26 12.83     at checkSumArrayBuffer (/opt/app/node_modules/fonteditor-core/ttf/util/checkSum.js:13:15)
remote: #26 12.83     at checkSum (/opt/app/node_modules/fonteditor-core/ttf/util/checkSum.js:80:16)
remote: #26 12.83     at /opt/app/node_modules/fonteditor-core/ttf/ttfwriter.js:185:26
remote: #26 12.83     at Array.forEach (<anonymous>)
remote: #26 12.83     at TTFWriter.write (/opt/app/node_modules/fonteditor-core/ttf/ttfwriter.js:174:24)
remote: #26 12.83     at TTFWriter.write (/opt/app/node_modules/fonteditor-core/ttf/ttfwriter.js:231:24)
remote: #26 12.83     at minifyTtf (/opt/app/node_modules/fontmin/plugins/glyph.js:97:29)
remote: #26 12.83     at Through2._transform (/opt/app/node_modules/fontmin/plugins/glyph.js:153:27)
remote: #26 12.83     at Through2.Transform._read (/opt/app/node_modules/fontmin/node_modules/readable-stream/lib/_stream_transform.js:159:10)
remote: #26 12.83     at Through2.Transform._write (/opt/app/node_modules/fontmin/node_modules/readable-stream/lib/_stream_transform.js:147:83)

The following warning is also generated

remote: #26 0.917 $ webpack --config ./webpack/webpack.prod.js
remote: #26 1.663 (node:27) V8: /opt/app/node_modules/ttf2woff2/jssrc/ttf2woff2.js:3 Invalid asm.js: Invalid member of stdlib

On install I also have the following warnings

warning fontmin-webpack > ttf2woff2 > node-gyp > request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
warning fontmin-webpack > ttf2woff2 > node-gyp > request > har-validator@5.1.5: this library is no longer supported

Some additional informations. The output is coming from docker with the image node:15.8.0-alpine

The ttf file is the fontawesome one.

happysalada commented 3 years ago

I was just checking briefly but it seems that the ttf2woff2 has a new version, maybe upgrading to the new version will fix this? (It's very naive, I know, just thinking that at least if the warnings are fixed, it's one less thing to worry about)

patrickhulce commented 3 years ago

I'll try to do more investigation later this week

patrickhulce commented 3 years ago

Alright thanks for the repro I have figured out the issue! The problem is that these files contain multi-byte UTF-16 characters (i.e. your example @zonque of glyphs: [ '\uF0493' ] isn't valid because JavaScript interprets that as \uF049 and then 3.

The immediate workaround is to disable autodetection from CSS where 5-character hex unicode definitions are valid, and manually split them into the multi-byte sequence equivalents. For example your cog icon @zonque would be.

      new FontminPlugin({
        autodetect: false,
        glyphs: [ '\uDB81\uDC93' ],
      })

which works as expected for me locally.

Longer-term I'll push out a fix that correctly handles these sequences from autodetection.

patrickhulce commented 3 years ago

fixed by e0743ba published in 3.0.1

zonque commented 3 years ago

Thanks a lot for working on this! I can confirm that the error is gone with 3.0.1, and the result is correct when glyps are manually specified. However, the auto-detection still doesn't work for me.

patrickhulce commented 3 years ago

However, the auto-detection still doesn't work for me.

How specifically does it not work? The font and stylesheet from your example was directly added to our test suite using autodetect, so I cannot reproduce an exception being thrown.

zonque commented 3 years ago

How specifically does it not work? The font and stylesheet from your example was directly added to our test suite using autodetect, so I cannot reproduce an exception being thrown.

No exception is being thrown, but from what I can see, the glyphs that are generated in the reduced file don't match the ones in the input. I've checked the generated woff file with FontDrop, and while the number of glyphs seem to match, their codes don't.

Bot sure if I'm holding it right tho. Which tools would you recommend to check?