Noto Khmer sample images rendered incorrectly

punchcutter commented 6 years ago

The sample images for Noto Sans Khmer and Noto Serif Khmer on https://www.google.com/get/noto/ were somehow generated with bad fonts or something.

The source files are fine and the generated fonts are fine, but the images show components that don't line up like you can see here:

notoserifkhmer_bad_sample

All of the sample images show the same problem.

dougfelt commented 6 years ago

I'm not seeing this problem, or this text. Perhaps I'm not looking at the right sample. Can you tell me more exactly which sample you're seeing this problem with?

punchcutter commented 6 years ago

Sorry, I should have added the direct links. Every Noto Serif Khmer sample is messed up. It's easier to see with the lighter weights, but either way you have to zoom in a bit.

And the same for the sans, but less terrible

punchcutter commented 6 years ago

I ran nototools/docker/make_getnoto/Dockerfile and I get the same result. It seems to be something happening with how TTF components are drawn with pangocairo. Larger sizes look fine, but at small sizes the components are somehow drawn in a different position so they no longer line up. PNG looks ok at any size, but at small sizes it's rasterized so hard to tell if the components actually got drawn the same way as in the SVG. New Khmer fonts with decomposed components are of course fine, but something is off with the TTF component drawing.

punchcutter commented 6 years ago

Tested with hb-view and the result is the same. It looks similar to what's happening here where components are scaled before being offset: https://github.com/googlei18n/noto-fonts/issues/1012 Also, it's only happening on composite glyphs with no outlines. In the screenshot above the two glyphs with the very noticeable shift are both made up of two components. Glyphs that have outlines plus components don't see the component shift.

punchcutter commented 6 years ago

So this seems to be happening in freetype ttgload.c in TT_Process_Composite_Component. The rounding done by FT_PIX_ROUND is shifting the components by different amounts. If I comment out the rounding code and rebuild freetype then everything looks good at all sizes.

        if ( subglyph->flags & ROUND_XY_TO_GRID )
        {
          x = FT_PIX_ROUND( x );
          y = FT_PIX_ROUND( y );
        }

Here is a before/after comparison. Top is the way it has always been. Bottom is with the rounding code commented out. khmer_sample_old_new

punchcutter commented 6 years ago

Also, this has nothing to do with Khmer, but that's where it was most obvious. Noto Sans and Serif also have the problem with any composite glyphs like ἈἉἊἋἌἍἎἏᾇᾒ, etc.

punchcutter commented 6 years ago

ROUND_XY_TO_GRID also discussed here: https://github.com/googlei18n/fontmake/issues/253

marekjez86 commented 6 years ago

nice catch and debugging. for some reason I think I've seen this before and might have reported it but now I cannot find anything related.

punchcutter commented 6 years ago

I'm not sure what the answer is because setting ROUND_XY_TO_GRID seems desirable for some situations like discussed at https://www.freetype.org/ttfautohint/doc/ttfautohint.html#composite-glyphs, but for rendering these sample images we really don't want it. I'm curious to know if there are other places where this might show up either because of freetype or when using some other library. In any case this particular Khmer issue will disappear once the new decomposed source files are run through the pipeline.

dougfelt commented 6 years ago

@behdad please advise how to proceed on this.

behdad commented 6 years ago

This is bizarre. I wonder if the generating code is not disabling hinting properly; then again, I would have assumed that to be default for SVG surface output.

behdad commented 6 years ago

@anthrotype do we set ROUND_XY_TO_GRID by default? My gut feeling is that we shouldn't.

punchcutter commented 6 years ago

ufo2ft has the flag set by default as @anthrotype wrote here: https://github.com/googlei18n/fontmake/issues/253#issuecomment-318810449 Exporting TTFs from Glyphs didn't have this flag set in the past, but now it does. https://forum.glyphsapp.com/t/set-round-xy-to-grid-flag-in-glyphs-app/2581

anthrotype commented 6 years ago

do we set ROUND_XY_TO_GRID by default? My gut feeling is that we shouldn't

yes, we do. What we could do is:

change the default to not set the flag
add a font-level as well as glif-level lib key (prefixed with ufo2ft domain) that enables ROUND_XY_TO_GRID respectively for all composite glyphs in the font, or for all components in a glyph.

I prefer to not add other command line options to fontmake.

Exporting TTFs from Glyphs didn't have this flag set in the past, but now it does.

@punchcutter does it set it all the time now, or only when the font contains hinting?

punchcutter commented 6 years ago

I think ROUND_XY_TO_GRID always gets set on TTF export. @schrifgestalt?

lemzwerg commented 6 years ago

I've just modified FreeType so that ROUND_XY_TO_GRID only has an effect if hinting is on. Additionally, the horizontal subglyph offset gets only rounded if subpixel hinting is off.

While this change improves rendering for ClearType, it still doesn't work as expected for the old grayscale mode. What about modifying the subglyphs to not use ARGS_ARE_XY_VALUES (i.e., using anchor points instead of offsets)?

anthrotype commented 6 years ago

thanks Werner!

What about modifying the subglyphs to not use ARGS_ARE_XY_VALUES (i.e., using anchor points instead of offsets)?

yeah, I and @moyogo have been thinking about that, but the problem is in the UFO format, glyph components can only have XY offset. http://unifiedfontobject.org/versions/ufo3/glyphs/glif/ I'm not sure how that could be implemented cleanly in our pipeline, but I agree it would be nice to be able to support that too.

I wonder, how do other rasterizers besides FreeType handle the same situation? What does the string in the OP's screenshot look like on Windows or macOS? Is the ROUND_XY_TO_GRID always enforced whether or not hinting instructions are present or executed?

lemzwerg commented 6 years ago

All of your questions are good ones, and I hope that someone is going to investigate this!

punchcutter commented 6 years ago

@anthrotype

The text is មនុស្សទាំងអស់ កើតមកមានសេរីភាព និងសមភាព ក្នុងផ្នែកសេចក្ដីថ្លៃថ្នូរនិងសិទ្ធិ។ មនុស្ស មានវិចារណញ្ញាណនិងសតិសម្បជញ្ញៈជាប់ពីកំណើត ហើយគប្បីប្រព្រឹត្ដចំពោះគ្នាទៅវិញទៅមក ក្នុង ស្មារតីភាតរភាពជាបងប្អូន។

Copied from here: https://github.com/googlei18n/nototools/blob/master/sample_texts/km-Khmr_udhr.txt

Two quick tests with NotoSerifKhmer-Thin unhinted TTF:

OS X 10.12.6 with TextEdit notoserifkhmer-thin_ttf_unhinted_10 12 6_textedit

You can see little blobs where the overlap is so it's also not lining up quite perfectly there.

Windows 10 with Word 2016 notoserifkhmer-thin_ttf_unhinted_win10_word2016

The size difference is just the zoom, but trying different sizes in both situations didn't change much. Windows looks fine like this.

lemzwerg commented 6 years ago

The alignment on OS X is OK, but the rendering isn't, AFAICS. The contours of the Khmer font slightly overlap, and obviously composite glyphs get rendered piecewise on this platform. This means that there is twice as much ink on the overlapping sections, making them darker, and hence the blobs.

I consider this behaviour a bug on OS X. Other people might say this is a feature to have quicker rendering since overlapping is rather infrequent – there exists the new MERG table; maybe it is necessary to add this to the Khmer fonts to ensure proper rendering on Macs.

lemzwerg commented 6 years ago

Note that FreeType has the same bug as OS X! You will also get darker parts where the glyphs overlap. Apparently OS X is using a more aggressive colour filter, increasing the visibility of the overlaps.

As a corollary, Windows uses a rendering technique that avoids extra darkening of overlaps. I guess this is the big advantage of ClearType's super-sampling method, which obviously handles everything as B/W at a high resolution, then applying colour filtering to render on the screen.

anthrotype commented 6 years ago

Thanks Zachary for the tests!

So, apart from the darkening of the overlaps, it seems that neither macOS nor Windows are affected by the issue with the ROUND_XY_TO_GRID flag.

Interesting, I didn't know about the MERG table https://www.microsoft.com/typography/otspec/merg.htm

Do you know any fonts that use it? Or if it's implemented anywhere? It looks like fonttools doesn't support it either.

lemzwerg commented 6 years ago

Never seen such a beast...

moyogo commented 6 years ago

On Windows 10 16257 there is a MERG table in :

LeelUIsl.ttf
LeelaUIb.ttf
LeelawUI.ttf
Nirmala.ttf
NirmalaB.ttf
NirmalaS.ttf
consola.ttf
consolab.ttf
consolai.ttf
consolaz.ttf
ebrima.ttf
ebrimabd.ttf
gadugi.ttf
gadugib.ttf
himalaya.ttf
malgun.ttf
malgunbd.ttf
malgunsl.ttf
segoeui.ttf
segoeuib.ttf
segoeuii.ttf
segoeuil.ttf
segoeuisl.ttf
segoeuiz.ttf
seguili.ttf
seguisb.ttf
seguisbi.ttf
seguisli.ttf
seguisym.ttf

twardoch commented 6 years ago

I think the MERG table is one of the new OpenType tables added in the old-style way (i.e. Microsoft came up with a private table, implemented it in Windows and then put it into the OT spec without any discussion). It is somewhat useful, though looks to me as a bit of an ad-hoc thing that addresses just one problem. It does not make any references to color fonts where “merging” is actually much more problematic. And it does not mention variable fonts.

behdad commented 6 years ago

Note that MERG table seems to be one of those dead-on-arrival ones. One person commented: "What’s more, Microsoft has shipped fonts with the same 12-byte MERG that does not specify any overlaps."

There are many other problems with MERG table. At any rate, that doesn't apply to components within a glyph.

simoncozens commented 2 years ago

Samples now look fine on the Noto web site.

notofonts / khmer

Noto Khmer sample images rendered incorrectly #21