Run text rendering tests on OpenType.js

[brawer]

Hi all, I’ve started a project for testing text rendering systems; the project is hosted/copyrighted by Unicode. Although this test suite is still very much in its infancy, you might still find it useful. Feel free to change its test framework so it runs node.js + OpenType.js on the test suite when passing --engine=OpenType.js; your pull requests would be very welcome. Also, don’t hesitate to extend the test coverage if you want; you’d help other projects too. https://github.com/icu-project/text-rendering-tests

[brawer]

Friendly ping, since this would have found https://github.com/nodebox/opentype.js/issues/273 much earlier. So here’s an offer. If somebody implements a command line tool in Node that can be called like this:

some-command --testcase=SomeID/1 --font=path/to/testfont.otf --render="Jj"

so the tool writes an SVG file to standard output, like the following output for --testcase=SomeID/1, --font pointing to this test font, and --render="Jj":

<?xml version="1.0" encoding="UTF-8"?>
<svg version="1.1"
    xmlns="http://www.w3.org/2000/svg"
    xmlns:xlink="http://www.w3.org/1999/xlink"
    viewBox="0 -455 535 1383">
  <symbol id="SomeID/1.J" overflow="visible"><path d="M125,184 L125,599 L57,599 Q41,599 41,613 Q41,620 46,637 L48,645 L261,645 Q265,639 265,629 Q265,618 258,609 L181,601 L181,201 Q181,77 140,0 Q99,-76 -4,-142 Q-22,-127 -29,-107 Q59,-40 92,23 Q125,87 125,184 Z"/></symbol>
  <symbol id="SomeID/1.j" overflow="visible"><path d="M100,134 L100,453 L19,453 Q16,461 16,467 Q16,476 22,486 L126,505 Q143,496 154,483 L154,151 Q154,27 113,-49 Q72,-126 -31,-192 Q-49,-179 -55,-158 Q35,-92 67,-30 Q100,31 100,134 Z M157,632 Q157,590 112,590 Q67,590 67,632 Q67,674 112,674 Q157,674 157,632 Z"/></symbol>
  <use xlink:href="#SomeID/1.J" x="0" y="0"/>
  <use xlink:href="#SomeID/1.j" x="296" y="0"/>
</svg>

If you do that, I’ll make the necessary changes to the Unicode text rendering test suite so it calls your tool for --engine=OpenType.js. Then, we’ll get test reports like this for free.

For the test cases with variable fonts, the test suite also passes an argument --variation=M1:0.2;wght:721.5. This means that axis M1 should be set to 0.2 and axis wght to 721.5. Since OpenType.js doesn’t yet implement variable fonts, you could just ignore the --variation argument for now.

[Jolg42]

Hi @brawer You're right! Thank you for that offer :smiley:

So it looks like the SVG needed is not too bad. I see you used symbols, is that a performance optimization or maybe it's required? Do you have other examples of SVG to generate, maybe more complex with repetitions and non-latin characters?

[brawer]

Indeed, the test suite would expect your tool to emit the same structure as in the example, with symbols whose names are derived from the glyph names in the font (or “gid1234” if the font does not contain textual glyph IDs).

Here’s an example with a non-latin repetition, using TestGVAROne.ttf. Although this is a variable font, pre-OpenType 1.8 implementations such as OpenType.js should be able to handle it just fine.

$ some-command --font=fonts/TestGVAROne.ttf --testcase=GVAR-1/400 --render=彌彌彌

This should give this output:

<?xml version="1.0" encoding="UTF-8"?>
<svg version="1.1"
    xmlns="http://www.w3.org/2000/svg"
    xmlns:xlink="http://www.w3.org/1999/xlink"
    viewBox="0 -253 3000 1220">
  <symbol id="GVAR-1/400.gid2" overflow="visible"><path d="M359,-94 L359,555 L926,555 L926,0 Q926,-89 840,-94 Q803,-94 754,-90 L737,-12 Q795,-23 828,-23 Q852,-23 852,12 L852,484 L434,484 L434,-94 Z M180,-90 Q135,-90 86,-86 L74,-8 Q133,-16 168,-16 Q199,-16 212,0 Q225,17 228,62 Q231,107 232,165 Q234,223 234,262 L66,262 L82,547 L238,547 L238,703 L63,703 L63,773 L309,773 L309,477 L152,477 L145,332 L309,332 Q309,263 307,192 Q306,99 300,38 Q293,-24 275,-49 Q257,-75 229,-82 Q201,-90 180,-90 Z M478,8 L441,51 Q469,85 492,125 Q471,162 445,191 L480,238 Q504,207 522,175 Q537,201 551,238 L598,207 Q577,156 555,121 Q581,82 598,51 L555,8 Q541,35 521,66 Q492,24 478,8 Z M477,258 L441,297 Q469,327 493,359 Q470,390 445,422 L480,457 Q493,442 521,404 Q541,434 555,465 L598,434 Q581,393 555,359 Q590,311 598,297 L555,258 Q536,292 521,311 Q502,283 477,258 Z M719,8 L676,55 Q711,88 736,123 Q717,153 684,191 L723,238 Q739,217 751,199 Q763,181 767,174 Q777,192 797,238 L840,207 Q823,159 801,121 Q832,69 842,51 L801,8 Q777,51 766,66 Q734,25 719,8 Z M605,-86 L605,715 L680,715 L680,-86 Z M352,566 L324,629 Q435,652 531,691 L563,637 Q511,612 454,594 Q398,577 352,566 Z M715,258 L676,301 Q711,329 735,358 Q716,385 684,422 L723,457 Q736,442 766,404 Q781,428 797,465 L840,434 Q823,395 801,359 Q829,319 842,297 L801,258 Q786,282 766,309 Q742,281 715,258 Z M352,699 L352,770 L941,770 L941,699 Z M934,566 Q873,591 818,607 Q764,624 723,633 L746,691 Q864,668 961,625 Z"/></symbol>
  <use xlink:href="#GVAR-1/400.gid2" x="0" y="0"/>
  <use xlink:href="#GVAR-1/400.gid2" x="1000" y="0"/>
  <use xlink:href="#GVAR-1/400.gid2" x="2000" y="0"/>
</svg>

Here’s another example that uses the Zycon font. Actually, it seems that OpenType.js cannot parse this font, although other parsers (fonttools, CoreText, HarfBuzz/FreeType) have no problems with it.

$ some-command --testcase=GVAR-5/6 --font=fonts/Zycon.ttf --render=🌝 

should produce this output:

<?xml version="1.0" encoding="UTF-8"?>
<svg version="1.1"
    xmlns="http://www.w3.org/2000/svg"
    xmlns:xlink="http://www.w3.org/1999/xlink"
    viewBox="0 0 1000 1000">
  <symbol id="GVAR-5/6.gid15" overflow="visible"><path d="M989,499 Q989,295 845,152 Q701,8 498,8 Q295,8 151,152 Q7,295 7,499 Q7,702 151,845 Q295,989 498,989 Q701,989 845,845 Q989,702 989,499 Z M964,499 Q964,692 828,829 Q691,966 498,966 Q305,966 168,829 Q32,691 32,498 Q32,305 168,168 Q305,31 498,31 Q691,31 828,168 Q964,305 964,499 Z M774,703 L672,723 L575,692 L633,669 Q633,669 627,660 Q622,651 622,643 Q622,622 638,607 Q655,592 676,592 Q700,592 714,608 Q729,624 729,647 Q729,659 724,669 Q719,680 719,680 Z M430,691 L332,725 L236,703 L286,680 Q286,680 280,665 Q274,649 274,642 Q274,617 292,604 Q310,590 332,590 Q350,590 366,604 Q383,617 383,636 Q383,643 379,656 Q375,669 375,669 Z M607,423 Q607,423 600,441 Q592,459 579,465 Q565,472 548,467 Q530,462 530,462 L501,673 L471,463 Q471,463 450,468 Q430,473 416,466 Q402,459 395,439 Q387,420 387,420 L448,433 Q448,433 467,421 Q486,409 504,409 Q521,409 539,420 Q557,432 557,432 Z M657,296 Q631,281 584,273 Q537,265 499,265 Q445,265 400,279 Q355,292 340,299 Q354,258 400,232 Q446,206 500,206 Q553,206 599,236 Q645,266 657,296 Z"/></symbol>
  <use xlink:href="#GVAR-5/6.gid15" x="0" y="0"/>
</svg>

Thanks in advance for the tool!

[fdb]

I've added a bin/test-render script that creates the SVG file in the correct format.

I've also looked at integrating it into text-rendering-tests, and it shouldn't be too hard. I'll try to submit a PR tomorrow.

[Jolg42]

Thank you @fdb! (https://github.com/nodebox/opentype.js/commit/220f784f003b6807c6ce122aa509b893c1e75a97)

Just one thing, it looks like you used glyph.name here instead of a Glyph ID, I don't know if this can be a problem for emojis for example? If this a problem, I think the easiest would be to only do the "gid" notation. Is it ok for you @brawer?

[brawer]

The test suite works with glyphs names.

[Jolg42]

@brawer Great! Just thinking... Is the name supposed to be from the https://github.com/adobe-type-tools/agl-aglfn specification to make it work?

[panayotoff]

On all rendering tests, the SVG is flipped on the X axis.

[brawer]

The current test reports don’t seem flipped; see README for instructions how to generate them.

[panayotoff]

I ran the example from ./bin/test-render and the output SVG looks flipped. Also, using glyph.path.toSVG() path is upside own.

[brawer]

Well, flipping does not seem to be the reason why OpenType.js currently fails so many test cases. However, looking at the HTML output for test case GLYF-1 rendered by OpenType.js, the SVG does look a bit funky. Formatting differences aside, the paths aren’t in the expected order. For example, the first three path instructions in the output for test case GLYF-1 should be M199,97 L313,97 Q396,97; but OpenType.js produces M199,97 L199,97 L313,97.

Does OpenType.js sometimes shuffle/reverse its output paths, perhaps unintentionally? Or emit the same path twice? Note below how much longer the path from OpenType.js is compared to the expected result. The test suite requires that output paths must be in the same order as in the rendered font.

<h3 id="GLYF-1">GLYF–1: Composite Glyphs</h3>
  <div class="desc">
    The <a href="../fonts/TestGLYFOne.ttf">font</a> for this test case
    has a <em>‘glyf’</em> table whose entry for U+0123 LATIN SMALL
    LETTER G WITH CEDILLA is a <em>composite glyph</em>. It has two
    components: one for the base glyph, and another one for the
    accent. A correct implementation must place the accent on top of the
    g.
  </div>

  <table>
    <tr>
    <th>Expected</th>
    <td class="expected" ns0:font="TestGLYFOne.ttf" ns0:id="GLYF-1/1" ns0:render="ģ"><svg version="1.1" viewBox="0 -455 533 1383"><symbol id="GLYF-1/1.gcommaabove" overflow="visible"><path d="M199,97 L313,97 Q396,97 444,61 Q493,25 493,-36 Q493,-108 428,-151 Q363,-195 255,-195 Q150,-195 90,-156 Q30,-117 30,-49 Q30,25 117,74 Q92,94 92,129 Q92,177 141,220 Q86,266 86,345 Q86,419 134,464 Q183,510 262,510 Q330,510 375,476 L482,489 Q498,490 498,475 Q498,469 494,454 L491,443 L408,443 Q438,400 438,345 Q438,271 389,225 Q341,180 262,180 Q211,180 171,200 Q143,175 143,141 Q143,97 199,97 Z M313,53 L178,53 Q165,53 151,56 Q85,13 85,-46 Q85,-94 131,-122 Q177,-151 255,-151 Q338,-151 388,-119 Q438,-88 438,-36 Q438,53 313,53 Z M387,345 Q387,401 353,434 Q319,467 262,467 Q205,467 171,434 Q137,401 137,345 Q137,289 171,256 Q205,223 262,223 Q319,223 353,256 Q387,289 387,345 Z M268,671 L302,650 Q306,640 306,631 Q306,590 262,590 Q214,590 214,645 Q214,669 230,707 Q246,746 271,780 Q292,777 304,765 Q272,707 268,671 Z" /></symbol><use x="0" y="0" xlink:href="#GLYF-1/1.gcommaabove" /></svg></td>
  </tr>
   
  <tr>
    <th>Observed</th>
    <td class="observed" ns0:id="GLYF-1/1"><svg version="1.1" viewBox="0 -455 533 1383"><symbol id="OBSERVED/GLYF-1/1.gcommaabove" overflow="visible"><path d="M199 97L199 97L313 97L313 97Q396 97 444.50 61L444.50 61L444.50 61Q493 25 493-36L493-36L493-36Q493-108 428-151.50L428-151.50L428-151.50Q363-195 255-195L255-195L255-195Q150-195 90-156L90-156L90-156Q30-117 30-49L30-49L30-49Q30 25 117 74L117 74L117 74Q92 94 92 129L92 129L92 129Q92 177 141 220L141 220L141 220Q86 266 86 345L86 345L86 345Q86 419 134.50 464.50L134.50 464.50L134.50 464.50Q183 510 262 510L262 510L262 510Q330 510 375 476L375 476L482 489L482 489Q498 490 498 475L498 475L498 475Q498 469 494 454L494 454L491 443L408 443L408 443Q438 400 438 345L438 345L438 345Q438 271 389.50 225.50L389.50 225.50L389.50 225.50Q341 180 262 180L262 180L262 180Q211 180 171 200L171 200L171 200Q143 175 143 141L143 141L143 141Q143 97 199 97ZM313 53L313 53L178 53L178 53Q165 53 151 56L151 56L151 56Q85 13 85-46L85-46L85-46Q85-94 131-122.50L131-122.50L131-122.50Q177-151 255-151L255-151L255-151Q338-151 388-119.50L388-119.50L388-119.50Q438-88 438-36L438-36L438-36Q438 53 313 53ZM387 345L387 345L387 345Q387 401 353 434L353 434L353 434Q319 467 262 467L262 467L262 467Q205 467 171 434L171 434L171 434Q137 401 137 345L137 345L137 345Q137 289 171 256L171 256L171 256Q205 223 262 223L262 223L262 223Q319 223 353 256L353 256L353 256Q387 289 387 345ZM268 671L268 671L302 650L302 650Q306 640 306 631L306 631L306 631Q306 590 262 590L262 590L262 590Q214 590 214 645L214 645L214 645Q214 669 230 707.50L230 707.50L230 707.50Q246 746 271 780L271 780L271 780Q292 777 304 765L304 765L304 765Q272 707 268 671Z" /></symbol><use x="0" y="0" xlink:href="#OBSERVED/GLYF-1/1.gcommaabove" /></svg></td>
  </tr>
   
  <tr>
    <th class="conformance-header">Conformance</th>
    <td class="conformance-fail" ns0:id="GLYF-1/1">✖</td>
  </tr>
</table>

[Connum]

To give a quick status update a few years later: OpenType.js is passing more and more tests in the unreleased master state which is going to be 2.0.0 in the future: unicode-text-rendering-report_aa719a7.zip

Several tests are visually OK, but marked as FAILED due to minor rounding differences. I still haven't managed to fix that for all of them, but it has gotten better. I think our rounding is fine, but there must be a reason other libs result in different values.