testing utilities

[PhilterPaper]

Hi Johan,

I have manually updated my Text::Layout (on Windows 10) to 0.028_002. The t/ tests run OK, but the tests/ have lots of problems. If you intend for these to be run by random people on strange systems such as Windows (and not just by yourself), you might want to address these issues.

• The most common appear to be that there is no $HOME env variable on Windows, requiring add_fontdirs(...) calls to be changed to add_fontdirs('/Windows/Fonts/'). There is no $HOME/.fonts or /usr/share/fonts/ on Windows, resulting in both Perl errors and TL error messages. This might be feasible if you can reliably discover that you are running on Windows (or not).
• The underline.pl test is calling for font "times normal normal". I vaguely recall having to deal with this (or at least, reporting it), some time ago, but I don't recall what the fix was.
• Some of the requested fonts aren't normally found on Windows, anyway, but you might want to leave naming of alternate fonts to users (add a comment with suggestions, such as arial.ttf replacing ITCGarmond-Light.ttf). FreeSans, FreeSerif, and fireflysung are three others, and there may be more (I already have some oddball stuff installed on my system).
• Finally, there are a bunch of tests reporting that they can't find the Pango.pm module in @ INC (I don't have it installed). Windows users are likely not to have Pango installed by default (at least, with Strawberry Perl), so maybe a test to see if Pango is installed would be in order? It depends on how clean and polished you want the sample tests to appear to be. I guess the same could go for HarfBuzz::Shaper, and even PDF::API2 vs. PDF::Builder.

For your viewing pleasure, I attach some tools I use on PDF::Builder to check things after a major change, and before a CPAN release. 1_pc.pl.txt 2_t-tests.pl.txt 3_tests.pl.txt Obviously they need to renamed to strip off the .txt suffix. On PDF::Builder they're in the tools/ directory. They run OK from your root directory (parent of t/), but may leave PDFs in undesirable places.

1_pc runs PerlCritic, at level 4, and is very picky (even after I had it exclude a lot of stuff). It's up to you whether you want to update your code with their suggestions. 2_t-tests runs all the t tests, and works fine. 3_tests is from PDF::Builder's examples directory. This is the one that gave all the problems described above.

Add: I don't know if I missed it first time around, or it was added in 0.029, but the first t-test needs a -T flag on the command line: 2_t-tests.pl.txt If more tests need flags, a more elegant way might be used.

[PhilterPaper]

The underline.pl "times normal normal" issue may be #8 (missing font registration). Perhaps this is a difference in behavior between Linux and Windows? The test utility may have to be updated.

You also mentioned in #3 that you didn't intend to publish such tests (meant for your use only). I think they would be useful for giving working sample code, encouraging use of the library, but only if they're cleaned up to work on all platforms.

[sciurius]

Yes, that remains a bit of a problem... I'll see what I can improve.

[PhilterPaper]

According to your reply on #8, a Linux system automatically falls back to a built-in fontconfig system (if I understand you correctly). This fontconfig does not exist in Windows, which requires that the font family be registered manually. This major difference in behavior is a problem, as a Perl program using Text::Layout will behave considerably differently on Linux and Windows.

Is it possible to add some code during setup (new() method?) to run on Windows only, to emulate the fallback to fontconfig? The idea would be to align the behavior of Linux and Windows platforms. It would eliminate the need for an application to check which platform it's running on, and do extra work for Windows (i.e., the manual font family registration). Otherwise, if it's too big a project to fully converge the behavior, you at least need to fully document the differences in running on the two platforms, and show some cookbook instructions for code to get around Windows' lack of fontconfig.

[sciurius]

Working with fonts is hard to make platform-agnostic. Corefonts are okay, everything else needs explicit configuration. And different config on Linux, Windows, etc.. Worse, while Windows has a well-defined fonts folder, there are no standard fonts (locations) on Linux. It would be easier to make a fonts registry config for Windows than for Lonx...

If it comes to fallback, you're in trouble anyway. Fallback only guarantees to yield a sensible font and is dependent on the system and config. Maybe it is best to disable the fontconfig fallback on linux? It's better to get an error that your font cannot be found, than an inferior substitute.

[PhilterPaper]

Windows does appear to cram all its fonts into \Windows\Fonts\; I'm surprised that Linux doesn't have a similar standard location! Windows ships with a shipload :-) of .ttf fonts, along with lots of .fon (bitmapped?) and a few .ttc and .CompositeFont files, whatever those are. I have a few .otf, but I think I put them there. There appear to be no T1 fonts, and I don't recall other font files elsewhere, except as put in random places by applications (e.g., Firefox putting CJK .ttf/.otf fonts into \Program Files\Adobe\Acrobat DC\Resource\CIDFont\, also Java and CPAN play this way). Windows scatters around a bunch of .ttf fonts, but I'm not sure they're complete (may be subsets for specific uses, or just redundant).

What fonts come with Windows appear to be quite different from those that come with Linux (as mentioned above). Perhaps a solution, at least for the examples, is to provide separate font name lists for Windows and Linux, in a manner similar to CSS's font-family list, where you might have a generic 'serif' as the last Hail Mary play for both Windows and Linux lists. If you want to work with me on finding similar fonts on Windows to somewhat match what you use in Linux, I'll be happy to try out things.

Regarding "fallback", Linux does work without manually registering fonts, which is why your examples fail on Windows. I'm not sure we're talking about the same thing: some sort of "fontconfig" that's built into Linux, but not Windows. I'll leave it up to you as to whether to search a list of fonts and bail out if the specified one isn't found. Perhaps a diagnostic message that a fallback substitute was used, rather than bailing, would be appropriate? There could be a flag to allow/forbid fallbacks or the use of a list of font names, if a user wants precise control over what font is used. You certainly want to think twice about picking any old random font just to have something show up, but maybe that's desirable in certain cases (but not others).

Perhaps this would work: give separate font-family-like lists for Linux and Windows. If you want precise control over the font used, just have one item in each (e.g., Helvetica and Arial). If you don't mind a "similar" font being used, add some more to the list. If you can't find any of those, blow up if you get to the end of the list, except if a "generic" such as sans-serif if given, in which case the code grabs anything at hand. No need for a allow/forbid fallback switch.

[sciurius]

I'm surprised that Linux doesn't have a similar standard location!

Well, it has. Sort of. Problem is there are many standard locations and different distro's have their own views on where to put the fonts.

For example, Deja-Vu:

Fedora 35           /usr/share/fonts/dejavu-mono-fonts/DejaVuMono.ttf
Debian 10           /usr/share/fonts/truetype/dejavu/DejaVuMono.ttf
Fedora 35           /usr/share/fonts/dejavu-sans-mono-fonts/DejaVuSans.ttf
Debian 10           /usr/share/fonts/truetype/dejavu/DejaVuSans.ttf
Fedora 35           /usr/share/fonts/dejavu-serif-fonts/DejaVuSerif.ttf
Debian 10           /usr/share/fonts/truetype/dejavu/DejaVuSerif.ttf

Theoretically I could search the known locations for the names of some standard fonts (e.g. DejaVu, Droid, Liberation, maybe msttcorefonts).

What fonts come with Windows appear to be quite different from those that come with Linux (as mentioned above). Perhaps a solution, at least for the examples, is to provide separate font name lists for Windows and Linux, in a manner similar to CSS's font-family list, where you might have a generic 'serif' as the last Hail Mary play for both Windows and Linux lists.

Yes, but I would then try to achieve that the family resolves to the same font on all platforms. And even then... The msttcorefonts are old, e.g. the Arial font as distributed by the free msttcorefonts has much less glyphs than the current Windows10 font. So it seems to boil down to same results → same font files.

Regarding "fallback", Linux does work without manually registering fonts, which is why your examples fail on Windows. I'm not sure we're talking about the same thing: some sort of "fontconfig" that's built into Linux, but not Windows.

Precisely that. fontconfig is standard part of modern Linux systems. Even though the software is platform agnostic (“Fontconfig does not: [...] depend on the X Window System in any fashion”) noone seems to have taken the troubles to port it to windows.

Perhaps a diagnostic message that a fallback substitute was used, rather than bailing, would be appropriate? There could be a flag to allow/forbid fallbacks or the use of a list of font names, if a user wants precise control over what font is used.

Yes and no. If the user requests 'serif' or 'sans', she may get an arbitrary but reasonable serif resp. sans font. But if she asks for 'arial' or 'calibri' she may expect to get exactly that font, or bail out.

Perhaps this would work: give separate font-family-like lists for Linux and Windows.

The ChordPro documentation has a page on using Text::Layout::FontConfig, with examples. See..

BTW, did you take a look at the GlobalSerif.CompositeFont et al files?

[PhilterPaper]

fontconfig is standard part of modern Linux systems. Even though the software is platform agnostic (“Fontconfig does not: [...] depend on the X Window System in any fashion”) noone seems to have taken the troubles to port it to windows.

If it's entirely open source, might it be made compatible with Windows? That is, it's not depending on any deeper Linux-only font information, but could get what it needs from .ttf and .otf files? That might make a great project for someone to port it to Perl. I'm guessing that it hasn't been done yet because it's much more complicated than that (Text::Layout::FontConfig is an interface to Linux's fontconfig, and not an emulator?).

did you take a look at the GlobalSerif.CompositeFont et al files?

I hadn't, but just did. They appear to be a mapping of Unicode points to specific font files for families of fonts (e.g., GlobalMonospace lists Courier New for a number of code points in a Monospaced font). Would that be sufficient information for a fontconfig port to do what you need? At least it could give you a generic fallback (monospace, serif, etc.). Of course, if someone has added more fonts on a Windows system, they might want to make use of them.

If the font list contains both a Linux-only and a Windows-only font, and possibly a generic fallback, you could get away with a single list and not separate Linux and Windows lists. E.g., "Helvetica, Arial, sans-serif" should hit Helvetica on Linux, Arial on Windows, and some generic sans serif if neither. If you happen to have true Helvetica on your Windows system, it would pick that.

Add: I see there is a ShiftMediaProject/fontconfig on GitHub, that might be a drop-in Windows replacement for Linux's fontconfig. It's unfortunate that it would be a separate component to install, which means very few Windows users (even those running Perl) are likely to install it, unless someone makes the effort to do something with Alien::Build to make installation seamless, and prereq it on the CPAN build for Text::Layout and possibly other packages.

[PhilterPaper]

I don't think there's a need to hold this ticket open any longer. You're welcome to reopen it if you think there is.