What platforms are supported (or will be supported)?

[corliss]

It looks like Windows, Mac and Linux are currently supported, is this correct?

What would it take to support Xamarin.iOS, Xamarin.Android and Windows 10 Universal?

It would also be very helpful if this is made into a NuGet package.

I'd be happy to help.

[Robmaister]

Right now this project is in it's infancy. Only a small fraction of the HarfBuzz API has been exposed and it's only been tested on Windows and theoretically works on Linux. #6 (or anyone who has a Mac to build HarfBuzz on or point me to a copy distributed with OS X) will add OS X support.

I'm not too familiar with the Xamarin mobile platforms or the new Win 10 Universal stuff, though I will be looking more into UAP for Robmaister/SharpFont#61.

I can put up a NuGet package for this in the next day or so.

I would appreciate any and all help! Work/life have been keeping me pretty busy lately so I haven't had a chance to dig deeper into new platform requirements or even fill out the DllImports for HarfBuzz. I should be pretty quick to respond to PRs and pushing out new package versions, etc. though

[Robmaister]

Oh and to give a very clear answer for the second question in the title, as many platforms as possible.

[corliss]

Thanks for the reply. a) Could you give me a sense of the dependencies needed for HarfBuzz? b) I notice in the sample that there is a method to render to Gdi. Obviously this is not portable. c) Currently the SharpFont.HarfBuzz project references SharpFont. The SharpFont.HarfBuzz.Example project references freetype6, libglib2.0, libharfbuzz-0, libiconv-2.dll, and libintl-8.dll. Which of these are actually required, and why?

[Robmaister]

a) On the managed side, SharpFont.HarfBuzz depends on SharpFont (wraps FreeType) and SharpFont.Dependencies (provides freetype6.dll in different configurations).

On the native side, HarfBuzz depends on FreeType (for metrics and font loading I believe) and GLib (interface with GNOME?)

b) Yeah, that's a function in SharpFont that depends only on System.Drawing, which should be portable-ish under Mono (they have an implementation of GDI+ called libgdiplus). It's a soft dependency on System.Drawing because the rest of the library works just fine if you don't reference System.Drawing in your own project.

I'm totally fine #if guarding it out or whatever is needed to support more platforms if they're more rigid about dependencies. The existing system works fine across desktop platforms though.

c) I grabbed the win32 binaries from the HarfBuzz site and poked at it with dependency walker, the tree looks like this:

libharfbuzz-0.dll -> freetype6.dll, libglib-2.0-0.dll libglib-2.0-0.dll -> libintl-8.dll -> libiconv-2.dll

I haven't looked too deep into why HarfBuzz needs libglib-2.0, but that's where all the dependencies come from. It may be possible to compile a special version of HarfBuzz that doesn't need libglib-2.0 and all the other dependencies go away.

Eventually, I would like to put all these binaries somewhere in SharpFont.Dependencies.

Hopefully that answers all of your questions, if not feel free to ask more :smile: I'm going to bed now so don't expect a response for a few hours.

[Robmaister]

NuGet package is up https://www.nuget.org/packages/SharpFont.HarfBuzz/0.1.1

[corliss]

Forgot to say thanks! I've been busy on other stuff, but will be looking into this again.

Has there been any more progress on supporting multiple platforms? It'd be great to have UWP and iOS for starters. At some point I may dive into this, but the list of dependencies is daunting.

Are there others interested in this by the way? What about the core harfbuzz team, Behdad, etc? If a few interested & knowledgeable people get excited about this a fairly complete package could possibly happen quickly.

Cheers.

[Robmaister]

Sorry about the delay, I just finished wrapping up the last of my college courses. I haven't had the time to actually make any progress on this just yet.

After a quick bit of googling, it looks simple enough to remove the glib dependency by running configure --without-glib http://comments.gmane.org/gmane.comp.freedesktop.harfbuzz/4035 meaning the only two binaries are libharfbuzz-0.dll and freetype6.dll.

I haven't reached out to anyone else yet, but I agree that getting more interest would result in a complete package really quickly. Most of the work that needs to be done is just really repetitive (making DllImport function definitions, wrapping them in class methods) and the more people working on it, the quicker it'll get done.

I may find some time this weekend to do some of that extra work or get the non-glib build working or something.

[corliss]

Great, whittling down to two dll's will make things much easier.

Thinking ahead, a Portable Class Library will make it easy to include the nuget in cross-platform projects. The SQLitePCL.raw project (https://github.com/ericsink/SQLitePCL.raw) is a good example of such a project. It uses the "bait and switch" approach to handling platform dependencies.

[corliss]

Hi - just checking in, it's been a while. Just wondering if you had a chance to look into the non-glib build. If that happens, I'd be happy to help contribute with wrapping more of the API and/or cross-platform support. Cheers.

[khaledhosny]

HarfBuzz uses GLib for Unicode functions (to get few Unicode character properties an normalisation). It can also use ICU for this, or a small bundled library called UCDN. If HarfBuzz is built without GLib or ICU, it will use UCDN. Alternatively you can provide the Unicode callbacks to interface with something else (e.g. if C# comes with some Unicode library).

FreeType is used for loading fonts and glyph metrics etc. You can also build HarfBuzz without FreeType and use HarfBuzz’s internal font functions. You can also provide font callbacks to interface with something that C# provides.

[Robmaister]

Hey all, been very busy with work and life in general. I can't promise any deadlines, but I definitely do want to build a version without glib. I would also ideally expose the callbacks so that it can interface with a native C# thing or whatever people want. I'll give a loose timeframe of about a month or so.

[corliss]

@khaledhosny - thanks for the information. Ideally what we need is a way to enumerate the fonts on any C# platform, and then use HarfBuzz to shape text. I'm leaning towards porting the Freetype font inspection functions to C#. How hard would you say this is?

Further out, how difficult would you say it is to port HarfBuzz completely to portable C#? That is the best of all worlds, How many lines of code are we dealing with?

@Robmaister - no problem. I will try to make some progress in the meantime.

[HinTak]

Font enumeration is an inherently non-portable process. Different platforms have different ApI's for doing that.

C# comes with a very extensive unicode library, under the System.Encoding namespace. Font Validator uses it for processing and checking for encoding issues in the 'name' table, etc. I suppose if you really want to you can hook it up to harfbuzz to replace glib's. "Callback" in c is called reverse-P/Invoke in C#... Again there is an example in Font Validator - Font Validator 2.0's rasterization test is essentially FreeType reverse-P/Invoke'ing a delegate that Font Validator provides.

[corliss]

@HinTak, did you mean System.Text.Encoding? Could you provide a link to font validator's rasterization test? I searched the font validator source but find no references to either System.Encoding or FreeType.

[HinTak]

Sorry, yes. I counted 32 files which imports the System.Text. namespace so it is heavily used. All reference to FreeType is in, both directly and indirectly (i.e. via SharpFont): https://github.com/HinTak/Font-Validator/blob/master/Compat/Compat.cs

The reverse P/Invoke is the code surrounding UnmanagedFunctionPointer and diagnostics_Function.

[corliss]

@HinTak : Thanks. I was looking at the Microsoft repo for Font-Validator, not your private repo.

1. What platforms does this run on, of the following: Winforms, UWP, Xamarin.iOS, Xamarin.Mac, Xamarin.Android?

2. Building with Visual Studio is unclear, in this issue: https://github.com/HinTak/Font-Validator/issues/8. How can I build the library and exe with Visual Studio? A set of detailed steps in the README would be great. I don't mean to be demanding but this is something other users might appreciate as well.

Thanks!

[HinTak]

@corliss : 'upstream' had been inactive since going open, and not expected be active anytime soon.

1. Winform and command line - there is a cocoa gui wrapper on mac os to the command line tool. And I mean cocoa (objective c) - not Xamarin.Mac. That wrapper is on the embedded-python branch, not master.

2. Contribution welcome. Since I don't use visual studio https://github.com/HinTak/Font-Validator/issues/8 is what it is, a plan and an current-state report for a low priority enhancement. It is what it is because nobody is spending time on it any time soon. You are welcomed to have a go.

[khaledhosny]

I'm leaning towards porting the Freetype font inspection functions to C#. How hard would you say this is?

I’d start with using HarfBuzz’s internal font functions, should be a couple lines of code and drop Freetype dependency. The font functions shouldn’t be hard to implement if you have API that provide what you need (character to glyph index, glyph metrics etc).

Further out, how difficult would you say it is to port HarfBuzz completely to portable C#? That is the best of all worlds, How many lines of code are we dealing with?

No idea, though I’d advice against that unless it is something you are committed to going forward. Harfbuzz is under active development, new and bug fixes all the time and the last we need is another, slightly incompatible, OpenType layout engine.

[HinTak]

The last time I counted, FontVal (as in "upstream") was 18 months worth of full time effort - 200 lines 5 80. Call that 80,000 lines of C# code. People said they wanted to rewrite it to python. It didn't happen and hasn't happened. That's a fairly complete font-parsing library, and somewhat comparable in size to freetype. If you have problem finding your way around it and building it with visual studio (if that's your choice of tools), I don't think you are equipped to port freetype or harfbuzz to c#. There is "how hard" for people who can and who want to, and there is "how hard" for people who just want it done but don't know how :-).

I'll say, give it a go and experience it first hand :-). If your question of "how hard" is about trying to get somebody else to start doing it to tell you how, you might be waiting a long time and perhaps also disappointed.

[corliss]

@khaledhosny - thanks for the information. About GLib -

HarfBuzz uses GLib for Unicode functions (to get few Unicode character properties an normalisation). It can also use ICU for this, or a small bundled library called UCDN. If HarfBuzz is built without GLib or ICU, it will use UCDN. Alternatively you can provide the Unicode callbacks to interface with something else (e.g. if C# comes with some Unicode library).

I found this thread and the UCDN project. As you say, HarfBuzz compiles its own version of UCDN when GLib or ICU are not present.

Why does HarfBuzz not just use UCDN always? Is it to allow a developer flexibility in the version of the Unicode specification? Or some other type of customization?

FreeType is used for loading fonts and glyph metrics etc. You can also build HarfBuzz without FreeType and use HarfBuzz’s internal font functions. You can also provide font callbacks to interface with something that C# provides.

As above, are there any tradeoffs when using the internal font functions rather than FreeType?

[khaledhosny]

Why does HarfBuzz not just use UCDN always? Is it to allow a developer flexibility in the version of the Unicode specification? Or some other type of customization?

If you already using another Unicode library you probably want to use it for consistency and to avoid data duplication etc. As you have found already, UCDN is a relatively new addition it was contributed by its authors who didn’t want to depend on ICU or GLib.

As above, are there any tradeoffs when using the internal font functions rather than FreeType?

I use them myself in new projects and so does Chrome (Firefox in comparison uses its own font functions), but also they are a new addition and miss few small features, like not supporting DFONT files on Mac (if you create the face from blob rather than with a table function). If you are not already using Freetype then the internal font functions is a safer bet.

[corliss]

@khaledhosny - Thanks, that's very helpful.

Now I'm back to the issue of font enumeration. Specifically, I would like to enumerate installed fonts in a way that is consistent across platforms, the way a web browser does. The corresponding CSS specification goes into all the complexities, but I haven't found a library that produces something similar. I also need to do font fallback.

I know Pango has font enumeration, but that seems specifically for use with Cairo. FontConfig's docs imply that it is for unix-like systems only.

Font enumeration is implemented by browsers as well as productivity apps, so - is there a font enumeration library that works well with Skia, and maps font files to typefaces/styles uniformly across OSes?

[Robmaister]

@corliss If there isn't one, the Windows side is fairly simple to implement - EnumFontFamilies should have all the data you need. I'd be willing to help implement it in that case.

https://msdn.microsoft.com/en-us/library/windows/desktop/dd162615(v=vs.85).aspx

[HinTak]

Fontconfig runs on windows and Mac os also. Cairo on all platforms depends on it. I also routinely cross-compile cairo for windows and Mac os X. It is a bit of a pain to cross-compile libpng then freetype then fontconfig then pixmap before cairo, for windows and Mac.

[khaledhosny]

AFAIK, there is no library that handles font enumeration and fallback the way CSS specifies it, each browsers rolls its own solution of top of system libraries and implement the CSS spec in varying degrees.

Though FontConfig is cross-platform, it is indeed designed to work on Linux and similar Unices and it doesn’t integrate will on Windows or Mac, and it doesn’t implement the CSS spec either.

Pango is a complete text layout system, you can use it with Cairo, Freetype or GDI (on Windows) an Core Text (on Mac). It handles font enumeration, bidi, line breaking, justification, shaping (using HarfBuzz), so it is few layers up than HarfBuzz. You might find this document useful http://behdad.org/text/, if you haven’t seen it before.

[corliss]

Thanks for the responses.

I have read http://behdad.org/text/ previously and browsed it again. Sadly it doesn't say anything about enumeration and only mentions fallback.

Appreciate all the work over the years to put together the text stack. As a relative newcomer, it seems that font enumeration is a (the?) major missing piece.

For historical reasons OSes and apps expose the same fonts in very different ways. However it seems we're headed towards a more consistent representation by (family, size, width, weight, slope), due to the web. Do you agree? If so, this ought to be embodied in code that can be used universally, just as HarfBuzz has in many cases taken over the job of shaping from the OS.

@Robmaister -

@corliss If there isn't one, the Windows side is fairly simple to implement - EnumFontFamilies should have all the data you need. I'd be willing to help implement it in that case. https://msdn.microsoft.com/en-us/library/windows/desktop/dd162615(v=vs.85).aspx

Would font enumeration then become part of SharpFont.HarfBuzz? I'd be happy to work on various backends as well. System.Drawing.FontFamily.Families is probably the managed equivalent to EnumFontFamilies.

@HinTak and @khaledhosny - do you differ in your views of FontConfig? As you mention, it doesn't implement the CSS spec. But does it make implementing the spec easier?

@khaledhosny -

Pango is a complete text layout system, you can use it with Cairo, Freetype or GDI (on Windows) an Core Text (on Mac). It handles font enumeration, bidi, line breaking, justification, shaping (using HarfBuzz), so it is few layers up than HarfBuzz.

Unfortunately Pango is LGPL and therefore a non-starter for mobile apps.

Quick question: http://behdad.org/text/ mentions the need to detect languages/scripts, for example in the CJK problem. How is this done? Am I correct in assuming that the environment needs to provide this, e.g. via HTML's lang attribute? Or does Unicode itself provide a mechanism, e.g. escape codes?

[HinTak]

To be honest, font enumeration with Windows and Mac OS X is a lot easier than Linux. On windows, fonts are in a flat directory %WINDOWS%/Fonts , and Mac OS's fonts are in only two directories, /Library/Fonts and /System/Library/Fonts . There is no concept of "user" font on windows; on Mac, user fonts are in ~/Library/Fonts . (of course all platforms also support loading specific named font files from an arbitrary directory).

These are much easier than the Linux situation. Hence, configuring fontconfig on windows and Mac OS X is a no-brainer.

[HinTak]

This is drifting extremely off-topic.

A few points:

• font enumeration is much easier on windows and mac os than on Linux. On Linux you can use fontconfig, and use the same abstraction with fontconfig on windows and mac os. Also if you are using Xamarin Studio to do mobile development (how else do you do C# but Xamarin?) , Xamarin's System.Drawing.FontFamily.Families is implemented through fontconfig anyway, so it is fontconfig one way or another, directly or indirectly. There is no need to go on and on about it. Post your questions about fontconfig to the relevant mailing lists at freedesktop. Questions about Xamarin's experimental CoreGraphics based System.Drawing should go to the Xamarin forum.

• LGPL allows dynamic linking with proprietary applications. It is not a problem for mobile development on current devices. Android does dynamic linking since the beginning. Only issue is iOS older than 8 did not support dynamic linking, so you cannot support older i-Devices. Not a big limitation.

• Xamarin's font scaling is done through Pango, Cairo and Freetype, etc on non-windows. Again this is not discussed because it is really not interesting - those need to know about it, read Xamarin's code directly. It is open-sourced. Most people don't need to know, nor want to know. If you know enough about Xamarin to find Xamarin's functionality not satisfactory, you use SharpFont and SharpFont.HarfBuzz . Many of the questions seems to demonstrate a lack of reading or understanding with Xamarin/Mono.

Anyway, I don't think it is appropriate to go off-topic.

[Robmaister]

@corliss I'm not sure it should. The reason why I split SharpFont.HarfBuzz out to a separate repo/assembly is to allow people to use only what they need, instead of having one giant SharpFont.dll that contains bindings for all font libraries in C#.

I would rather keep SharpFont.HarfBuzz containing only a HarfBuzz wrapper, and perhaps move the main SharpFont assembly to SharpFont.FreeType. SharpFont would then become a collection of higher-level functions that depend on one or more of these libraries, or just generally helpful things like cross-platform font enumeration. If there's some other structure that you think would work better, let me know. I'm open to suggestions.

I would also like to come up with a few use-cases so that the higher-level functionality doesn't spiral out of control/scope. I lean more towards the gamedev side of things and would like to include some stuff for building font atlases, etc. Our downstreams include 3 of the most popular C# game engines, so a game dev use-case makes sense. I'm really not sure about the rest of our users. If we want to go this route, I'll open up a new issue for this on the new repo and we can discuss it further there.

@HinTak This repo is still in it's early infancy stage and the question that started this issue is pretty general, trying to figure out what is and isn't included here is still completely valid and appropriate.

.NET Core is also doing away with System.Drawing as it's not cross-platform and has all sorts of design problems, so System.Drawing.FontFamily.Families isn't available to everyone nowadays. Perhaps like their font rendering, it also won't provide enough information for everyone (the reason I build SharpFont in the first place).

There are plenty of people who need access to lower-level font data rather than the "just give me a rendered string in a bitmap" requirements of most people. Since a lot of those libraries tend to black box all the underlying details, some re-implementation or binding of higher-level APIs must be done.

One of the cases I want to support are people who want to render text in games using font signed distance fields or the multi-channel version. In these cases, the GPU does the final rasterization and your vertex buffer is your text layout. Exposing the entire stack is necessary in that case, especially if we want game engines to start supporting internationalization better.

[corliss]

@Robmaister - I agree.

"Font enumeration" was too limited a description of my needs, which are:

1. Platforms: Windows desktop (WinForms), macOS, iOS, Android, Windows UWP. (Linux and Windows WPF are not currently of interest to me, but I'm happy to build something that runs on those as well.)
2. Enumerate the installed fonts in two ways: a) Physical fonts, i.e. font files. b) Abstract fonts, which have the properties (family, width, weight, slope). A font selector can be built from this enumeration.
3. Support the five generic font families (monospace, serif, sans-serif, cursive and fantasy.)
4. Resolve an abstract font (see above for definition) for a language/script. The result is a set of physical fonts, in fallback order. Calling code just needs to try each of these in sequence. This is how font fallback can be implemented.
5. Given two systems - possibly running different OS'es - but with the same set of installed fonts, resolving an abstract font must produce identical results.

To be clear, this does not involve any code for layout, shaping, rendering or hit testing. The calling code can choose what technology to use for those.

Does this match your needs as well?

[HinTak]

Not interested in further off-topic discussions.

[Robmaister]

@corliss It's something I'd be willing to maintain and help develop a bit. Some details on a use-case would be nice so that we know how to focus it, though that can wait until things are restructured.

I looked more into nuget and renaming/forwarding packages, and it looks like there's no real support. So if we were to restructure the whole project I'd have to get several downstreams to point SharpFont to SharpFont.FreeType, many will be confused when SharpFont doesn't give them the assemblies that they want. I've seen this happen with NUnit before so it's not entirely unheard of.

I'll continue looking into making this transition easier for users, hang tight until then and we can discuss it in much more detail on another issue.

@HinTak Feel free to unsubscribe from this issue then. Right now this project covers maybe 10% of the HarfBuzz API and there are like 4 or 5 people interested in having anything to do with the project.

I'm not looking to slot every part of a conversation into separate issues just yet, it's a lot more productive to have one or two larger conversations to do the initial/early stage planning for a new project.

[HinTak]

I already unsubscribed - you @ me resubscribed me. Please don't do that. I am unsubscribing again now.

[corliss]

My use case is pretty standard: a document-based application that supports editable rich text. To do this correctly from first principles involves the entire font stack, starting with font enumeration as described above. An author's font choice is a strong hint. Font fallback is required when a font is unavailable or does not have the necessary glyphs. Doing this consistently across OSes.

[GregoryMorse]

If I am not mistaken, font enumeration is not allowed in App Store apps using the previously mentioned API or its System.Drawing wrapper. However, DirectX accessible easily via SharpDX as shown in this CocosSharp example here allows font enumeration in Windows for the phone: https://github.com/mono/CocosSharp/blob/master/src/Nodes/Labels/CCLabel-Windows.cs