HTML5: Unicode characters not displaying even with system fallbacks on web

hydroper commented 1 year ago

Godot version

4.1.rc2

System information

Windows 11, gl_compatibility

Issue description

It's an issue on HTML5 export.

I'm not doing any complex string assignment to my Label's text, not messing with code points or anything.

func display_item(item: PS3Item) -> void:
    self.item = item
    var suffix := "" if item.quantity == 1 else " ⨉ " + str(item.quantity)
    $button/content/container/right/label.text = self.item.name + suffix

I've added fallback to a system font for every font I use in my project and re-imported the fonts. The editor didn't ask for which system font to choose; I leaved things as default except for italic and weight. As you can see allow_system_fonts is true for the system font fallback.

The result I get (I'm publishing the project to itch.io; unfortunately I'm not sure how to pass a SharedArrayBuffer allow header locally so that I can test the minimal reproduction project (which in turn doesn't use custom fonts)):

Steps to reproduce

Add a default theme or apply it to a control
Add system font fallbacks as described in the issue, without changing anything except weight (700 for bold) and italic. Use it in that theme.
Add a Label with some text containing ⨉ and add that label to the main 2D scene.

Minimal reproduction project

Unfortunately, I tried using sirv-cli to serve the export locally, but it requires an SharedArrayBuffer header and I'm not sure which is it; I've searched for it.

The following project doesn't use custom fonts, so it's not exactly like my production project, but like I said I'm using system font fallbacks, so there's no reason it doesn't work.

godotfont.zip

hydroper commented 1 year ago

Forgot to to mention it occurs in HTML5 only.

akien-mga commented 1 year ago

System fonts are not supported on HTML5, so this is likely not a bug.

Calinou commented 1 year ago

This is indeed not a bug, as described in the documentation.

@bruvzg We should print a warning when attempting to load a system font when the platform doesn't support it. (That said, this requires the user to look at browser console, which isn't guaranteed to happen given the web platform doesn't make it possible for us to make DevTools appear from JavaScript.)

hydroper commented 1 year ago

Right... so in the future they may be supported, right?

Calinou commented 1 year ago

Right... so in the future they may be supported, right?

There is no stable API that's supported in all browsers to access system fonts yet. This will probably take a while to materialize, if it ever happens.

bruvzg commented 1 year ago

System fonts API is currently available in the desktop Chrome/Edge 103+ only: https://developer.mozilla.org/en-US/docs/Web/API/Window/queryLocalFonts. It's considered an experimental feature, but I guess it might be worth adding support for it, since Chrome/Edge is like 70% of browsers.

akien-mga commented 1 year ago

Usually Godot users aren't satisfied with supporting only 70% of the market, and if the remaining 30% show no fonts, it's a pain. (They already consider Godot 4 unusable due to things like macOS Chrome using OpenGL instead of Metal, or Safari not supporting COEP:credentialless.)

So we can implement the feature, but I don't think anyone should/would rely on it until it gets implemented in at least Safari.

bruvzg commented 1 year ago

desktop Chrome/Edge 103+ only

Quickly tested this API, and seems like it works on Windows only (on macOS and Linux, it's returning empty font list), so support is even more limited. Also, it's showing up permission popup when accessing font list/fonts.

hydroper commented 1 year ago

Curiosity: would it be possible to render a system font through a transparent HTML5 canvas (CPU-based) and send it to the GPU?

Calinou commented 1 year ago

Curiosity: would it be possible to render a system font through a transparent HTML5 canvas (CPU-based) and send it to the GPU?

This would be extremely complex, visually inconsistent with Godot's own rendering and also very slow – you don't want stuttering to occur during gameplay.

rrous commented 1 year ago

I made some testing on itch.io with NotoSansMath-Regular.ttf (not System font). I use math glyps and some of them are OK, some NOT. For me works: \u003D\u2192\u2194\u2225\u007C\u007C\u2220\u25B3\u27C2\u2208\u22C2 Not working: \u00B0\u25A1\u25A2\u25AD I don't know whether the problem is with web server config or web browser (Edge) or Godot itself.

mischiefaaron commented 1 year ago

This is a snag for a project I'm working on right now that is released with a web export. I would like to see the Japanese text that works in other exports, but the characters show up as hexcodes when I run it in both Chrome & Firefox using KDE and Debian.

rrous commented 1 year ago

The problem is probably only with Unicode Fonts sets. They are usually very poorly designed - missing a lot of glyphs :( I found this pretty rich one, so it might help you :): https://www.fontspace.com/freeserif-font-f13277 Search for chars (e.g. U+232B) in Fonts works here: https://www.fileformat.info/

dkaste commented 10 months ago

Curiosity: would it be possible to render a system font through a transparent HTML5 canvas (CPU-based) and send it to the GPU?

This would be extremely complex, visually inconsistent with Godot's own rendering and also very slow – you don't want stuttering to occur during gameplay.

The current Harfbuzz+FreeType text rendering running in WASM is already very slow. There's a good chance using the browser's builtin text rendering would be faster since it is native code.

Also, it's not too complex. I've done it before in a personal project and it was pretty straightforward.

canvas.getContext("2d", { willReadFrequently: true })
ctx.fillText()
ctx.getImageData()
Boom. You have the raw pixel data of the text you wanted.

See my proposal: https://github.com/godotengine/godot-proposals/issues/5741

bruvzg commented 10 months ago

Also, it's not too complex. I've done it before in a personal project and it was pretty straightforward.

It's straightforward to render simple text (and should be already doable via JavaScriptBridge interface). But integrating it with Godot UI system is not (most likely impossible, since you won't be able to extract required data from the browser text renderer).

dkaste commented 10 months ago

@bruvzg What data would be missing? Do you mean sizing information? The TextMetrics API covers that. fontBoundingBox[Ascent,Descent] was AFAIK the main piece that was missing (from Firefox), but that was added recently in Firefox 116.

bruvzg commented 10 months ago

What data would be missing?

No info for bounding boxes and mapping of individual glyph (necessary for carets, selection, highlights, hit testing, other text effect), as well as no ability to render each glyph individually (some RTL effects can move glyphs, text clipping).
No full control over line breaking and alignment.

Note: you can't process complex scripts or/and bidirectional text using multiple fillText calls to render individual characters/words (or measure parts of the text using measureText on a substring), in general the minimal fully independent unit of text is the paragraph.

godotengine / godot