ARABIC END OF AYAH (U+06DD) followed by a number ۝١ is rendered incorrectly

[munzirtaha]

Description

Entering the end of ayah symbol followed by a number, should encircle the number, .e.g. U+06DDU+0661 should be rendered as ۝١ It works correctly in LibreOffice Writer using Noto Sans Arabic font. This failed using typst

Reproduction URL

https://typst.app/project/pFeXLY0VVrRyfu5vU24Say

Operating system

Linux

Typst version

• [ ] I am using the latest version of Typst

[LaurenzV]

That's how it looks to me, is that not how it's supposed to look?

At least it looks pretty similar in the browser:

Although I don't speak Arabic.

[LaurenzV]

I'm pretty sure that's a font issue. Here's what I get using the font "Segoe UI" in Word:

And here the result when I change it to Noto Sans Arabic:

[LaurenzV]

I just tried it in Libre Office Writer, too, and got the exact same result.

[munzirtaha]

@LaurenzV what you have with "Segoe UI" in Word is the correct rendering. May be your "Noto Sans Arabic" font is not updated the latest?

[LaurenzV]

Could you show a picture of how it looks for you?

[munzirtaha]

[LaurenzV]

Not sure then, would have to look into it. But in any case, if something is wrong then it's probably a bug in rustybuzz, because that's the library we use for shaping.

[munzirtaha]

Umm! This is getting nasy then. With rustybuzz announcing it "is effectively dead". What's the easiest way to test the library directly?

[LaurenzV]

I'll look into it when I find the time to do so.

[munzirtaha]

Doesn't rustybuzz handle the shaping part only? I am not sure about the rendering part. The shape function output matches that of harfbuzz as can be seen here:

$ ./shape --font-file NotoSansArabic-Regular.ttf --text "۝١"
uni0661.small=1@538,80+0|uni06DD=0+1370

$ hb-shape NotoSansArabic-Regular.ttf "۝١"
[uni0661.small=1@538,80+0|uni06DD=0+1370]

[LaurenzV]

It only handles the shaping part.

But the rendering with harfbuzz looks wrong for me too (even though different to what appears in Typst):

hb-view NotoSansArabic-Regular.ttf "۝١"

So not sure, either a harfbuzz issue or something is wrong with the font (or something is wrong with my computer).

[munzirtaha]

Although I don't speak Arabic.

May be I should explain the problem in a simplified way and remove any part that requires any Arabic knowledge. In Unicode terms this is called "subtending marks". In this case, the first character is just a circle and the second is the number one. The circle is empty but when followed by a number, the fonts and the shaping engine has their logic to substitute the number one with a smaller version that fits inside the circle and move it to fit perfectly. Of course not all the fonts support the full range of Arabic features, but as you figured out "Segoe UI" supports subtending, and you can do your tests with it.

That's said, assuming your version of noto fonts are current, try the --direction option so as to prevent incorrect auto-detection. Also, try other fonts like Amiri, Calibri, Traditional Arabic, Arabic Typesetting. Whichever works for you is enough to compare. Let's even use the Latin number "1" instead of the Arabic one.

$ pacman -Q harfbuzz
harfbuzz 8.3.0-2
$ hb-info NotoSansArabic-Regular.ttf |rg Version
Version: Version 2.012; ttfautohint (v1.8.4.7-5d5b)
$ hb-view --direction=ltr NotoSansArabic-Regular.ttf -u 6dd,661

$ hb-view --direction=ltr NotoSansArabic-Regular.ttf -u 6dd,31

hb-view --direction=ltr Amiri-Regular.ttf -u 6dd,31

Let me know how it goes, and if you need any help we can ask the harfbuzz gurus for advice.

[LaurenzV]

Okay, for some reason the font on Google Fonts was veeery outdated. 😅 Even though I downloaded a fresh copy... But downloading it directly from GitHub I can now reproduce it.

So yeah, maybe a rendering issue after all. But as I said, I don't know when I can investigate it, will probably take a while, since I'm also still relatively new in that area.

[munzirtaha]

Yes, I understand. But after these findings, can you point to which library could be the cause and whether the upstream label is still correct?

[LaurenzV]

Actually, nevermind, seems like it isn't a bug, for some reason Typst was just still loading an old Noto Arabic for me... So here is what I did:

1) Delete all local instances of Noto Sans Arabic. Verify that Typst isn't able to detect any instances of Noto Sans Arabic by invoking cargo run -- fonts

2) Download NotoSansArabic-Regular.otf from here: https://notofonts.github.io/

3) Put it in some folder on your PC.

4) Create the following file;

#set page(width: auto, height: auto, margin: 30pt)
#set text(size: 40pt, font: "Noto Sans Arabic")

۝١

4) Run it with cargo run -- c --font-path PATH_TO_YOUR_FOLDER typst.typ typst.png.

5) Result:

[munzirtaha]

Umm! I am using Arch Linux which already uses the latest version of noto fonts but I see now that the bug is fixed in the latest commits after the release. I am not sure which one but my current version of typst

> typst -V
typst 0.10.0 (70ca0d25)
typst compile noto.typ noto.png

whereas,

> ./typst -V
typst 0.10.0 (f5a2a9a9)
typst compile noto.typ noto.png

Anyway, I am glad that it's been fixed and thanks to you and whoever fixed it.

[munzirtaha]

Just update the version of the font on typst.app before closing.

[Enivex]

Just update the version of the font on typst.app before closing.

The web app has a different repository https://github.com/typst/webapp-issues/issues

It would be better if you created an issue there to update the font.

Edit: Remember to link this issue.