Show Non English text in scenic

[thantez]

Hi. I tried to show Persian text by text primitive in a Scenic app but Scenic doesn't display it well.

This is my code:

g
|> text("متن نمونه", t: {0, 700}, fill: :white, font: @font_hash)

I described issue in this Stack Overflow post: https://stackoverflow.com/questions/60906959/show-non-english-text-in-scenic

[boydm]

Totally understand the issue. The problem is that the font rendering engine is quite simplistic at this time and doesn't support right-to-left, up-down, character clusters, automatic ligatures and such.

In fact, this is one of the big reasons it is not 1.0. The font rendering in particular is incomplete. I would really love to base font rendering it on Slug or something like that instead of the old-school font atlas technique, which makes all of this so difficult, but that isn't doable in an open source project at this time.

If there is need, I could be persuaded to add right-to-left, but this is the first request I've seen for right-to-left. At least you aren't asking for upward slanting right-to-left! knowing how to position the ligatures vertically to make it look right always made my head hurt to think about...

[boydm]

More thoughts... I have no problem adding right-to-left, although it is a pile of work going back into both the font metadata parser and the renderer and is impossible for me to do without partnering with someone who natively reads/writes it.

We should talk about how it should work since I don't feel qualified to just guess. Then, as things currently stand it needs to fit in my other work schedules, which have been surprisingly crammed. Also would need to possibly partner with someone (you?) to get it done.

[boydm]

Last thought for now.

Please post a link to an open true-type font that you would use. Something natively right-to-left that would look good at a relatively low resolution and please doesn't slope up and to the left. Then I can play around with parsing the tables and see if I can capture the direction out of it. That would be the first step...

[thantez]

Thanks for your attention and acting for adding RTL feature. I will be proud to helping you about this subject.

About Persian Open Source font, there is a list of them. But I recommend this font: https://github.com/rastikerdar/vazir-font and this is Vazir's Monospaced mode (that probably is better than regular mode): https://github.com/rastikerdar/vazir-code-font

[boydm]

Well... No promises I can get it in any time soon. It is a pile of work.

Can I ask what you are trying to build with it? Will help me prioritize the work.

[thantez]

Excellent. No problem.

Muslims have a holy book called the Qur'an, whose language is Arabic and whose Persian translation is available.

So, I had an idea to ​​writing a program that would show me a verse from this book with a translation every day when I turn on the computer (or when Linux will boot).

Here or here I tried to implement this idea and used the images on the internet to display the verse itself. For showing the Persian translation, I wanted to use text primitive, which I encountered with this problem.

Another solution of this problem is converting text of translation of each verse to an image (with Mogrify) and then show that image. This solution costs more than showing the sentence as text in scenic.

Anyway, I will wait for you to add this feature to scenic. Thank you for all your efforts.

[jthacker]

I'd like to express interest in this feature as well and can potentially help contribute if given the right direction. My goal is to render simplified Chinese characters. I can live without the prettiest rendering for this application. I tried a bunch of TTF fonts that all failed at the metric extraction phase. I was successful with one (https://www.google.com/get/noto/#mono-mono), however it just renders as rectangular outlines in the app. From your conversation above, it sounds like the font rendering engine may be the limiting factor here. My hope is that Chinese characters have less of the complex requirements like right to left or automatic ligatures that we might be really close to making this happen. Let me know if you think this is feasible and or if you have any pointers for how I might efficiently approach the problem.

[boydm]

Hi @jthacker.

I appreciate the offer to help out. This is a rather difficult subject, so maybe we should start by talking about the metrics extraction. Would probably help me get my head wrapped around it again too. Up for a call next week?

[niklasnisbeth]

This is a difficult problem. Each script will bring its own set of gotchas, I think. There is already a great free software solution in the form of the HarfBuzz library. I am not sure exactly what kind of dependencies it will bring in or how much work it would be to include it, but I think it is close to the gold standard for this stuff.

[niklasnisbeth]

I've been digging some more into Harfbuzz and the text rendering in Scenic today. I don't know a lot about text rendering, and I'm not much of a C programmer, so I'm not sure I'd be able to do all this work even disregarding time constraints. But I'm interested in Scenic and localization (and have some knowledge of beautiful human things like perso-arabic script from before bleeping computers took over my life, but I digress), so I thought I'd go for a dive. Apologies if this is basic stuff for anyone reading.

Harfbuzz doesn't render text as such. It does what's called "shaping", which as I understand it is picking out the needed glyphs and giving them exact positions based on a string and a font. For non-western scripts this is often non-trivial. The shaped string is then passed on to a library like Freetype which outputs pixels.

Harfbuzz is not exactly tied to Freetype, but it needs to be integrated with a font rasterizing library, and for Freetype that work is already done. It's possible that it could be integrated with the font rasterizer Scenic uses, but I think it'd be quite a bit more work. To adapt the example here, we'd need to implement draw_glyph and also bridge the Harfbuzz font type to STB's internal representation. At least this is what I get from a few hours of digging with not a lot of context (beyond this).

It looks like the most primitive graphics in Scenic are drawn by a library called Nanovg. This is baked into the driver source code, so basically forked? Has it diverged? Upstream Nanovg does have optional support for Freetype, which I think would simplify integrating Harfbuzz shaping. But I am guessing this STB renderer was picked over Freetype for reasons of simplicity and resource usage (as far as I know Freetype is considered a "big" library?), and maybe Scenic doesn't want to switch (or add Freetype as a compile-time option) for a feature that might not see much use.