Implement text shaping and bi-di (Rendering RTL and other CTL text)

0x416c69 commented 3 years ago

Original post from https://github.com/ocornut/imgui/issues/3793#issuecomment-860695396

The biggest set back for me (and couple of others) which prevent us to use ImGui in production for our players is how limited its text rendering is and the lack of support for complex text layout (shaping, bi-di, etc). This is because the majority of my players write in Persian (which is basically the Arabic script)

So I've searched on google and from #1228 I knew I had to implement this myself.

So I went ahead and did.

example_win32_directx9_2021-06-14_17-25-41

I had to heavily modify ImGui's freetype integration, in fact glyph range and merge mode no longer work because when you do text shaping, you have to load all the glyphs, some of the glyphs are not mapped to any codepoint since they are ligatures. Then I had to rewrite almost every part of the text rendering functionality resulting in triple the amount of the current code written for text rendering. There are many caching and tricks in place to prevent shaping and other stuff in each frame.

https://user-images.githubusercontent.com/10671711/121898390-9238f100-cd38-11eb-85a6-0a9e151fbacc.mp4

Word wrapping was not easy to get right... Since I fully support bi-di texts, word wrapping is quite complicated and there wasn't much resource about this online.

And if that wasn't enough

https://user-images.githubusercontent.com/10671711/121898642-d926e680-cd38-11eb-921e-952b92f258ff.mp4

stb_textedit is not meant to do RTL or at least I haven't seen anyone use stb_textedit for RTL input widget. So I spent a good amount of time making stb_textedit work alongside RTL and bi-di text. The blinking cursor was also very complicated to do and selections are no longer just one rectangle for each line and as you can see in the video, although there's still a single Stb.select_start and Stb.select_end, multiple portions of text are being selected and it's because of bi-di. I've also added right align to input widget.

Honestly this wasn't an easy task and since ImGui's codebase was not ready for this I had to heavily edit ImGui itself. I've also separated the atlas for each font and the font texture get built in another thread (async load) so it will no longer block the rendering thread. (This wasn't required but I did it to speed up the loading)

Open source libraries used: HarfBuzz (for shaping), SheenBidi (for bi-di text), FreeType

I'll put the link to the repository here after I've refactored some parts of the code.

Hoping to get complex layout text rendering on our favorite GUI library soon ❤️

ifarbod commented 3 years ago

This is big! Good job @0x416c69, I'm looking forward to this!

There are some big game engines and enterprise applications using Dear ImGui, but having no complex text rendering limits their i18n solutions to few languages, hopefully, this will mitigate that.

mgood7123 commented 2 years ago

is there a pull request for this?

lokinmodar commented 2 years ago

I wanna know too

is there a pull request for this?

ocornut commented 2 years ago

Hello @0x416c69 ,

I'll put the link to the repository here after I've refactored some parts of the code.

Have you had a chance to post this? Would be curious to see the changes you made to make it happens. Even based on an old version it would be useful to have them published somewhere.

0x416c69 commented 2 years ago

Have you had a chance to post this?

Unfortunately not yet. I've finished it (actually rewritten it) and I've been caught up finishing my project which uses this version which made me to have my own fork of ImGui and stop getting the updates from the upstream branch which is sad.

I'm really busy trying to get my project to release which should happen soon and after that I'll upload the code on a repo or I'll probably apply the diffs to v1.83 (my fork is based on this version)

Hopefully if I can manage to fit it in the ImGui's codebase with ImGui's standards as an optional extension (like the freetype integration) that'd be great since I won't have to stay on an old version myself.

ocornut commented 2 years ago

Thanks for your answer. My humble suggestion, would be, as step 1, it would be useful to release what you have, even in a "as-in" half finished state and even if designed in a way that makes it hard to upgrade, so the example is out there. At least to me it would be a useful data point to think about. I likely won't act directly on it soon, but the information may help steer some upcoming technical design a little. And I might be able to suggest way to design it in a way that would facilitate upgrades or be more likely to be adopted in main line one day. (You may share privately if you think that's more adequate).

dbechrd commented 2 years ago

@0x416c69 What is the status of this? Has anything been shared privately, or do you think you'll ever be able to post the work done so far publicly (i.e. do you own it, or does your employer and it's proprietary?) It's been 6 months, so I'm just curious what progress, if any, has been made. Even having a hacked together solution based on an older version of ImGui would be very instructive and could allow others to build a more robust solution based on your initial findings.

saudgl commented 1 year ago

Any update bro. Could pls Just post even if not completed

Jony01 commented 1 year ago

Is there any progress with RtL in imgui? @0x416c69 What about your branch?

0x416c69 commented 1 year ago

I'm sorry that I couldn't reply back, the source code is indeed proprietary and under NDA contract. This is finished and ready, I'm trying to get permission to share the source code with @ocornut so that it'll be possible to lay better foundations for this feature in ImGui's codebase.

This might however take a while so if someone wants to implement this, they can do so by using the open source libraries I've mentioned in my original post, it takes some effort but it's certainly possible.

dbechrd commented 1 year ago

@0x416c69 Understood. Thanks for the effort and the response here!