search

LemmyNet / lemmy-ui

The official web app for lemmy.
https://join-lemmy.org/
GNU Affero General Public License v3.0
879 stars 334 forks source link

Add bidi / RTL support #347

Open ahangarha opened 3 years ago

ahangarha commented 3 years ago

Is your proposal related to a problem?

Considering the very nature of social media platforms (like Lemmy), people with different languages would create content. It becomes problematic when one creates content in RTL languages (like Persian, Arabic, Urdu,...). Lemmy is not ready to render RTL (or even mixed RTL/LTR) text properly.

Describe the solution you'd like

There are two approaches in general. One is through HTML modification which can be done by add dir="auto" attribute to elements containing user generated text or by wrapping it in <bdi> tags. The other approach would be through CSS by adding unicode-bidi: plaintext. In either of approaches, there should be avoidance in using absolute positioning or spacing. For example instead of text-align: left we should use text-align: start or instead of padding-left we should use padding-inline-start unless there be some strong UX reason.

Describe alternatives you've considered

--

Additional context

Just keep in mind that bidi (bidirectional text) is different from using RTL or LTR in general based on the language user choose for the interface.

I have opened this issue here not on back-end because I think this can be managed on front-end.

Todo

dessalines commented 3 years ago

I have no familiarity with this, but anyone should feel free to give it a try.

Nutomic commented 3 years ago

I found this page which gives a lot ofdetails on how bidirectional text works: https://www.w3.org/International/articles/inline-bidi-markup/

cadars commented 3 years ago

Another great resource: https://rtlstyling.com/posts/rtl-styling

ahangarha commented 3 years ago

Another great resource: https://rtlstyling.com/posts/rtl-styling

Please notice that bidi is not about RTL. One can use RTL if it is known that the content is RTL. We are talking about a social media in which people may write text in different languages and even wrote mixed content. We don't have and we don't want to have rich text editor in which the user can specify which direction should be applied on a specific portion of the text. We want to handle this situation automatically which is called bidirectional text support.

In most of the cases, adding dir="auto" to the element which contains the text is the trick. In addition we need to use other values that absolute left|right for text alignment or padding/margin or even borders.

cadars commented 3 years ago

In most of the cases, adding dir="auto" to the element which contains the text is the trick.

Yes, to each element if you want true mixed content support. (And to the top element only for nested elements like ul or blockquote https://codepen.io/cadars/pen/poeXMXZ )

I did a quick review of what would be needed to fully support RTL languages in lemmy-ui:

  1. Editor: add dir="auto" to all elements users can create, like @ahangarha suggested.
  2. Add the proper lang attribute to the html element when switching the language. For now it's always lang="en".
  3. Add dir="rtl" to the html element for RTL languages.
  4. Switch to logical CSS properties:
ahangarha commented 3 years ago

Things like RTLCSS wouldn't help as per my understanding. We need to modify certain styles. For example:

cadars commented 3 years ago

In an ideal world, yes, but lemmy-ui uses Bootstrap themes, and rewriting them is no small feat; so using a “converter”, while being a less elegant solution, sounds way more practical.

ahangarha commented 3 years ago

Let's say some bidifix. I don't know when I would find time to work on this, but it should easy to implement.

dessalines commented 2 years ago

Whoever takes this on, I want to mention that lemmy-ui uses bootstrap, and it has RTL instructions here: https://getbootstrap.com/docs/5.0/getting-started/rtl/

ahangarha commented 2 years ago

Whoever takes this on, I want to mention that lemmy-ui uses bootstrap, and it has RTL instructions here: https://getbootstrap.com/docs/5.0/getting-started/rtl/

Again, this is for explicitly define direction. Such solutions won't work for us. We need dynamic directions based on the content.

jayabie commented 2 years ago

Any update on this case guys? We need the RTL ASAP please.

dessalines commented 2 years ago

PRs adding this welcome.

ahangarha commented 2 years ago

I checked dependencies. Lemmy uses markdown-it for handling markdown. I have success in convincing marked to add bidi support (they made a plugin for it) but still struggling with developers on markdown-it.

There are some workaround. If lemmy developers are ok with adding this feature in two steps, I think we can start working on it. We can make markdown sections to follow the first character's direction till markdown-it add bidi support or a plugin be made for it. then we can remove the relevant modification for these section since it will be fully handled by markdown parser.

If it is ok, I try to find some time to create local development environment.

dessalines commented 2 years ago

Sweet, let me know if you need any help setting up a development environment.

ahangarha commented 2 years ago

In few week I will come back again. First I try to do it on my own and if I failed, I will seek help.

ahangarha commented 2 years ago

I just wanted to inform you that we are working on a plugin for markdown-it which is used in this project for parsing md to html. This plugin called markdown-it-bidi and is under development. not ready to be used in production but I think it is on the track.

I decided to do this so that not only we can help lemmy to fix this issue but many more projects can benefit from. It is under LGPL which I think is very good :). So if I didn't follow up here, that was the reason. Hopefully in sometimes we can add that in this project and...

If anyone interested, please check the code and share your thoughts.

jayabie commented 2 years ago

@dessalines @ahangarha Friends, no progress with this yet?

ahangarha commented 2 years ago

@jayabie I am looking for an opportunity to open some time for it.

jayabie commented 2 years ago

@dessalines @ahangarha

No update my friends?

ahangarha commented 2 years ago

@jayabie I am going to try now. As long as there is no nested elements (like list within list) the plugin is usable even now.

But let me try it on the actual project in development environment. If successful, I send PR. Would it be fine? I mean within 2 days I might ask to assign this issue to me. Would it be fine?

jayabie commented 2 years ago

Hello @ahangarha

I'm just a guy seeking for RTL, i think you should ask @dessalines if any extra permissions needed.

Nutomic commented 2 years ago

But let me try it on the actual project in development environment. If successful, I send PR. Would it be fine? I mean within 2 days I might ask to assign this issue to me. Would it be fine?

@ahangarha There is no rush, no one else is working on this. Just take your time and make a pull request when you are ready. Also feel free to ask if anything is unclear.

ahangarha commented 2 years ago

@Nutomic I have issue in building backend. How can I get help? I don't think here is a good place.

Nutomic commented 2 years ago

@ahangarha What error do you get? Make sure to run git submodule init && git submodule update before build. And follow the documentation.

https://join-lemmy.org/docs/en/contributing/contributing.html

ahangarha commented 2 years ago

This was and still is the error:

➜  lemmy git:(main) cargo check
   Compiling lemmy_utils v0.16.5 (/****/lemmy/crates/utils)
    Checking futures v0.3.21
    Checking actix-server v2.1.1
    Checking opentelemetry v0.17.0
error: failed to run custom build command for `lemmy_utils v0.16.5 (/****/lemmy/crates/utils)`

Caused by:
  process didn't exit successfully: `/****/lemmy/target/debug/build/lemmy_utils-53c738f32a60fcf3/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-changed=translations/email/en.json
  cargo:rerun-if-changed=translations/email/pt.json
  cargo:rerun-if-changed=translations/email/ko.json
  cargo:rerun-if-changed=translations/email/fi.json

  --- stderr
  Error: Fmt(Os { code: 2, kind: NotFound, message: "No such file or directory" })
warning: build failed, waiting for other jobs to finish...
error: build failed

It seems it cannot get translations.

Nutomic commented 2 years ago

So you are missing the submodule. Did you run the commands from my previous comment? And what does git status show?

ahangarha commented 2 years ago

Yes. I ran.

I ran clone with --recursive and also I ran the commands you gave me. Nothing happened. see:

➜  lemmy git:(main) git submodule init && git submodule update
➜  lemmy git:(main)
➜  lemmy git:(main) git submodule status                      
 3f86b5c40796fa83054e2226e36effff3b93198a crates/utils/translations (remotes/origin/lemmy-ml-registration-message-252-g3f86b5c)
dessalines commented 2 years ago

We also saw issues with that when people were running old versions of rust, or not properly using rustup to get new rust versions.

If the files are there, make sure you're at least on rust 1.60.

What does rustc --version say?

dessalines commented 2 years ago

Also lmk if I can help in any way... we desperately need RTL, but I have no experience with it, how it should work, or how to test it.

ahangarha commented 2 years ago

My rust version is 1.59.0.

For RTL (I would say bidi not RTL) it depends. I had a presentation about this in LinuxAppSummit 2022. I have to prepare its video and publish. Otherwise, we need to spend some time to work on it. Sometimes it gets tricky.

dessalines commented 2 years ago

Why is that submodule on such an old commit.

run git submodule update --remote

If that doesn't work, delete the folder and run this exactly:

git clone https://github.com/lemmynet/lemmy --recurse-submodules --remote-submodules

ahangarha commented 2 years ago

I ran the first command. It made some changes but didn't fix the issue.

I removed the repo and made new clone with the second command but again the same issue. These are the last few lines of the result:

    Checking bcrypt v0.12.1
   Compiling diesel v1.4.8
   Compiling darling v0.13.4
   Compiling darling v0.14.1
    Checking comrak v0.12.1
   Compiling string_cache_codegen v0.5.2
   Compiling phf_codegen v0.8.0
error: failed to run custom build command for `lemmy_utils v0.16.5 (/****/lemmy2/crates/utils)`

Caused by:
  process didn't exit successfully: `/****/lemmy2/target/debug/build/lemmy_utils-53c738f32a60fcf3/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-changed=translations/email/en.json
  cargo:rerun-if-changed=translations/email/fi.json
  cargo:rerun-if-changed=translations/email/ko.json
  cargo:rerun-if-changed=translations/email/pt.json

  --- stderr
  Error: Fmt(Os { code: 2, kind: NotFound, message: "No such file or directory" })
warning: build failed, waiting for other jobs to finish...
error: build failed
dessalines commented 2 years ago

This is a strange one. You're sure crates/utils/translations/email/ has those files?

Does the docker install have the same issue? If it doesn't, then it has to be your rust install. Otherwise CI would be failing.

ahangarha commented 2 years ago

I see the files in the folder:

➜  lemmy git:(main) ✗ l crates/utils/translations/email 
total 28K
drwxrwxr-x 2 sama sama 4.0K Aug  2 17:05 .
drwxrwxr-x 4 sama sama 4.0K Aug  2 17:05 ..
-rw-rw-r-- 1 sama sama 1.8K Aug  2 17:05 en.json
-rw-rw-r-- 1 sama sama 1.9K Aug  2 17:05 fi.json
-rw-rw-r-- 1 sama sama 1.2K Aug  2 17:05 it.json
-rw-rw-r-- 1 sama sama 1.8K Aug  2 17:05 ko.json
-rw-rw-r-- 1 sama sama 1.8K Aug  2 17:05 pt.json

I updated rust to 1.62.1 and build process could continue.

Nutomic commented 2 years ago

My rust version is 1.59.0.

I just tried to build with that version and it worked without any problems. Which OS/distro do you use?

ahangarha commented 2 years ago

Which OS/distro do you use? Kubuntu 22.04.

Then perhaps cargo needed to be upgraded. I don't know its version that time.

jayabie commented 2 years ago

Hello Friends, @Nutomic @ahangarha

I just wanna check if there is any update?

ahangarha commented 2 years ago

Hi. I couldn't run run the project locally :(

Nutomic commented 2 years ago

@ahangarha Please give some more details, like the commands you ran and errors you get.

ahangarha commented 2 years ago

I think I had issue in connecting font and back. If you have some time, maybe we can have a short video call and fix it. Then I should be able to work.

Nutomic commented 2 years ago

I dont like video calls but we can chat via matrix.org, my username is @nutomic:matrix.org

dessalines commented 2 years ago

Here's the dev chatroom.

dessalines commented 1 year ago

Looks like someone made an RTL Arabic lemmy! https://www.dwanya.co/

Could someone who speaks Arabic contact that site so we can get their changes incorporated into lemmy-ui?

jayabie commented 1 year ago

@dessalines

I do speak Arabic, but i couldn't find anything to use to reach them. they have no contacts at all.

dessalines commented 1 year ago

Maybe you could sign up, and either make a post, or private message the mods? Looks like it has open signups.

Nutomic commented 1 year ago

I already sent a message. But note that their site doesnt properly handle mixed ltr and rtl content, everything is forced rtl. This means it cant be merged into Lemmy in the current state even if they release the source.

dessalines commented 1 year ago

Apparently they paid someone to code this, and keep the source code private. They would have done much better to pay someone with RTL experience to update lemmy-ui itself, then they could still get code updates. As it they'll be stuck on old versions of lemmy-ui.

jayabie commented 1 year ago

Hello guys @dessalines @Nutomic ,

should i move with the signup, or we wait for the response to @Nutomic ?

Yes, after a further check found they didn't share their code.

dessalines commented 1 year ago

Lets wait for them to respond to nutomic first.

xximj commented 1 year ago

Hi guys Sorry to let you wait so long .. The code is available here : https://github.com/xximj/dwanya Works nice with v0.16.6.

I wish i've seen this conversation before working on the code , very interesting information shared by @ahangarha. Thanks for all your efforts guys.