search

mu-editor / mu

A small, simple editor for beginner Python programmers. Written in Python and Qt5.
http://codewith.mu
GNU General Public License v3.0
1.41k stars 435 forks source link

Incorrect formatting of "right to left" language strings. #752

Open ntoll opened 5 years ago

ntoll commented 5 years ago

This was originally reported by @Farshidmi on our Gitter chat channel.

The problem is that when creating strings that contain languages that read from right-to-left (like Persian) then Mu (or more likely, QScintilla) doesn't handle things correctly. The picture below shows how the Persian script bleeds out of the quotation marks which delineate the string.

fasid_issue_windows10

(It's important to note, that the program output is correct though).

What should the editor really look like? Well, @Farshidmi has taken a screen shot of a side-by-side comparison with Notepad++ (which handles things correctly).

farsid_text_in_mu

carlosperate commented 5 years ago

Just as a quick note, Notepad++ is also based on Scintilla, so perhaps the bug is in the QScintilla or PyQt wrappers.

ntoll commented 5 years ago

Scintilla does not properly support right-to-left languages like Arabic and Hebrew. While text in these languages may appear correct, it is not possible to interact with this text as is normal with other editing components.

See: https://scintilla.sourceforge.io/ScintillaDoc.html

carlosperate commented 5 years ago

Interesting, I wonder how Notepad++ has dealt with this then 🤔

ntoll commented 5 years ago

@carlosperate that's interesting... so why does Notepad++ work?

ntoll commented 5 years ago

@carlosperate hahaha.. OK... we're on the same page. Perhaps the outcome for us is to investigate Notepad++'s work-around.

carlosperate commented 5 years ago

It sounds like Scintilla should perhaps still display the text correctly, I think the main issue is interacting with it (text selection and things like that).

I would search online for the simplest possible scintilla and qscintilla examples possible and see if the text is displayed correctly in one of those.

ntoll commented 5 years ago

OK... I see what's going on. There are different versions of the Scintilla website and they're not in sync. The same document on scintilla.org (updated in 2018) says this:

Scintilla provides only limited experimental support on Windows for right-to-left languages like Arabic and Hebrew. While text in these languages may appear correct, interaction with this text may not work correctly as occurs with other editors.

So it looks like it's definitely a problem they (upstream) are aware of.

tjguk commented 5 years ago

I've just tried it in SciTE which is effectively the reference implementation of Scintilla (and, incidentally, my default editor). On Windows, obviously,

I don't have a particular problem: I used Google translate as the OP suggested and copied/pasted the resulting Persian text into SciTE as print("سلام نړۍ")

I could select and paste into, eg, a Word document apparently correctly. When selecting the selected text reversed, although the resulting paste was correct. Whether that was an error or merely conventional for selecting rtl text, I'm not sure.

tjguk commented 5 years ago

FWIW it's clear that the implementation is not complete; in Word (which I'll take to be correct), the text cursor switches direction in the rtl portion of the text; in SciTE it reverses the underlying string. But I don't know how significant this all is.

ntoll commented 5 years ago

@tjguk aha... thank you. So, if QScintilla is up to date with Scintilla, and we use the latest QScintilla in Mu, then perhaps it should "just work" (tm) on Windows. ;-)

tjguk commented 5 years ago

NB https://www.scintilla.org/ScintillaHistory.html v4.1.0 (June 2018) and a couple of later refs to "bidirectional"

carlosperate commented 3 years ago

Hopefully this might be resolved in the next version of Qscintilla, so we can check the next time the dependencies are updated for the next alpha/beta release.

dybber commented 3 years ago

I just tried with the most recent Qscintilla (2.11.6, November 2020), which also requires us to update to PyQt5.15.2. The issue is still present, so it doesn't seem like an issue we can solve now.

I wanted to try the SciTE editor, to check whether the issue is present in their own editor (https://www.scintilla.org/SciTE.html) However, on Mac OS X it's apparently $42, so I think someone on Win/Linux should test that.

dybber commented 3 years ago

Oh, I see @tjguk already did check that it works on SciTE, then I don't have any idea about how to proceed here.

ntoll commented 3 years ago

This certainly feels like an upstream problem. I'm also unsure what to suggest as a next step. As far as I can tell this has a limited blast radius of Windows QScintilla..??