search

ankidroid / Anki-Android

AnkiDroid: Anki flashcards on Android. Your secret trick to achieve superhuman information retention.
GNU General Public License v3.0
8.64k stars 2.23k forks source link

Hebrew cards display formated right aligned, but in reverse character order #1102

Closed hssm closed 9 years ago

hssm commented 9 years ago

Originally reported on Google Code with ID 199

What steps will reproduce the problem?
1.download the hebrew basic vocabulary deck
2.open on ankidroid

What is the expected output? What do you see instead?
The expected output is that the Hebrew characters in the word appear from right to
left, right aligned.

What version of the product are you using? On what operating system?
0.4.2 on Cyanogenmod 6 (Froyo)

Please provide any additional information below.
It is particularly apparent for words that have a final letter (final nun, etc, where
the final letter shows up against the right margin)
The problem is also present in ankionline.

Reported by mordecai.veldt on 2010-09-04 17:47:16

hssm commented 9 years ago 
Big problem indeed.

Just to be sure, can you please try this same deck on Anki Desktop and tell us whether
it works there or not? If yes, we will have to check whether Anki Desktop uses a special
WebView trick.

I am not sure to understand the final letter thing, could you please post a screenshot
here?

Anyone has seen this problem with other right-to-left languages? Casey, is Arabic working?

Reported by nicolas.raoul on 2010-09-05 01:02:55

hssm commented 9 years ago 
basically, the word should look like this: 
אבן
but it is displayed like this:
ןבא

The final letter thing is just what really tipped me off that it was screwed up.

summary of what I've tested:
desktop, program: works
desktop, ankionline (chrome) : works
phone, ankidroid: reversed
phone, android browser: reversed
phone, dolphin browser: reversed

Reported by mordecai.veldt on 2010-09-05 05:52:33

hssm commented 9 years ago 
I've since installed ankimini on the phone, and I have the same problem.

Reported by mordecai.veldt on 2010-09-08 00:29:17

hssm commented 9 years ago 
so the final letter aspect is that some characters have different symbols when placed
at the end of a word vs. earlier in a word. You could compare it to g vs. G however,
i believe you have to type them differently as well (therefore the incorrect placement
of a final letter stands out but is not actually part of the problem... mordecai should
verify this)

Furthermore, this used to be a problem when typing Hebrew into MS Word so maybe you
could find out how they or others in the past got around that. I hope that's helpful,
keep up the good work.

Reported by ErcDaMerc on 2010-09-29 23:18:24

hssm commented 9 years ago 
ErdDaMerc, that is correct, that the final letter are actually different characters
that are typed.  I mentioned it primarily because it's an obvious sign that the word
is written backwards.

Reported by mordecai.veldt on 2010-09-30 11:05:51

hssm commented 9 years ago 
I don't understand why normal web pages display hebrew properly, but the ankionline
has the word backwards.

Reported by mordecai.veldt on 2010-10-27 14:34:34

hssm commented 9 years ago 
A strange thing is that this does not seem to be a usual problem for Android apps.

A brutal solution could be to hard-code <html dir="rtl">

mordecai.veldt, when you wrote "phone, android browser: reversed", do you mean for
any website?

Unfortunately it is not easy for me to test here :-/

Reported by nicolas.raoul on 2011-01-13 10:08:05

hssm commented 9 years ago 
No, it is not reversed for any website.  most (can't think of a wrong one) websites
appear correctly in both the android browser and dolphin browser.  (see comment 6)

Reported by mordecai.veldt on 2011-01-13 10:30:35

hssm commented 9 years ago 
I suspect this is an Android issue, it has been lacking proper Hebrew support for a
while as far as I know.

That seems similar to our issue:
http://stackoverflow.com/questions/4393184/android-2-3-hebrew-support-in-webview

Reported by inigo.aldana on 2011-01-14 07:06:43

hssm commented 9 years ago 
We can try a little hack, if we detect any chunks of hebrew character to surround them
with dir="rtl"...

Reported by inigo.aldana on 2011-01-14 14:02:50

hssm commented 9 years ago 
I have conclusive proof that indicates that this is indeed an Android bug.

When I view the deck from ankiweb via my desktop, the words appear correct, but when
I view the same word from ankiweb via my Android phone browser (dolphin), I see it
wrong.

Doesn't mean we shouldn't fix it, but it means that we can only (as far as I can think
of) fix it by employing a special case for Hebrew, as outlined above.

Reported by inigo.aldana on 2011-01-15 00:54:07

hssm commented 9 years ago 
I would love for this to be fixed since the only decks I study right now are Polytonic
Greek and Hebrew.  The Greek font in Android 2.2 does not cover all the accents so
that deck is not usable and the Hebrew appears left-to-right as mentioned here. Therefore
I can't study at all on my phone, which makes me sad :(  I realize it is largely an
Android bug/problem, but if there is a way to hack in rtl support until Android 2.x
fixes it completely, I would be very grateful! Anyway, thanks for everything and keep
up the good work!

Reported by jasonrhine on 2011-01-17 15:22:28

hssm commented 9 years ago 
Hi Jason,

I plan to work on that, but my hands are full at the moment. Syncing is still buggy
and is in high demand, so I'll try to finish that one first.
I'll get on Hebrew font support as soon as syncing is functional, which should be very
soon, as the release can't be delayed long.

Thanks for the feedback and the testing

Reported by inigo.aldana on 2011-01-18 02:29:52

hssm commented 9 years ago 
Ok, I had a look at this and it seems that android is really bad in this area.
RTL tags don't work, or at least I didn't manage to get them working.

Instead, I'm trying to reverse the order of the characters in the string manually,
while trying to keep the punctuation in place. I don't know anything about Hebrew,
is there a text similar to "quick brown fox..." for Hebrew that I can use for testing?

Kostas

Reported by inigo.aldana on 2011-01-21 22:44:25

hssm commented 9 years ago 
well, not that I'm aware of for quick brown fox, but the punctuation ('dots' 'points'
'niquodot') are treated as their own character, so if you reverse the string, it should
work for the punctation as well.  (currently the points appear, but seem to be 'backwards
order' as the rest)
Even Word(TM) and other 'big' things sometimes struggle with end-of sentence and or
 parenthetical punctuation in RTL languages.

you could try something like this:

שָׁלוֹם. אַתָּה נוֹתֵן ליִ כֶּסֶף? י
the last "apostrophe" which is actually a yod character I included b/c otherwise the
question mark jumps to the other end, like this:
שָׁלוֹם. אַתָּה נוֹתֵן ליִ כֶּסֶף?

This is simple phrase that says "Hello.  You (masculine) are giving me money?" I included
the vowel points, which would be more likely to be used in flashcards than in modern
Hebrew. Each vowel point is treated as a character which follows the character to which
it is applied.  There can be two point characters applied to the same character (for
instance, the ש character has one dot on top to say if it's 's' or 'sh' and also a
vowel mark on the bottom, so that שָׁ is actually three characters to a computer.)

If you need more, let me know.

Reported by mordecai.veldt on 2011-01-21 23:38:45

hssm commented 9 years ago 
Thanks!

That's really useful, especially the warning that the more than one vowels might follow
a consonant.

One more question. The punctuation like question marks, fullstops etc... are they the
same as western ones?

And when you create the flashcards, do you still use the trick with the apostrophe,
ie should we fix trailing western punctuation automatically?

What about numbers? Do they follow RTL or LTR? Can you provide a sentence with the
number 2011 in it?

Cheers!
Kostas

Reported by inigo.aldana on 2011-01-22 00:19:55

hssm commented 9 years ago 
punctuation is generally the same as western punctuation (comma, period, question mark,
quotes)  There is an apostrophe-like symbol [(')<--this is the actual hebrew character]
which is used rarely, to modify hebrew letters to express western sounds (for transliterations
or names, etc)  for instance, chips=צִ'פס and there is a double-apostrophe-like character
which is used even more rarely.  I believe they are technically different (maybe different
unicode) from the normal single and double apostrophe, but they usually look the same
to me.

I haven't used flashcards that have needed sentence punctuation.  Do not know.

Numbers follow LTR:
הַשָּׁנָה הִיא 2011, והַשָּׁנָה הַבָּאָה תִּהְיֶה 2012. י
(The year is 2011,  and the next (coming) year will be 2012.)

numbers follow LTR.

Reported by mordecai.veldt on 2011-01-22 01:35:41

hssm commented 9 years ago 
This is very good example and highlights why the apostrophe is needed at the end.

Without the apostrophe, my algorithm would think that the Hebrew texts ends at "תִּהְיֶה"
and that " 2012." is part of the LTR script and should follow to the right of it!

Anyway, it won't be perfect, I don't think I'll bother with complex bidirectional text
as in:
He first said "הַשָּׁנָה הִיא 2011" and then he added "והַשָּׁנָה הַבָּאָה תִּהְיֶה
2012. י"

What an interesting language! Even copy-pasting the above and placing the quotes was
a challenge! And when I press open parenthesis, a closing one appears ;)

I expect that 99% of the Hebrew cards have the field either fully in Hebrew, or fully
in non-Hebrew. If that's not the case, we make an enhancement later.

Reported by inigo.aldana on 2011-01-22 02:08:59

hssm commented 9 years ago 
Hmmm, I spend some time yesterday on this and now I'm convinced it's not possible to
show the points correctly. No matter how I arrange the characters with or without Unicode
control characters, with or without directional tags, the points simply refuse to be
placed above or below the consonant.

The internet was informative that 2.2 android supports Hebrew in all components EXCEPT
the WebView, which is the one we need.

So, I think the next best thing is to hide the vowels completely and show the rest
in the correct order. Sorry, I can't find a way to make it work properly.

Kostas

Reported by inigo.aldana on 2011-01-22 18:37:07

hssm commented 9 years ago 
> the last "apostrophe" which is actually a yod character I
> included b/c otherwise the question mark jumps to the other end

You should use the RLM character, here's one between the dashes: -‏-. The RLM character
is a nonprinting Right-Left Mark. I'll give an example sentence without:
שלום, עולם!
And an example sentence with an RLM at the end:
שלום, עולם!‏
You can find this character in the Lyx Hebrew layout, it's Shift-ט. There is a corresponding
LRM character on shift-א as well.

I am now downloading an Android virtual machine to test this issue. I don't yet have
an Android device but I'll see what I can do with the virtual machine. Please CC me
any other RTL issues that crop up and I'll triage. Thanks!

Reported by dotancohen on 2011-01-22 18:37:25

hssm commented 9 years ago 
Thanks all for the investigation and thanks Kostas for the fix :-)
Switching priority to "Low", as now most cards are reasonably readable.

Reported by nicolas.raoul on 2011-01-24 09:22:21

hssm commented 9 years ago 
Actually, I just realised something regarding this issue.

If the Hebrew text doesn't contain vowels, the WebView displays it correctly.

Therefore, the fix that we should do is just remove any vowels from the text. It's
not perfect, but as close we get it to working.

Reported by inigo.aldana on 2011-01-24 19:49:41

hssm commented 9 years ago 
That's a good insight Inigo, and it squares with why other Hebrew things work on my
Android. (Most 'normal' Hebrew doesn't include the vowel points).

Reported by mordecai.veldt on 2011-01-24 19:54:40

hssm commented 9 years ago 
Well, it works "fine" in my device now (Nexus One, Android 2.2), but this behaviour
might be depended on the Android version, because if I'm not mistaken, Google added
(is adding actually) support for Hebrew gradually. Punctuation should be working fine,
numerals... I didn't try them, but I have commented code ready for them in case they
appear RTL.

I would advice you try it and report here if it's still reversed, or if I messed up
anything else.

Oh, there's a new preference (sorry translators) for this, under Experimental, it's
disabled by default, so don't forget to switch it on.

It hope it's included in the next beta.

Reported by inigo.aldana on 2011-01-24 20:04:19

hssm commented 9 years ago 
Thanks a lot for looking into this and doing what you can! I guess it is the accents
in both Greek and Hebrew that mess stuff up in 2.2... good to know.

maybe in 2.3 or 2.4 or 3.0 Google will finally support fonts better :(  

Thanks again.

Reported by jasonrhine on 2011-01-29 16:42:44

hssm commented 9 years ago 
This issue occurs also in Arabic. The desktop and WWW version work fine, but on Android,
the order is reversed. I tried the same text without vowels, but the text-inversion
continues to exist.
I also tried introducing the RTL character in the desktop version, but that doesn't
seem to proagate to Android.

Reported by andreas.stuber@expertflow.com on 2011-02-13 22:42:07

hssm commented 9 years ago 
I think we can close this issue. We did as much as we could.

Maybe we need to open a new one for arabic? There was another thread related, but I'm
not sure where.

Reported by inigo.aldana on 2011-03-27 17:48:44

hssm commented 9 years ago 
Fixed in AnkiDroid 0.6, available on Google's Android Market.

Reported by nicolas.raoul on 2011-04-22 02:32:00

hssm commented 9 years ago 
I just downloaded AnkiDroid from the Android Market (using HTC EVO Android 2.2) and
I am having the reversed hebrew problem.  I created a deck on the Anki desktop version
that works great! When I copy the exported deck to my sdcard and open it up in AnkiDroid,
all the hebrew is ltr and not rtl. Attached is my deck

Reported by daved314 on 2011-05-26 10:52:11

hssm commented 9 years ago 
if you hit 'menu' then go to preferences->other you can enable "force RTL for Hebrew"
which should fix the problem (though it eliminates pointings)

Reported by jasonrhine on 2011-05-26 16:25:10

hssm commented 9 years ago 
Jasonrh: I Use Anki in Fedora and do not have a "preference/other" option. is there
anyting I can do to have it working?

Reported by Alessio.AguirrePimentel on 2012-02-24 15:50:48

hssm commented 9 years ago 
@ Alessio.. 

This is the wiki for ankidroid not the desktop version of Anki :)  But Hebrew should
work fine in Fedora's desktop version as long as you have the correct Hebrew font installed
and chosen.

Reported by jasonrhine on 2012-02-24 17:09:59

hssm commented 9 years ago 
G'day,

@ Kostas

I see that you chose to remove the vowels because you couldn't get pointings above
and below characters, but only next to?

It would be great to get the pointsins displayed, even if not 100% correct. Any chance
of getting another option. Such that the options are the current 'force RTL with no
pointing' and the new 'force RTF with messy pointings'.

Thanks

Jason

Reported by jason.marriott on 2012-03-07 00:01:53

hssm commented 9 years ago 
Hi Jason,

I'll have a look at it. I don't remember how messy it looks, so no promises on the
results. So... I guess I'll have to open this ticket again :(

Kostas

PS: I know the tag says Arabic, it's the same RTL problem we have, so it fits.

Reported by inigo.aldana on 2012-03-09 09:17:57

hssm commented 9 years ago 
Thanks Very much Kostas

Reported by jason.marriott on 2012-03-13 00:15:18

hssm commented 9 years ago 
I'm having the exact same issue in Hebrew, does anyone know when/if this is planned
to be fixed in WebView. I need the dots under the letters.

Thanks,

Gali

Reported by Gali.Ungar on 2012-03-19 00:23:03

hssm commented 9 years ago 
It doesn't sound too promising Gali.  It's funny, many years ago when I was first studying
Biblical Hebrew I discovered Anki for my Palm PDA which was perfect (and is now free).
 Then after that palm was destroyed and I started teaching Hebrew, I purchased a palm
m100 off eBay for the sole purpose of running Anki.  I gave that away over a year ago
after I got my Android phone because I thought surely there was some droid flashcard
program that could replace Anki.  Boy have I been disappointed.  I'm about to start
looking on ebay again for a palm...

Reported by mr.kevin.austin on 2012-03-20 03:09:47

hssm commented 9 years ago 
Perhaps I spoke too soon.  Some are reporting the Hebrew RTL issue is fixed in Android
4.0.  Now you just have to wait for Android 4.0 to be rolled out for your device (which
may never happen depending on which device you use).  My phone is on the list for the
2nd wave of updates so I'll look forward to trying it out later this year.  Maybe if
anyone has a device already running Android 4.0 could try out Hebrew/Arabic in AnkiDroid
and report the results?

Reported by mr.kevin.austin on 2012-03-20 04:06:54

hssm commented 9 years ago 
I have 4.0.3 Running on my SGII and RTL works, but pointing is not under characters,
but next to.

Reported by jason.marriott on 2012-03-20 04:46:26

hssm commented 9 years ago 
So 4.0.3 does not work than. ICE CREAM SW is useless to me then.

google FIX these issues now --and older versions while you at it.

Thanks,

Gali

Reported by Gali.Ungar on 2012-03-29 03:55:21

hssm commented 9 years ago 
Yet another fix for this based on this thread has gone in:
https://groups.google.com/forum/?fromgroups#!topic/anki-android/n9JpDiQ_dgU

Available in 2.0beta11

Reported by inigo.aldana on 2012-07-29 21:43:15

hssm commented 9 years ago 
Issue 551 has been merged into this issue.

Reported by inigo.aldana on 2012-07-29 21:48:44

hssm commented 9 years ago

Reported by inigo.aldana on 2013-01-07 18:57:09