Wrong map translation to english in Japan

[alexm51]

When I select other languages - used original names when at selected is not available. English always translated, but because of difference between chinese and japanese languages difficult to detect - most of elements with wrong translation.

[nicolas-raoul]

Confirmed with OsmAnd 2.3 nightly:

In Japanese, 朝日 has only one transliteration which is asahi, but in OsmAnd it is transliterated to Zhao Ri which is how it sounds in Chinese. Similarly, 駅 should be eki instead of Xia, 六本木 should be roppongi instead of Liu Ben Mu.

How to fix this bug:

• The best way to transliterate Japanese is to use a library like Kuromoji or Mecab. Both are open source.
• A simple way to know whether a POI's label should be transliterated as Chinese or Japanese is to perform this calculation:

106.38 + longitude - 0.666*latitude = ?  Japanese if positive, Chinese if negative

[chatan]

:+1: Being an english reader living in Japan makes this issue a real pain in the butt.
If it is too hard to transliterate to Japanese vice Chinese, at least change the code [when english is selected language] so it doesn't transliterate at all when there is no English in the OSM map? At least then I can play "match the kanji" :grin:. Displaying things like the Liu Ben Mu example are worthless.
Cheers

[vshcherb]

If you change map language to japanese, it won't transliterate any more. Using local names, should not transliterate also.

[chatan]

vshcherb, Yes That is true, but that hides the English names also which isn't helpful to English readers. I edited my post to clarify [When English is selected language...] My current pain in the butt work around is I constantly switch languages between English(& Chinese) and Local(Japanese) depending on if there is English translation contained in OSM data, which quite a bit is missing. I'm not really qualified to add much English, although I have tried to do some.
Cheers

[vshcherb]

I know that feeling, we have plans to have double names (english and local). It just will take some time, cause we don't have big team. And library transliteration we are using https://github.com/gcardone/junidecode is not good enough may be...

[pl71]

Transliteration from Bulgarian to English is also not quite correct. Where I can correct this - in OSMAND code or in the above mentioned library - https://github.com/gcardone/junidecode ?

[pieleric]

@vshcherb The problem with Japanese transliteration is that currently it's transliterated as if it was Chinese (because they share the same characters). However, Chinese pronunciation and Japanese pronunciation have nothing to do with each other in 90% of the cases. For example, "Tokyo" would be written in transliterated Mandarin as "Dong Jing". It makes things worse than not doing anything!

The issue with Japanese, is that it's much harder to transliterate than Chinese, and probably almost any other language. So a generic library like junidecode might never handle this correctly. It could be worthy to investigate using a specialised library, such as the ones recommended by Nicolas: Mecab or Kuromoji.

[rrobek]

I'm also interested in this and have experimented a bit with Japanese transliteration in the last week. I found Kakasi-java (the "Kanji-Kana Simple Inverter" library) easiest to use. (also hosted on GitHub) For testing, I have written a map preprocessing tool based on the library. More information and a lot of nice maps on my website.

[nicolas-raoul]

Indeed, if the present issue is not solved by OsmAnd quickly, pre-processed maps are a temporary solution to make OsmAnd at least usable. @rrobek: I have used Kakasi (and actually I am the maintainer for the Java version of Kakasi) and I can tell you that Kakasi is poor at transliteration, unfortunately. I can help you set up Kuromoji or Mecab if you want :-) By the way, does OsmAnd use name:ja_rm? The display priority should be: name:en > name:ja_rm > Transliteration

[rrobek]

I have updated my transliteration tool to use Kuromoji. The results have improved a lot. New maps are available at my website.

[vshcherb]

@rrobek How do you differentiate japanese & chinese glyphs? Aren't they same unicode characters? BTW: http://rrobek.de/japanmaps.html the link is down

[rrobek]

@vshcherb I don't. It's a data preprocessing app, and I'm only using it for map data from Japan. So I just assume it's Japanese, not Chinese.
But you are right. I think that for a general solution, the only possibility is to differentiate based on the geographic location. (Indeed: Many kanji glyphs are used both in Japanese and Chinese, and may even have the same meaning, but almost all of them are pronounced, and thus transliterated, completely differently. It's a mess...)

@rrobek https://github.com/rrobek How do you differentiate japanese & chinese glyphs? Aren't they same unicode characters?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/osmandapp/Osmand/issues/1396#issuecomment-210876723

[eindgebruiker]

Would it be possible, as a temporary solution, to just not translate if there is no ja_rm version available? Currently the names Osmand displays are not just wrong, they are false.

[vshcherb]

Transliterate (which is not translate) could be disabled in Configure Map - Map Language.

[tomato42]

transliteration being on or off for Japanese has no impact, they are always transliterated as if they were Chinese! This is with version 3.2

[mikesievers]

This is still an issue in 3.2.2 and makes me switch to maps.me when I go to Japan, as the Chinese transliteration is useless for Japan. I would also suggest to use the ja_rm tag when present (if the wished for language is not present). ja_rm (for "Romaji") already contains the transliteration into Romanic characters (the alphabet English or German speaking people are used to). I believe mimicking the behavior of e. g. Maps.me would be the best solution. If this were solved, it would make osmand my go-to app for maps in Japan as well. The behavior I am suggesting is:

When transliteration is enabled: display the ja_rm value if present or the local name if it is not.

[tomato42]

if present ja_rm could be shown in preference for default name, but according to BCP 47 and (OSM wiki), it should be ja-Latn (so both should be used and if both are present the latter should be used)

[mikesievers]

if present ja_rm could be shown in preference for default name, but according to BCP 47 and (OSM wiki), it should be ja-Latn (so both should be used and if both are present the latter should be used)

Thanks for adding that, I followed your links and also found https://wiki.openstreetmap.org/wiki/Multilingual_names going into more detail (along with a note that name:ja_rm and name:ja-Latn need clarification, as name:ja-Latn is the standard needed for i18n).

Adding the last comment, I believe that should yield the following the logic (assuming english is the currently selected language and the map displays a region in Japan):

• if name:en is present, display that and break (stop looking for alternatives)
• if transliteration is NOT enabled, display the local name (name or possibly name:ja) and break
• if transliteration is enabled and name:ja-Latn is present, display that and break
• if transliteration is enabled and name:ja_rm is present, display that and break
• if transliteration is enabled and neither name:ja-Latn or name:ja_rm is present, display local name

I have not seen any ja-Latn tags in Japan yet but a lot of ja_rm, but it sounds like ja-Latn is the way to go in the future with i18n tools in mind.

As an example from a current node for Starbucks, the following tags are possible (name:ja-Latn added by me here to incorporate the comment from tomato42):

name=スターバックス name:ja=スターバックス name:en=Starbucks name:ja-Latn=Sutābakksu name:ja_rm=Sutābakksu

For a foreigner, I believe that name:en would be the preferred version. Next best would be either name:ja-Latn (which I have inserted above without having seen it in the wild before, according to https://wiki.openstreetmap.org/wiki/Multilingual_names ) or name:ja_rm (seen in the wild as above).

In any case, any of the versions above is more readable than the current transliteration, because for that one would need to know Chinese in order to guess the character being displayed and also Japanese in order to know how to pronounce it. (Japan uses, among others, the Chinese characters. The meaning should be the same, but the pronunciation/reading is very different.) This would be completely counter to the intention of transliteration, effectively rendering it useless for Japanese. While at the same time not making use of the already used name:ja_rm or possibly name:ja-Latn tags.

Finally, how maps.me does it seems like a good version - write the english name and the local/japanese name below. This helps especially when name:ja-Latn or name:ja_rm is not intuitively readable for non-japanese speakers (as in the example).

And thanks for working on Osmand+, I do enjoy it very much!!

[mikesievers]

I am looking at the representation of an area within Shinjuku, Tokyo right now on Osmand+ 3.3.6 with map language set to "en" and transliteration disabled. The result is as expected, names are in English or in Katakana/Kanji where en is probably not available.

From my point of view this is now solved, thank you!

[chatan]

YES! Looks good. This makes OsmAnd+ more more functional and useful.
Thanks OsmAnd team!

[tomato42]

yes, I can confirm that it doesn't transliterate to Chinese/Mandarin in Japan, but it doesn't transliterate to romaji even if the transliteration option is enabled (IOW. I see kanji and katatana with transliteration option enabled)

this is with 3.3.6

[vshcherb]

Non-sense transliteration is disabled, so it should help to see proper english transliteration though library to support prope japanese takes too much memory, we are looking how to provide it as a map download.

[bernysk]

look at https://rrobek.de/jmtranslate.html

[vshcherb]

Thanks @bernysk

[vshcherb]

Merged to #6292