Add translations of language names to locale install list

amandastevens commented 3 years ago

New problem description (3.4) The language lists (both in the software installer and e.g. on the language toggle plugin) list language names in the current language:

In cases where the same language is available in 2 character sets (e.g. Serbian), it's not clear which is which.

For users who find themselves using the software in a language they don't speak, they may not know their language's name in the current UI language.

Suggestion (Alec): Include the language name in both the current UI and listed language's own locale.

Old problem description (3.3.x)

When I install a new locale on an OJS 3.3 site as an Administrator in the Site Settings, the list of locales presented shows the locale names in their native script. If I can't read that language, it's difficult to identify which locale I need to install. I was only able to identify which locale was Simplified Chinese by consulting the registry/locales.xml file.

I suggest that the locale list show the language name in its native script and a translation of it into English, German, Spanish, Indonesian, etc., depending on the interface language being used.

This would be especially helpful for multi-OJS site administrators.

This is the current locale list: Language list

NateWr commented 3 years ago

I'm not sure if we have every language translated into every other language. @asmecher do you know if this is part of the ISO library we're using?

(Also, I didn't know we have a Scottish Gaelic translation of OJS!! :joy:)

asmecher commented 3 years ago

The php-isocodes library we use for countries and currencies does include 639-3 language list. Note that it's harder to customize entries once they're delegated to that library, though; see https://github.com/asmecher/customCountryList for a plugin that allows adjustments to the country list.

amandastevens commented 3 years ago

Thanks, I didn't know about that plugin.

asmecher commented 2 years ago

In the main branch (pre-3.4), the language lists have now been converted to the current language (probably since https://github.com/pkp/pkp-lib/issues/6328). However, this isn't a great solution either. I've updated the issue to show both formulations of the problem and scheduled this issue for 3.4; for 3.3.x, you can customize registry/locales.xml as a work-around.

Some examples we can look to on major websites:

debian.org:

un.org:

wordpress.org (probably relying on https://github.com/pkp/pkp-lib/issues/7512):

Note that many major sites do not appear to have language selectors on them -- probably due to their reliance on Accept-Language headers (7512), making it appear transparent to the user. However, while that would be nice for an automatic pre-selection, IMO there's value in making our multilingual support explicit.

NateWr commented 2 years ago

So is the idea to have the list show in the current locale, then it's own locale?

Change Language ✔ English Arabic (عربى) Ukrainian (українська)

asmecher commented 2 years ago

Yes, IMO that would be the best result. The encoding would be implicit (e.g. Latin vs. Cyrillic) but I don't see any benefit in exposing that more explicitly.

touhidurabir commented 2 years ago

So is the idea to have the list show in the current locale, then it's own locale?

Change Language ✔ English Arabic (عربى) Ukrainian (українська)

@asmecher In that case do we plan to remove the country name ? Right now we shows as

Arabic (Iraq)
Spanish (Spain)
Spanish (Mexico)
French (Canada)
French (France)

we have same language with different country . if we plan to remove the country, then it make no sense to have multiple language like French for Canada and France . in that case, we will have something like this

Arabic (عربي)
Spanish (Español)
French (Français)

but I think that will be a breaking change . Should we do something like

Arabic (Iraq)(العراق - عربي)
Spanish (Spain)(España - Español)
French (France)(France - Français)

so basically the format will be like LANGUAGE_NAME_IN_ENGLISH(COUNTRY_NAME_IN_ENGLISH)(COUNTRY_NAME_IN_LANGUAGE_LOCALE - LANGUAGE_NAME_IN_LANGUAGE_LOCALE)

what do you think ?

NateWr commented 2 years ago

We should remove the country name. This causes a fair amount of confusion and sometimes controversy.

touhidurabir commented 2 years ago

We should remove the country name. This causes a fair amount of confusion and sometimes controversy.

@NateWr in that case how are we going to show the locale lists as French (Canada) as locale value fr_CA and French (France) has locale value fr_FR ? So if a journal/context has already installed fr_CA but in our list we can only show

French (Français)

without country . so what value it should be ? we will have no way to show option for both value fr_CA and fr_FR .

NateWr commented 2 years ago

I did an informal survey on Mattermost with some large hosts and I found that this case is not really used. The only cases they found where two variants of the same language with the same character set were installed were a few pt_BR and pt_PT. The publishers thought this was likely an attempt at inclusion because before our language always included the country.

In other words, they just wanted Portuguse but because our UI said "Portugese Brazil" and "Portugese Portugal" they felt they needed to enable both so that authors weren't led to believe they only supported Brazilian or Portugese authors.

That's what I mean about confusion. Countries are a bad proxy for language and rarely map onto real-world usage.

asmecher commented 2 years ago

@touhidurabir, the country is already excluded in the main branch; see https://github.com/pkp/pkp-lib/issues/7871 for details about that change.

asmecher commented 2 years ago

Touhidur and I talked about this today and especially the case where we do have to present several languages with different country codes alongside each other (e.g. pt_PT and pt_BR). This won't be common for end users but there is the case above that Nate mentioned (https://github.com/pkp/pkp-lib/issues/7352#issuecomment-1238251497) and it will be a problem e.g. on the configuration forms and installation form.

We decided that a good approach that covers all cases might be to examine the list before presentation to the user, determine whether there are multiple candidates for a language (e.g. with 2 countries), and if there are, then include the country name in the resulting list.

For most users, this will mean only showing the language name -- but for those with multiple candidates, it will also include the country.

NateWr commented 2 years ago

The case that I mentioned in my comment was actually a case against using country names. Journals were enabling the same language in two countries because the country names made the language seem exclusive to one country or another.

Do we know of a real-world use case where a journal wants to have two country options for the same language with the same character set?

asmecher commented 2 years ago

On the installation form (and the settings forms) we need something that'll distinguish between fr_CA and fr_FR, and can't just list "French" twice. So those lists need to be specific.

For end-user-facing content, they should be as general as possible -- listing only the language Portuguese / Português unless the journal has chosen to enable both Portuguese / Português (Brazil) and Portuguese / Português (Portugal). In the latter case the country should be presented too, so it's not just Portuguese / Português twice. I agree that this is a boundary case but I think it would be nicer to go with an option that wouldn't leave existing users up a creek.

There probably aren't many cases where two countries should be listed for the same language, but I'm willing to bet one exists that'll be important to someone. Thinking of cases where...

the language is formally standardized in two places e.g. Swiss Standard German (de_CH): https://en.wikipedia.org/wiki/Swiss_Standard_German, and/or
places where it's politically important to distinguish between locales (I'm sure you can think of a few of these).

@NateWr, for a more general solution to the issue that led a journal to enabling two Portugueses, I'd like to stop forcing language codes to specify countries in the first place, in line with standard locale code usage. See https://github.com/pkp/pkp-lib/issues/8210.

NateWr commented 2 years ago

Ok, makes sense. It won't "fix" things for existing users who have already enabled two languages to compensate. But it will improve things going forward. Maybe we can mention it in our release notebook, too.

touhidurabir commented 2 years ago

@asmecher please review the PRs

pkp-lib --> https://github.com/pkp/pkp-lib/pull/8289 ojs --> https://github.com/pkp/ojs/pull/3552

asmecher commented 2 years ago

@touhidurabir, this looks very good in the installation/settings forms when viewing the English case. When I switch over to French and view the list, it appears that many language names still appear in English (along with their own language name) -- do you see the same?

Lastly, the language selector in the front end needs to be adjusted too; at the moment, if you install both "Portuguese" variants, they'll be distinct and clear in the setup forms but then both translate to "Portuguese" in the front end. Example (in French)

touhidurabir commented 2 years ago

When I switch over to French and view the list, it appears that many language names still appear in English (along with their own language name)

@asmecher yes, this is happening for me also . I have checked in the main branch(which does not include this changes related to this issue PR yet), this is happening there also . For example , when I set French(France) as primary locale, can see the language Armenian when it should be Arménien for French . I guess may be some language name not getting translated properly.

the language selector in the front end https://github.com/pkp/pkp-lib/issues/7352#issuecomment-1248423344

it's done .

Updated PRs

pkp-lib --> https://github.com/pkp/pkp-lib/pull/8289 ojs --> https://github.com/pkp/ojs/pull/3552

touhidurabir commented 2 years ago

@asmecher updated PRs. Tests passed for all apps .

pkp-lib --> https://github.com/pkp/pkp-lib/pull/8289 ojs --> https://github.com/pkp/ojs/pull/3552 omp --> https://github.com/pkp/omp/pull/1214 ops --> https://github.com/pkp/ops/pull/364

asmecher commented 2 years ago

@touhidurabir, I still see a problem with the language toggle block plugin when both forms of Portuguese are enabled; they are not distinguished:

Also, I would recommend duplicating the common.withForwardSlash locale key for all locale/*/common.po files in pkp-lib. Normally I wouldn't suggest doing this but

the translation that's used for English will work in 90% of cases, and
when no translation is available, the language won't appear at all in the drop-down -- the user will only see ##common.withParenthesis##, which will make it hard to navigate.

touhidurabir commented 2 years ago

@asmecher I have update all the locale presentation (if haven't missed any) with the new approach that includes both site and context wise display . Also updated the locale/*/common.po file . Please review . if all ok , then I will port the changes to other apps .

asmecher commented 2 years ago

@touhidurabir, the front-end display now shows too much information:

See https://github.com/pkp/pkp-lib/issues/7352#issuecomment-1248423344 for a description of how the public-facing display should look.

For the above case, for example, it should be:

Language

English Français Português (Brazil) Português (Portugal)

Each language should be given in its own translation (thus "Français" rather than "French", even though the current UI language is English). The Brazil/Portugal distinction for Portuguese is shown because two Portuguese options are available; the region is not relevant for other languages (because there are not multiple forms of the same language enabled) so it should not be shown for them.

touhidurabir commented 2 years ago

@asmecher Please review the PRs . As per our last discussion, all requested changes have been implemented .

pkp-lib --> https://github.com/pkp/pkp-lib/pull/8289 ojs --> https://github.com/pkp/ojs/pull/3552

marcbria commented 2 years ago

I discovered today this thread... jumping form a forum where somebody complains about the "Español (España)" label (that by the way it triggers some political tensions that have my full sympathy).

If we are just talking about the installation langs, make sense to show a full "Lang (dialect)" list, but if we are talking about the lang selectors (and even the galley labels) IMO, @NateWr made a valid point with "Do we know of a real-world use case where a journal wants to have two country options for the same language with the same character set?"

If make sense and it's not too late I would ask to reconsider the idea of making visible the dialect variant and reduce the label to the lang (ie: instead of "Español (España)", keep it to "Español") EXCEPT if somebody enables 2 langs of the same type.

I think this solution will make everybody happy and in terms of code is as easy as find a match in the first two chars of the enabled langs.

@asmecher too late for this?

jonasraoni commented 2 years ago

Hiding the country when there's no need to disambiguation sounds good 👌

About enabling variants: I have to say the pt_PT sounds weird/wrong for me, a pt_BR speaker. I'd rather have an interface in en_US than pt_PT, so I think it makes sense to enable both :)

asmecher commented 2 years ago

@marcbria and @jonasraoni, this is what's already proposed/accepted in https://github.com/pkp/pkp-lib/issues/7352#issuecomment-1248423344.

jonasraoni commented 2 years ago

I've seen it, I'm just giving my support :)

marcbria commented 2 years ago

Sorry... I don't know if I read 7352 too fast or it's my poor reading comprehension in English, but I read you again and your comment was perfectly clear.

Thank you Alec. I will report it back in the forum.

asmecher commented 2 years ago

Thanks, @touhidurabir! I've manually merged the two PRs into main (for OJS and pkp-lib) due to merge conflicts. However, I jumped the gun -- there are still a couple of things needing attention...

There's a quirk with the language toggle block; it appears to always display the selected language's name in English. Do you see the same issue? Example: In this screenshot the currently selected language is Portuguese (Brasil) -- note that it says Portuguese instead of Português. Switching over to Português (Portugal) causes the error to appear there (as with English and French).
Could you port the OJS repo changes to OMP and OPS? [edit: I'll take care of that - Alec]

asmecher commented 2 years ago

On second thought, I'll take care of the port to OPS and OMP, but the English-language presentation is yours to look at!

touhidurabir commented 2 years ago

English-language presentation is yours to look at!

@asmecher looking into it .

marcbria commented 2 years ago

That's great!

@asmecher, to properly report back in forum: The change will be for 3.4 or greater, isn't it? If yes, any plans to port back to 3.3?

Thanks!

asmecher commented 2 years ago

This will be in 3.4, but won't be back-ported to 3.3.

NateWr commented 2 years ago

It looks like work on this has been merged to main. However, I just looked at an up-to-date checkout of main and the country names are showing up in the form language switcher even though I only have one country of a language enabled.

language-country

In the screenshot above, my active locale is en_US. I would expect to see:

language

Sorry if this is known and still being worked on.

touhidurabir commented 2 years ago

@NateWr good catch, it seems I missed that part to update . Also there is still one small change that I have missed and working on it . But do you think we should show it in primary locale or based on each language locale ? for example , for the user top right menu , we are showing each language name based on that language locale , check the screenshot of my local installation

I think the country name should be removed unless there is 2 same language (as in that case, we need some way to differentiate it) , So my guess it it should be like

Français | English

or

French/Français | English/English

Please correct me if I misunderstood it .

NateWr commented 2 years ago

In the forms we should always show the language name in the current locale. So in English it would be:

French | English

This is different from the other places, because here the user is entering information rather than switching languages, and we assume only that they know the language they are viewing the UI in.

NateWr commented 2 years ago

Another place to check is in the form field labels. In a multilingual form, the fields in languages other than the current language have a special label:

msgid "form.multilingualLabel"
msgstr "{$label} in {$localeName}"

So, when viewing the form in English, the label for the French input field for "Title" will be:

Title in French

Here, too, the language name should be shown in the current language.

touhidurabir commented 2 years ago

@asmecher please review the following PRs which add patch for missing changes .

pkp-lib --> https://github.com/pkp/pkp-lib/pull/8395 ojs --> https://github.com/pkp/ojs/pull/3599

asmecher commented 2 years ago

Merged, thanks!

asmecher commented 2 years ago

@touhidurabir, is this ready to be closed from your perspective?

touhidurabir commented 2 years ago

is this ready to be closed from your perspective?

@asmecher I believe so . we can re-open again if more changes require again in some other place .

asmecher commented 2 years ago

Great, thanks!

asmecher commented 2 years ago

(@NateWr, please let us know if you need more on this issue to resolve the presentation as it shows in the new submission wizard.)

touhidurabir commented 1 year ago

Based on feedback from @NateWr , some more fixing/patching needed for this one as there persist problem in the following cases

[x] In the publication forms (Submissions > Submission > Publication > Title & Abstract / Metadata / Permissions & Disclosure / Issue).
[x] In the contributor forms (same location, but Contributors)
[x] In the submission wizard (Submissions > New Submission)
[x] In the file uploads form (Submissions > Submission > Upload File > Review Details)
[x] In the issue forms (Issues > click issue > Issue Data
[x] In the distribution settings (Settings > Distribution > License / Search Indexing )
[x] In the submission settings (Settings > Workflow > Submission > Author Guidance)
[x] In the file component settings (Settings > Workflow > Submission > Components > Add a Component)
[x] In the review settings (Settings > Workflow > Review > Reviewer Guidance and Review Forms > Create Review Form)
[x] In the publisher library (Settings > Workflow > Publisher Library > Add File)
[x] When adding/editing an email template (Settings > Workflow > Emails > Add and edit templates > Click Edit for Discussion (Copyediting) > Add template)
[x] Add navigation menu item (Settings > Website > Setup > Navigation > Add Item)
[x] Add announcement and announcement types (First enable announcements at Settings > Website > Announcements to see the main navigation item on the left. Then go to Announcements > Add Announcement / Add Announcement Type.)
[x] Create and edit section (Settings > Journal > Sections > Create Section / Categories > Create Categories)

need to update the generalised solution to address one overall implementation for all above cases and if possible, without the changing the view based implementation .

asmecher commented 1 year ago

One more: in the administration area, when you use the Install Locale button to select a new language to install, the raw locale codes should be shown (along with the names) to distinguish between ambiguities.

Here's an example of an ambiguous entry -- they are for uz_UZ@cyrillic and uz_UZ@latin.

touhidurabir commented 1 year ago

@asmecher Patch PR . If the tests get passed and all ok, I will forward port it to OMP/OPS. I personally think it now has better implementation than before .

pkp-lib --> https://github.com/pkp/pkp-lib/pull/8589 ojs --> https://github.com/pkp/ojs/pull/3736

asmecher commented 1 year ago

We still have this problem on the installation form; can you add the locale codes there as well?

touhidurabir commented 1 year ago

the raw locale codes should be shown (along with the names) to distinguish between ambiguities. We still have https://github.com/pkp/pkp-lib/issues/7352#issuecomment-1407216035 on the installation form; can you add the locale codes there as well?

@asmecher already added .

touhidurabir commented 1 year ago

@asmecher updated PRs

pkp-lib --> https://github.com/pkp/pkp-lib/pull/8589 ojs --> https://github.com/pkp/ojs/pull/3736

unless we want to make any more changes, it should be good to ported to OMP/OPS .

touhidurabir commented 1 year ago

@asmecher PRs

pkp-lib --> https://github.com/pkp/pkp-lib/pull/8589 ojs --> https://github.com/pkp/ojs/pull/3736 ops --> https://github.com/pkp/ops/pull/462 omp --> https://github.com/pkp/omp/pull/1327

pkp / pkp-lib

Add translations of language names to locale install list #7352

Language