Multilang messages not used

[benknoble]

Describe the bug

In terminal vim with +multi_lang and locale as follows, the messages are all in French.

In macvim, however, even when running in the terminal, messages are not localized. They remain in English.

Note that I'm not talking about menus, the translations for which I understand were recently borked. I'm talking about messages, and the interface such as the "quit anyway?" prompt.

# locale
LANG="fr_FR.UTF-8"
LC_COLLATE="fr_FR.UTF-8"
LC_CTYPE="fr_FR.UTF-8"
LC_MESSAGES="fr_FR.UTF-8"
LC_MONETARY="fr_FR.UTF-8"
LC_NUMERIC="fr_FR.UTF-8"
LC_TIME="fr_FR.UTF-8"
LC_ALL=

To Reproduce

Detailed steps to reproduce the behavior:

1. Run vim --clean (or gvim --clean, etc.)
2. Type :lang (with a non-English locale) -> Current language: "fr_FR.UTF-8/fr_FR.UTF-8/fr_FR.UTF-8/C/fr_FR.UTF-8/fr_FR.UTF-8"
3. Notice not even the :intro screen is translated

Expected behavior

Terminal vim translates all messages and such.

Environment (please complete the following information):

• Vim version

  VIM - Vi IMproved 8.1 (2018 May 18, compiled Oct 30 2019 11:57:56)
  macOS version
  Included patches: 1-2234
  Compiled by travis@Traviss-Mac.local
  Huge version with MacVim GUI.  Features included (+) or not (-):
  +multi_lang

• OS: macOS 10.14.5
• Terminal: Terminal and GUI

Additional context

This really bugs me because it defeats the muscle memory I have built up to respond to prompts and read messages—I'm used to the French now. I don't want to go back to English messages. I rarely use the :gui part of MacVim, but it's nice for external applications (i.e. away from the terminal). I could always remove it from my shell's PATH but continue to use it as a graphical app, but that too is unsatisfactory.

[eirnym]

@benknoble This is a known TODO, that MacVim has only English version and was discussed many times. One of the main reasons is translation of the main menu in GUI and :help topics. You can contribute a translation (or more than one) if you need it.

[benknoble]

@eirnym Perhaps I wasn't clear—I'm not referring to help topics or to the menus.

I'm talking about messages. The little things that popup and say, do you want to quit? Hit enter to continue... Swap file detected, press a button:

Or this copy-paste of my (normal vim) translated intro

  VIM - Vi Amélioré
                         version 8.1.2250
                     par Bram Moolenaar et al.
                     Vim est un logiciel libre

               Sponsorisez le développement de Vim !
tapez  :help sponsor<Entrée>    pour plus d'informations

tapez  :q<Entrée>               pour sortir du programme
tapez  :help<Entrée>  ou  <F1>  pour accéder à l'aide en ligne
tapez  :help version8<Entrée>   pour lire les notes de mise à jour

vs. macvim (even in terminal):

               VIM - Vi IMproved

               version 8.1.2234
           by Bram Moolenaar et al.
  Vim is open source and freely distributable

         Become a registered Vim user!
type  :help register<Enter>   for information

type  :q<Enter>               to exit
type  :help<Enter>  or  <F1>  for on-line help
type  :help macvim<Enter>     for MacVim help

As far as I can tell, terminal vim ships with the messages translations. I don't care about the menus and the help topics (the french ones are so out of date as to be practically useless, and I don't have the time to work on them). These messages are, however, muscle memory (the prompts use keys that depend on the first letter of the action to take, which obviously differ w.r.t. language).

[benknoble]

I believe all you need is this from vim/vim: https://github.com/vim/vim/tree/26967617a30e55aedc98b8f14b841d88469abd0e/src/po

[eirnym]

I agree with you. Because of the reasons I've mentioned, all other translations are disabled in this repo.

[eirnym]

776 is basically the same issue on that topic

[benknoble]

all other translations

This is not the same as "no menu translations," which I see in :h macvim-todo:

- Localized menus are not supported.  Choosing anything but "English" in the
  "International" pane of "System Prefences" may break the menus (and
  toolbar).

I would have expected translating internal messages to be completely achievable (since it is a simple string mapping at runtime), irrespective of the gui menu issues (which has to do with Apple's system). I understand (I think) why we have :set langmenu=none—but that doesn't explain why, if :lang mess gives the right locale, those messages aren't actually used.

[eirnym]

Menu translations should be done on a different level than translation for the rest of the ViM Core. Vim Core has it's translations in editable files, while translations of a MacOS app is done in other ways. So yes, this confirms, that the menu translations are missing.

Currently, most of menu done in menu.vim files, so it will be easier, but the main menu (MacVim) and the preference pane has been done as any other MacOS app. You need to add whole set off translations there.

So I invite you to contribute translations for many languages for parts which are missing in Vim Core or have to be done in the other way than has been in Vim Core.

[eirnym]

Let me find for you some of the places required for translations

[eirnym]

In the project there's Xcode string catalogs named src/MacVim/English.lproj for English, and it's needed to translate at the beginning.

After this, we need to translate menu.vim generated for English users. We shouldn't forget to make sure that Sparkle framework has translations wee need.

Also you need to know about differences with input, and edit commands accordingly. I use keyboard Polish for programmers, so I have both Polish and all keys placed like in English keyboard.

After these two parts has been done, help documentation should be also translated.

[benknoble]

I'm going to repeat what I started with: I don't have any interest in menu or help translations for MacVim at this time.

What I care about is the messages translation. They are all already translated in vim "core." I do not see why they cannot be used as is for MacVim's text-based message.

[eirnym]

I agree with you. The language selection in MacOS applications is done in a different way than in terminal and for other systems. To make it consistent other translations has been disabled.

If you insist, I can give you the exact answer what is the problem with enabling localisation.

[ychin]

Let me take a look.

[yanncoupin]

I hoped that this would also fix issue #11 that 5 years later is still an ongoing problem and is eerily similar, but alas, no. Could the acquired knowledge fixing this also help to fix the other issue?

[benknoble]

@yanncoupin Yeah, I think these are a bit different. Here, I'm talking very specifically about translation of messages/error codes/etc., which was already done in vim core--there has been some confusion on this issue, so let me re-emphasize it is not about the menus, which I understand won't be translated anytime soon.

Your issue is interesting though--seems like vim relies on some functions that depend on locale under the hood, and maybe doesn't appropriately force the locale for its parsing?

At any rate, when I launch vim (terminal, macvim from GUI, or macvim from terminal), I have locale set to French/UTF-8; that is, the environment variables are set in my terminal in addition to the language in Preferences.

The issue I have here is it seems like it would be easy to use the same translation mechanism as vim core for the messaging text (not the menu text) for display to a user in an error or on :intro or something.

[eirnym]

After the latest locale changes, MacVim correctly sets locale. The locale change was disabled back in a day because of absence of macOS app menu, Vim menu and Vim help translations.

Doing Vim menu translation is very simple as menu is quite similar to GVim and we can borrow it from there. My opinion is macOS app menu and changes in help could be in English for a while and translation help is requested.

@ychin What do you think on this matter?

[ychin]

My recent change (#1036) is somewhat related but not completely. @yanncoupin I couldn't quite repro the issue, but please use another issue to track as it's not related to this.

Anyway, @benknoble, I tried the Vim bundled with macOS Catalina, and built vanilla Vim, and I couldn't get the language translation to work. Do you know if this is supposed to work on normal Vim (aka not-MacVim)? Your problem description made it sound like this usually works, but I couldn't quite get a non-English :intro to show up even with vanilla Vim.

[benknoble]

@ychin yes, with the locale set as described, terminal vim gives messages in french. More details on that when I get in front of a computer.

[benknoble]

@ychin see the screenshot:

# locale
LANG="fr_FR.UTF-8"
LC_COLLATE="fr_FR.UTF-8"
LC_CTYPE="fr_FR.UTF-8"
LC_MESSAGES="fr_FR.UTF-8"
LC_MONETARY="fr_FR.UTF-8"
LC_NUMERIC="fr_FR.UTF-8"
LC_TIME="fr_FR.UTF-8"
LC_ALL=

[ychin]

Sorry but I still can't repro @benknoble . Can you show me exactly:

1. what Vim you are using (the version number 8.2.1200 doesn't seem like the one bundled with macOS. Did you build it yourself? Can you show me :version?)
2. what exact commands you used to trigger this?
3. What macOS version you are running?

For what it's worth, I tried using macOS Catalina's system vim and also building vanilla Vim myself (./configure && make). The steps I tried were (running this from an English system locale):

1. Run vim --clean
2. :lang fr_FR or :lang fr_FR.UTF-8
3. :intro.

Anything else?

[ychin]

Ok I figured it out. I think you are installing Vim from Homebrew and not using system Vim. And that works because it has gettext set up. I can look into fixing this. Please try to point out how you have installed/configured Vim in the future (plus :version printouts) since that tends to give a lot more info.

Edit: Actually, that doesn't seem like it. Yeah do you mind providing more info where you managed to get a localized version of Vim on macOS?

[eirnym]

Found explanation why translations are not used in MacVim: https://github.com/macvim-dev/macvim/issues/776

[benknoble]

Please try to point out how you have installed/configured Vim in the future (plus :version printouts) since that tends to give a lot more info.

@ychin yeah fair enough; thanks for taking the time on this. You'd think I would know that by now, having dealt with lots of issues on a popular vim plugin 😅

# this is homebrew vim
# vim --version
VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Jun  2 2020 16:45:43)
macOS version
Included patches: 1-850
Compiled by Homebrew
Huge version without GUI.  Features included (+) or not (-):
+acl               -farsi             +mouse_sgr         +tag_binary
+arabic            +file_in_path      -mouse_sysmouse    -tag_old_static
+autocmd           +find_in_path      +mouse_urxvt       -tag_any_white
+autochdir         +float             +mouse_xterm       -tcl
-autoservername    +folding           +multi_byte        +termguicolors
-balloon_eval      -footer            +multi_lang        +terminal
+balloon_eval_term +fork()            -mzscheme          +terminfo
-browse            +gettext           +netbeans_intg     +termresponse
++builtin_terms    -hangul_input      +num64             +textobjects
+byte_offset       +iconv             +packages          +textprop
+channel           +insert_expand     +path_extra        +timers
+cindent           +ipv6              +perl              +title
-clientserver      +job               +persistent_undo   -toolbar
+clipboard         +jumplist          +popupwin          +user_commands
+cmdline_compl     +keymap            +postscript        +vartabs
+cmdline_hist      +lambda            +printer           +vertsplit
+cmdline_info      +langmap           +profile           +virtualedit
+comments          +libcall           -python            +visual
+conceal           +linebreak         +python3           +visualextra
+cryptv            +lispindent        +quickfix          +viminfo
+cscope            +listcmds          +reltime           +vreplace
+cursorbind        +localmap          +rightleft         +wildignore
+cursorshape       +lua               +ruby              +wildmenu
+dialog_con        +menu              +scrollbind        +windows
+diff              +mksession         +signs             +writebackup
+digraphs          +modify_fname      +smartindent       -X11
-dnd               +mouse             -sound             -xfontset
-ebcdic            -mouseshape        +spell             -xim
+emacs_tags        +mouse_dec         +startuptime       -xpm
+eval              -mouse_gpm         +statusline        -xsmp
+ex_extra          -mouse_jsbterm     -sun_workshop      -xterm_clipboard
+extra_search      +mouse_netterm     +syntax            -xterm_save
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/usr/local/share/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H   -DMACOS_X -DMACOS_X_DARWIN  -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1       
Linking: clang   -L. -fstack-protector-strong -L/usr/local/lib -L/usr/local/opt/libyaml/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/readline/lib  -L/usr/local/lib -o vim        -lncurses -liconv -lintl -framework AppKit  -L/usr/local/opt/lua/lib -llua5.3 -mmacosx-version-min=10.15 -fstack-protector-strong -L/usr/local/lib  -L/usr/local/Cellar/perl/5.30.3/lib/perl5/5.30.3/darwin-thread-multi-2level/CORE -lperl -lm -lutil -lc  -L/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/config-3.8-darwin -lpython3.8 -framework CoreFoundation  -lruby.2.7     
# /usr/bin/vim --version | grep -o .gettext
-gettext
# mvim --version | grep -o .gettext
-gettext

what exact commands you used to trigger this?

Just vim with locale set appropriately (LC_ALL=fr_FR.UTF-8 vim should do it). vim --clean also works.

Actually, that doesn't seem like it. Yeah do you mind providing more info where you managed to get a localized version of Vim on macOS?

No, I think you're correct—I think it's that homebrew vim has gettext support compiled in. The other two (/usr/bin/vim and mvim—even with -v) don't show translations.

[ychin]

This is now fixed. MacVim will now recognize locale and show localized messages.

Not all messages are localized though, as MacVim-specific messages aren't localized. If you do :intro, this is what you will see in French:

                         VIM - Vi Amélioré

                         version 8.2.1412
                     par Bram Moolenaar et al.
                     Vim est un logiciel libre

            Devenez un utilisateur de Vim enregistré !
tapez  :help register<Entrée>   pour plus d'informations

tapez  :q<Entrée>               pour sortir du programme
tapez  :help<Entrée>  ou  <F1>  pour accéder à l'aide en ligne
          type  :help macvim<Enter>     for MacVim help

Note that the last sentence is not translated. (Feel free to submit PRs to fix it)

And of course menus are not localized per discussion here. I may look into that in the future and see how bad of a mix we have with localized and non-localized menu names.

If you are installing MacVim through Homebrew though it won't work yet. Their formula (macvim.rb) doesn't specify gettext as a dependency, which needs to be added for this to work. Will need to make a PR at https://github.com/homebrew/homebrew-core.