Switching languages doesn't take effect on some menu elements if BABEL_DEFAULT_LOCALE is not "en"

Atmostone commented 3 months ago

Bug description

Problem appears when I set default language not to English. If I add next lines to config file:

`BABEL_DEFAULT_LOCALE = 'ru'

LANGUAGES = { 'ru': {'flag': 'ru', 'name': 'Русский'}, 'en': {'flag': 'us', 'name': 'English'}, }`

Then English version breaks (most of menu links are still on Russian).

For example RU version:

And broken EN version:

How to reproduce the bug

Add next lines to config file:

`BABEL_DEFAULT_LOCALE = 'ru'

LANGUAGES = { 'ru': {'flag': 'ru', 'name': 'Русский'}, 'en': {'flag': 'us', 'name': 'English'}, }`

Switch language to English
Menu is still on Russian

Screenshots/recordings

Superset version

master / latest-dev

Python version

3.11

Node version

18 or greater

Browser

Chrome

Additional context

No response

Checklist

[X] I have searched Superset docs and Slack and didn't find a solution to my problem.
[X] I have searched the GitHub issue tracker and didn't find a similar bug report.
[X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

dosubot[bot] commented 3 months ago

To fix the issue where switching the default language from Russian to English in Apache Superset doesn't fully take effect, leaving some menu elements still in Russian, you need to ensure that the LANGUAGES dictionary is properly configured and not empty. In the provided configuration, LANGUAGES is set to an empty dictionary, which disables internationalization (i18n).

Here's the relevant part of the configuration:

# Babel config for translations
# Setup default language
BABEL_DEFAULT_LOCALE = "en"
# Your application default translation path
BABEL_DEFAULT_FOLDER = "superset/translations"
# The allowed translation for your app
LANGUAGES = {
    "en": {"flag": "us", "name": "English"},
    "es": {"flag": "es", "name": "Spanish"},
    "it": {"flag": "it", "name": "Italian"},
    "fr": {"flag": "fr", "name": "French"},
    "zh": {"flag": "cn", "name": "Chinese"},
    "zh_TW": {"flag": "tw", "name": "Traditional Chinese"},
    "ja": {"flag": "jp", "name": "Japanese"},
    "de": {"flag": "de", "name": "German"},
    "pt": {"flag": "pt", "name": "Portuguese"},
    "pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
    "ru": {"flag": "ru", "name": "Russian"},
    "ko": {"flag": "kr", "name": "Korean"},
    "sk": {"flag": "sk", "name": "Slovak"},
    "sl": {"flag": "si", "name": "Slovenian"},
    "nl": {"flag": "nl", "name": "Dutch"},
    "uk": {"flag": "uk", "name": "Ukranian"},
}
# Turning off i18n by default as translation in most languages are
# incomplete and not well maintained.
LANGUAGES = {}

To enable the translations, you should remove the line that sets LANGUAGES to an empty dictionary. The corrected configuration should look like this:

# Babel config for translations
# Setup default language
BABEL_DEFAULT_LOCALE = "en"
# Your application default translation path
BABEL_DEFAULT_FOLDER = "superset/translations"
# The allowed translation for your app
LANGUAGES = {
    "en": {"flag": "us", "name": "English"},
    "es": {"flag": "es", "name": "Spanish"},
    "it": {"flag": "it", "name": "Italian"},
    "fr": {"flag": "fr", "name": "French"},
    "zh": {"flag": "cn", "name": "Chinese"},
    "zh_TW": {"flag": "tw", "name": "Traditional Chinese"},
    "ja": {"flag": "jp", "name": "Japanese"},
    "de": {"flag": "de", "name": "German"},
    "pt": {"flag": "pt", "name": "Portuguese"},
    "pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
    "ru": {"flag": "ru", "name": "Russian"},
    "ko": {"flag": "kr", "name": "Korean"},
    "sk": {"flag": "sk", "name": "Slovak"},
    "sl": {"flag": "si", "name": "Slovenian"},
    "nl": {"flag": "nl", "name": "Dutch"},
    "uk": {"flag": "uk", "name": "Ukranian"},
}

By ensuring that the LANGUAGES dictionary is populated with the supported languages, the application should properly switch to English and update all menu elements accordingly ^[1].

To continue talking to Dosu, mention @dosu.

Atmostone commented 3 months ago

@dosu I already deleted LANGUAGES = {}, no effect. Menu elements are still written on language set by BABEL_DEFAULT_LOCALE. But if BABEL_DEFAULT_LOCALE set to "en", then there is no any problems.

dosubot[bot] commented 3 months ago

To resolve the issue where switching languages doesn't update all menu elements correctly when BABEL_DEFAULT_LOCALE is not set to "en", ensure that the locale is correctly set in the session. The menu_data function in superset/views/base.py is responsible for generating the menu data, including the locale information.

Here's the relevant part of the menu_data function:

def menu_data(user: User) -> dict[str, Any]:
    languages = {
        lang: {**appbuilder.languages[lang], "url": appbuilder.get_url_for_locale(lang)}
        for lang in appbuilder.languages
    }

    if callable(brand_text := appbuilder.app.config["LOGO_RIGHT_TEXT"]):
        brand_text = brand_text()
    return {
        "menu": appbuilder.menu.get_data(),
        "brand": {
            "path": appbuilder.app.config["LOGO_TARGET_PATH"] or "/superset/welcome/",
            "icon": appbuilder.app_icon,
            "alt": appbuilder.app_name,
            "tooltip": appbuilder.app.config["LOGO_TOOLTIP"],
            "text": brand_text,
        },
        "environment_tag": get_environment_tag(),
        "navbar_right": {
            "show_watermark": ("superset-logo-horiz" not in appbuilder.app_icon),
            "bug_report_url": appbuilder.app.config["BUG_REPORT_URL"],
            "bug_report_icon": appbuilder.app.config["BUG_REPORT_ICON"],
            "bug_report_text": appbuilder.app.config["BUG_REPORT_TEXT"],
            "documentation_url": appbuilder.app.config["DOCUMENTATION_URL"],
            "documentation_icon": appbuilder.app.config["DOCUMENTATION_ICON"],
            "documentation_text": appbuilder.app.config["DOCUMENTATION_TEXT"],
            "version_string": appbuilder.app.config["VERSION_STRING"],
            "version_sha": appbuilder.app.config["VERSION_SHA"],
            "build_number": appbuilder.app.config["BUILD_NUMBER"],
            "languages": languages,
            "show_language_picker": len(languages) > 1,
            "user_is_anonymous": user.is_anonymous,
            "user_info_url": (
                None
                if is_feature_enabled("MENU_HIDE_USER_INFO")
                else appbuilder.get_url_for_userinfo
            ),
            "user_logout_url": appbuilder.get_url_for_logout,
            "user_login_url": appbuilder.get_url_for_login,
            "locale": session.get("locale", "en"),
        },
    }

Ensure that the locale is correctly set in the session. If the locale is not set, it defaults to "en". You may need to check how the locale is being set in the session and ensure it is updated correctly when the language is switched ^[1].

Additionally, verify the Babel configuration in superset/config.py to ensure that the BABEL_DEFAULT_LOCALE and LANGUAGES settings are correctly configured:

# Babel config for translations
# Setup default language
BABEL_DEFAULT_LOCALE = "en"
# Your application default translation path
BABEL_DEFAULT_FOLDER = "superset/translations"
# The allowed translation for your app
LANGUAGES = {
    "en": {"flag": "us", "name": "English"},
    "es": {"flag": "es", "name": "Spanish"},
    "it": {"flag": "it", "name": "Italian"},
    "fr": {"flag": "fr", "name": "French"},
    "zh": {"flag": "cn", "name": "Chinese"},
    "zh_TW": {"flag": "tw", "name": "Traditional Chinese"},
    "ja": {"flag": "jp", "name": "Japanese"},
    "de": {"flag": "de", "name": "German"},
    "pt": {"flag": "pt", "name": "Portuguese"},
    "pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
    "ru": {"flag": "ru", "name": "Russian"},
    "ko": {"flag": "kr", "name": "Korean"},
    "sk": {"flag": "sk", "name": "Slovak"},
    "sl": {"flag": "si", "name": "Slovenian"},
    "nl": {"flag": "nl", "name": "Dutch"},
    "uk": {"flag": "uk", "name": "Ukranian"},
}

Make sure that the LANGUAGES dictionary includes all the languages you want to support and that the BABEL_DEFAULT_LOCALE is set to the desired default language. If LANGUAGES is empty, translations will not be available ^[2].

To continue talking to Dosu, mention @dosu.

Atmostone commented 3 months ago

@dosu Changing locale in menu_data didn't do anything useful, config.py is already correct, problem is somewhere else

dosubot[bot] commented 3 months ago

To resolve the issue where switching the default language from Russian to English doesn't fully take effect, leaving some menu elements still in Russian, you need to ensure that the session locale is correctly set to English. The menu_data function in superset/views/base.py is responsible for setting the language in the menu data.

Here is the relevant part of the menu_data function:

def menu_data(user: User) -> dict[str, Any]:
    languages = {
        lang: {**appbuilder.languages[lang], "url": appbuilder.get_url_for_locale(lang)}
        for lang in appbuilder.languages
    }

    if callable(brand_text := appbuilder.app.config["LOGO_RIGHT_TEXT"]):
        brand_text = brand_text()
    return {
        "menu": appbuilder.menu.get_data(),
        "brand": {
            "path": appbuilder.app.config["LOGO_TARGET_PATH"] or "/superset/welcome/",
            "icon": appbuilder.app_icon,
            "alt": appbuilder.app_name,
            "tooltip": appbuilder.app.config["LOGO_TOOLTIP"],
            "text": brand_text,
        },
        "environment_tag": get_environment_tag(),
        "navbar_right": {
            # show the watermark if the default app icon has been overridden
            "show_watermark": ("superset-logo-horiz" not in appbuilder.app_icon),
            "bug_report_url": appbuilder.app.config["BUG_REPORT_URL"],
            "bug_report_icon": appbuilder.app.config["BUG_REPORT_ICON"],
            "bug_report_text": appbuilder.app.config["BUG_REPORT_TEXT"],
            "documentation_url": appbuilder.app.config["DOCUMENTATION_URL"],
            "documentation_icon": appbuilder.app.config["DOCUMENTATION_ICON"],
            "documentation_text": appbuilder.app.config["DOCUMENTATION_TEXT"],
            "version_string": appbuilder.app.config["VERSION_STRING"],
            "version_sha": appbuilder.app.config["VERSION_SHA"],
            "build_number": appbuilder.app.config["BUILD_NUMBER"],
            "languages": languages,
            "show_language_picker": len(languages) > 1,
            "user_is_anonymous": user.is_anonymous,
            "user_info_url": (
                None
                if is_feature_enabled("MENU_HIDE_USER_INFO")
                else appbuilder.get_url_for_userinfo
            ),
            "user_logout_url": appbuilder.get_url_for_logout,
            "user_login_url": appbuilder.get_url_for_login,
            "locale": session.get("locale", "en"),
        },
    }

Ensure that the locale in the session is set to "en" (English). If the locale is not correctly set, you can manually set it in your session management code.

For example, you can add the following code to set the session locale to English:

from flask import session

# Set the locale to English
session["locale"] = "en"

Additionally, you should verify that the default language configuration in superset/config.py is set to English. Ensure that the BABEL_DEFAULT_LOCALE is set to "en" and that the LANGUAGES dictionary includes English:

# Setup default language
BABEL_DEFAULT_LOCALE = "en"
# The allowed translation for your app
LANGUAGES = {
    "en": {"flag": "us", "name": "English"},
    "es": {"flag": "es", "name": "Spanish"},
    "it": {"flag": "it", "name": "Italian"},
    "fr": {"flag": "fr", "name": "French"},
    "zh": {"flag": "cn", "name": "Chinese"},
    "zh_TW": {"flag": "tw", "name": "Traditional Chinese"},
    "ja": {"flag": "jp", "name": "Japanese"},
    "de": {"flag": "de", "name": "German"},
    "pt": {"flag": "pt", "name": "Portuguese"},
    "pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
    "ru": {"flag": "ru", "name": "Russian"},
    "ko": {"flag": "kr", "name": "Korean"},
    "sk": {"flag": "sk", "name": "Slovak"},
    "sl": {"flag": "si", "name": "Slovenian"},
    "nl": {"flag": "nl", "name": "Dutch"},
    "uk": {"flag": "uk", "name": "Ukranian"},
}

If the issue persists, it might be due to cached translations. Apache Superset uses .po files for translations, and sometimes the changes might not reflect immediately due to caching. You can try clearing the cache or restarting the Superset server to ensure that the new language settings take effect.

For example, you can clear the cache by running:

superset cache --clear

Then restart the Superset server:

superset run

This should ensure that the language setting takes full effect and all menu elements are displayed in English ^[1]^[2]^[3].

To continue talking to Dosu, mention @dosu.

goldjee commented 3 months ago

We've stumbled upon that in our environment too. As a workaround, we have switched the default back to English, still providing users an option to select Russian.

Atmostone commented 3 months ago

Does anyone know how can it be solved? Plus, tag for this issue should be "i18n", not "i18n:russian", because this problem seems to appear in all languages

Atmostone commented 3 months ago

All strings that are not translated are located in superset/initialization/__init__.py file. So it seems that their locale is set when superset starts up and then changing language doesn't affect it. Not sure how can it be solved though

Atmostone commented 3 months ago

I found how to fix this bug, but I can't push my fix branch to superset repository (unable to access 'https://github.com/apache/superset.git/': The requested URL returned error: 403). Does anyone know how to push it?

goldjee commented 3 months ago

It's normal that you can't push your changes to this repository directly. You need to fork apache/superset, create a new branch in your fork, commit your changes to it and create a pull request.

You can find some useful information on contributing here: https://superset.apache.org/docs/contributing/

rusackas commented 3 months ago

Yep, and please ping me on the pull request, @Atmostone, I'll be happy to help review it.

Iloer commented 2 months ago

@Atmostone could you describe the essence of the solution. thank you!

Atmostone commented 2 months ago

Solution is to this bug is to change all gettext in superset/initialization/__init__.py to lazy_gettext. Then everything works as it should and languages switch without problems

apache / superset