reloadInterval looks for locales (en-US) and not the language (en)

zhyph commented 2 years ago

🐛 Bug Report

I use i18next-http-backend to deal with changes in my translations while not deploying the whole application again, just changing the locales. Whenever I set a reloadInterval it tries to fetch from the locale not language, so when my i18n loads it fetches correctly from 'pt' but when the reloadInterval hits it tries to fetch 'pt-BR' resulting in a lot of 404.

To Reproduce

A minimal reproducible example. A codesandbox example or similar or at least steps to reproduce the behavior:

It would be kinda hard to recreate this, so I'm going to explain how it's happening. I have two folders named 'pt' and 'es', my front-end has a package that is transformed into a folder when it builds, so it becomes locales/'pt|es'/common.js, so when i18next needs to search for something it should be 'pt' or 'es'. With i18next.reloadResources() function it works fine, but I found a weird bug that it stops when your connection fails (this website only works with VPN and Proxy, so connections fail all the time) with reloadInterval it doesn't seem to happen, but it does breaks with the name of the language it's searching.

import i18next from 'i18next';
// Polyfill da api INTL
import 'intl';
import 'intl/locale-data/jsonp/en-US.js';
import 'intl/locale-data/jsonp/es.js';
import 'intl/locale-data/jsonp/pt-BR.js';
import { initReactI18next } from 'react-i18next';
import HttpApi from 'i18next-http-backend';
import LanguageDetector from 'i18next-browser-languagedetector';
import moment from 'moment';
import moment_timezone from 'moment-timezone';
import Request from './Request';
import numeral from 'numeral';
import { SetHichartConfig } from './vendor';
import { LOCAL_STORAGE_LANGUAGE } from './Constants';

i18next
  .use(initReactI18next)
  .use(HttpApi) // Registering the back-end plugin
  .use(LanguageDetector)
  .init({
    load: 'languageOnly',
    interpolation: { escapeValue: false },
    ns: [
      ...arrayOfStrings
    ],
    defaultNS: 'common',
    detection: {
      lookupLocalStorage: LOCAL_STORAGE_LANGUAGE
    },
    nonExplicitSupportedLngs: false,
    supportedLngs: ['pt', 'pt-BR', 'es', 'es-ES'],
    fallbackLng: {
      'pt-BR': ['pt'],
      'es-ES': ['es'],
      es: ['es'],
      pt: ['pt'],
      default: ['pt-BR']
    },
    backend: {
      loadPath: 'locales/{{lng}}/{{ns}}.json',
      reloadInterval: 10000,
    },
    debug: false
  })
  .then(() => {
    SetHichartConfig();
    moment.locale(i18next.language);

    const request = new Request();
    moment_timezone.tz.setDefault(request.getTimeZone());

    numeral.locale(i18next.t('configuration:locale'));

    // setInterval(() => {
    //   i18next.reloadResources();
    // }, 30000);
  });

export default i18next;

Expected behavior

As expected, I think it should just look for what is being set in load, in my case just the language as it's set as 'languageOnly'. I don't think I can explain in code.

Your Environment

runtime version: node v16, react, firefox, chrome, edge
i18next version: 19.8.4 (I've updated all my i18n packages but still the same, so I rolled back)
os: Linux
any other relevant information

One fix that I made was make a custom request fn, it works, but I still think it's a bug or something that I'm doing wrong. Code:

request: async (_options, url, _payload, callback) => {
    if (url) {
      let newUrl = url;
      const matches = newUrl.match(/pt-BR|es-ES/);
      if (matches && matches.length) {
        const language = matches[0].split('-')[0];
        newUrl = url.replace(matches[0], language);
      }
      const res = await fetch(newUrl);
      const data = await res.json();
      const callbackData = { data, status: 200 };
      callback(null, callbackData);
    }
  }

adrai commented 2 years ago

your supportedLngs array contains also the other languages... maybe removing them helps?

adrai commented 2 years ago

Maybe also the debug output would help?

zhyph commented 2 years ago

I removed 'pt-BR' and 'es-ES' from the supportedLngs array ['pt', 'es'], now it seems that it tries to search for the default fallback language on the fallbackLng object default: ['pt-BR'] and by removing it the reload never happens. Just to add more context, my lookupLocalStorage: LOCAL_STORAGE_LANGUAGE is always going to return my locale, because of how my project is setup.

zhyph commented 2 years ago

I noticed that when removing the default fallback language, the reload stops, and when changing it the reload grabs from whatever supported language you put there, any idea on how is this happening?

adrai commented 2 years ago

Can you set debug to true and share the logs?

zhyph commented 2 years ago

Sure thing. This is with the project in spanish es

The first load is fine, but after it searches for pt-BR

This one is without pt-BR and es-ES in the supportedLngs

My init in this print

{
  "debug": true,
  "initImmediate": true,
  "ns": [
    "accessManagement",
    "accountAnalysis",
    "analysisIncidentsPeriod",
    "attackAndFraudMap",
    "changePassword",
    "common",
    "configuration",
    "customRulesAnalysis",
    "deviceManagement",
    "environmentAnalysis",
    "errors",
    "faceSimilarityAnalysis",
    "feedUrl",
    "fraudAnalysis",
    "login",
    "maliciousEvents",
    "malwareAnalysis",
    "periodFilter",
    "presentationMode",
    "productAnalysis",
    "tokenManagement",
    "tokenParameterization",
    "transactionAnalysis",
    "transactionManagement",
    "transactionTypes",
    "transactionalOverview",
    "customColumns",
    "insights"
  ],
  "defaultNS": "common",
  "fallbackLng": {
    "pt-BR": [
      "pt"
    ],
    "es-ES": [
      "es"
    ],
    "es": [
      "es"
    ],
    "pt": [
      "pt"
    ],
    "default": [
      "pt-BR"
    ]
  },
  "fallbackNS": false,
  "whitelist": false,
  "nonExplicitWhitelist": false,
  "supportedLngs": [
    "pt",
    "es",
    "cimode"
  ],
  "nonExplicitSupportedLngs": false,
  "load": "languageOnly",
  "preload": false,
  "simplifyPluralSuffix": true,
  "keySeparator": ".",
  "nsSeparator": ":",
  "pluralSeparator": "_",
  "contextSeparator": "_",
  "partialBundledLanguages": false,
  "saveMissing": false,
  "updateMissing": false,
  "saveMissingTo": "fallback",
  "saveMissingPlurals": true,
  "missingKeyHandler": false,
  "missingInterpolationHandler": false,
  "postProcess": false,
  "postProcessPassResolved": false,
  "returnNull": true,
  "returnEmptyString": true,
  "returnObjects": false,
  "joinArrays": false,
  "returnedObjectHandler": false,
  "parseMissingKeyHandler": false,
  "appendNamespaceToMissingKey": false,
  "appendNamespaceToCIMode": false,
  "interpolation": {
    "escapeValue": false
  },
  "detection": {
    "lookupLocalStorage": "ad-language",
    "order": [
      "querystring",
      "cookie",
      "localStorage",
      "sessionStorage",
      "navigator",
      "htmlTag"
    ],
    "lookupQuerystring": "lng",
    "lookupCookie": "i18next",
    "lookupSessionStorage": "i18nextLng",
    "caches": [
      "localStorage"
    ],
    "excludeCacheFor": [
      "cimode"
    ]
  },
  "backend": {
    "loadPath": "locales/{{lng}}/{{ns}}.json",
    "reloadInterval": 10000,
    "addPath": "/locales/add/{{lng}}/{{ns}}",
    "allowMultiLoading": false,
    "customHeaders": {},
    "queryStringParams": {},
    "crossDomain": false,
    "withCredentials": false,
    "overrideMimeType": false,
    "requestOptions": {
      "mode": "cors",
      "credentials": "same-origin",
      "cache": "default"
    }
  }
}

And the error messages are the same as in the second screenshot, 404 looking for pt-BR in locales.

And if I remove fallbackLng it does not try to reload the resources, it only does the first load and just sits there.

My code in the first print:

supportedLngs: ['pt', 'pt-BR', 'es', 'es-ES'],
    fallbackLng: {
      'pt-BR': ['pt'],
      'es-ES': ['es'],
      es: ['es'],
      pt: ['pt'],
      default: ['pt-BR']
    },

and in the second one is the same but with supportedLngs like this ['pt', 'es']

When I console.log i18next.languages, i18next.language it logs [es], es-ES.

adrai commented 2 years ago

This looks really strange... if you just have es and pt, just remove all other languages, try to remove them everywhere... btw. remove also the old whitelist options and update all i18next dependencies

adrai commented 2 years ago

someone seems to change the language to es-ES

adrai commented 2 years ago

Are you calling changeLanguage somewhere? Or setting the lng option?

zhyph commented 2 years ago

Yep, unfortunately, I can't change my language to be 'es' because of this:

 if (languageCognito && languageLocalStorage != languageCognito) {
      i18next.changeLanguage(languageCognito, this._reloadPage);
    }

I use the language on cognito configuration to set my user language, and languageCognito tends to be 'pt-BR' or 'es-ES'. If I try to change everywhere in my code where a location is set, it becomes a whole loop where is doesn't match the language on cognito and the calls this._reloadPage function. But since this maybe the problem, one thing I can do is go with i18next.reloadResources because this looks like it's working or try to modify request in the backend plugin (as I've posted on the first comment). What do u think?

adrai commented 2 years ago

would be nice if I could reproduce this first

zhyph commented 2 years ago

I would try to reproduce it, but it really is a complex environment that I'm dealing with, a 6+ years old project with a lot of weird stuff happening. It looks like even tho the language is right, it's still trying to catch the default value on fallbackLng. I don't know how can I help you with this, but let me know if you need more info.

adrai commented 2 years ago

I think the main problem is your default fallback set like this:

"default": [
      "pt-BR"
    ]

the default fallback always needs to be "correct" => resolving to an existing language

change this to:

"default": [
      "pt"
    ]

zhyph commented 2 years ago

I think the main problem is your default fallback set like this:
"default": [
      "pt-BR"
    ]
the default fallback always needs to be "correct" => resolving to an existing language

change this to:
"default": [
      "pt"
    ]

I've tried this one, what happens is that it works on pt-BR since the default is set like this, but when I change to es-ES it falls on pt still, even though the language is a different one.

adrai commented 2 years ago

I can just say, having that default to pt-BR is 100% wrong. And I'm able to reproduce your issue with this. But switching to es-ES I've no problem if the default fallback is pt... it falls back to es. Sorry, but I'm not able to help more than this. so tldr;

your whole setup seems very strange
the default fallback is for sure 100% wrong => change to pt

zhyph commented 2 years ago

Sure, thanks for your help. I'm going to try this with all the packages at latest. See if this maybe is an older version problem.

i18next / i18next-http-backend