Some (3 char) IETF BCP 47 language tags returning the wrong language (2 char)

[mschoeffmann]

🐛 Bug Report

Some "3-character" IETF BCP 47 language tags are recognized as a different language (just shortened to 2 characters).

For example: Setting a Chrome Browser to "Hawaiian" should return the official tag haw, but now it returns ha which is the code for "Hausa". Also "Konkani" (=kok), which is in the official IETF list, returns ko which is "Korean".

So even if this middleware is only meant to detect "2ch" ISO 639-1 codes, it should at least not return wrong languages.

Reference Articles:

• https://en.wikipedia.org/wiki/List_of_ISO_639-2_codes
• https://en.wikipedia.org/wiki/IETF_language_tag

Extension for testing it in Chrome:

• https://chrome.google.com/webstore/detail/locale-switcher/kngfjpghaokedippaapkfihdlmmlafcc

To Reproduce

Un-commenting lookupHeader and lookupHeaderRegex in this example returns the right values in this use-case.

// middleware definition
const i18next = require('i18next');
const i18nextBackend = require('i18next-fs-backend');
const i18nextMiddleware = require('i18next-http-middleware');

i18next
  .use(i18nextBackend)
  .use(i18nextMiddleware.LanguageDetector)
  .init({
    backend: {
      loadPath: 'lang/{{lng}}.json',
    },
    detection: {
      order: ['header'],
      // lookupHeader: 'accept-language',
      // lookupHeaderRegex: /(([a-z]{2,3})-?([A-Z]{2,4})?)\s*;?\s*(q=([0-9.]+))?/gi,
    },
    fallbackLng: 'en',
    preload: ['en'],
  });

module.exports.handle = i18nextMiddleware.handle(i18next);

// adding middleware to app
app.use(i18n.handle);

// somewhere in a controller
console.log(req.header('accept-language'));
console.log(req.i18n.languages);

Make request in Hawaiian (US), Hawaiian, Kokani and with Google Chrome, the extension mentioned above will help ...

haw-US
[ 'ha', 'en' ]
haw
[ 'ha', 'en' ]
kok
[ 'ko', 'en' ]

Expected behavior

Returning the correct locale, or at least not the wrong one:

haw-US
[ 'haw-US', 'en' ]
haw
[ 'haw', 'en' ]
kok
[ 'kok', 'en' ]

Your Environment

• runtime version: node 16.19, express 4.18.2
• i18next version: 22.4.9
• i18next-http-middleware version: 3.2.1
• os: Mac, Windows, Linux

[adrai]

v3.2.2 should address this

[mschoeffmann]

Works like a charm. Thank you!

[adrai]

If you like this module don’t forget to star this repo. Make a tweet, share the word or have a look at our https://locize.com to support the devs of this project.

There are many ways to help this project 🙏