RBNF missing after upgrading

juanperi commented 2 years ago

I've updated from 2.25.2 to 2.26.0, and suddenly I started seeing the following error for nb locale:

iex(1)> IO.inspect(SylarLocalization.Cldr.put_locale("nb"), structs: false)
{:ok,
 %{
   __struct__: Cldr.LanguageTag,
   backend: SylarLocalization.Cldr,
   canonical_locale_name: "nb",
   cldr_locale_name: :nb,
   extensions: %{},
   gettext_locale_name: "nb",
   language: "nb",
   language_subtags: [],
   language_variants: [],
   locale: %{},
   private_use: [],
   rbnf_locale_name: nil,
   requested_locale_name: "nb",
   script: :Latn,
   territory: :NO,
   transform: %{}
 }}
{:ok, #Cldr.LanguageTag<nb [validated]>}
iex(2)> SylarLocalization.Cldr.Number.to_string(1234)
{:error, {Cldr.Rbnf.NotAvailable, "RBNF is not available for the locale :nb"}}

This was not happening in the previous version, though, where I would get:

iex(1)> IO.inspect(SylarLocalization.Cldr.put_locale("nb"), structs: false)
{:ok,
 %{
   __struct__: Cldr.LanguageTag,
   backend: SylarLocalization.Cldr,
   canonical_locale_name: "nb",
   cldr_locale_name: :nb,
   extensions: %{},
   gettext_locale_name: "nb",
   language: "nb",
   language_subtags: [],
   language_variants: [],
   locale: %{},
   private_use: [],
   rbnf_locale_name: :nb,
   requested_locale_name: "nb",
   script: :Latn,
   territory: :NO,
   transform: %{}
 }}
{:ok, #Cldr.LanguageTag<nb [validated]>}
iex(2)> SylarLocalization.Cldr.Number.to_string(1234)
{:ok, "1¬†234"}

Is this a bug? or is it related to the upgrade to release 41 (Update CLDR to [release 41] as per the changelog) and it is expected that we might lose access to localized numbers after cldr releases update?

Thank you very much for your help figuring this out!!

kipcole9 commented 2 years ago

Thanks for the report and sorry for the inconvenience. Its a bug for sure, and its surfaced because of some changes to the underlying CLDR data. As of CLDR-14493 the RBNF data for locale "nb" was removed. This coincided with a change to the locale hierarchy in which "nb" became a child locale to "no" (where "child" is used in the sense of locale inheritance). It seems it has taken quite some time for that change to ripple through the JSON generator that ultimately produces the majority of the data that ex_cldr uses.

TLDR; Yes, the RBNF data for "nb" has gone. However I suspect the intent is that the RBNF data for "no" is intended to be used.

Is is reasonable, from a language and linguistic point of view, that the RBNF rules for "no" are also appropriate and applicable for "nb"?

If I assume it is, then most definitely I am not applying any form of inheritance in resolving which RBNF rules to use. It will be a reasonably straight forward change to do so, but it will take a few days to be able to do that work.

kipcole9 commented 2 years ago

@juanperi, as of commit 3ad972, RBNF locale name resolution now inherits from a parent locale if there is one (and the parent for "nb" is "no" which does have RBNF data). Commit 687fb5 regenerates the known language tags and adds a test.

All tests pass so I am ready to release ex_cldr version 2.29.0 if you are comfortable that the RBNF for "no" is acceptable for locale "nb".

juanperi commented 2 years ago

i think it is totally acceptable. in fact, i talked a couple of times with some translators asking if we should use no directly. nobody could give me a direct answer 😂😃

btw man you were really fast at this. as always, thank you very much for your awesome work!

kipcole9 commented 2 years ago

Turns out it wasn't that difficult to fix. But I greatly appreciate the encouragement, it really helps with motivation.

I have published ex_cldr version 2.29.0 with the following changelog entry:

Migration

The plugs Cldr.Plug.SetLocale, Cldr.Plug.AcceptLanguage and Cldr.Plug.PutSession have been extracted to their own library, ex_cldr_plug. Therefore adding {:ex_cldr_plug, "~> 1.0"} to the deps of any application using these plugs is required.

Bug Fixes

Fixes resolving the RBNF locale name for locales that inherit the RBNF locale from a parent. This is true for at least the "nb" locale which in previous releases had its own RBNF locale data but now inherits from "no". Thanks to @juanperi for the report. Closes #175.

I'll leave this issue open for a little bit in case there is an issue I didn't find in testing.

Appreciate your collaboration!

juanperi commented 2 years ago

in production already! working as expected!!

thank you very much!

have a nice day 😊

kipcole9 commented 2 years ago

Great to hear, thanks for the feedback. Closing!

elixir-cldr / cldr

RBNF missing after upgrading #175

Migration

Bug Fixes