Better handling of InvalidPluralizationData

[nikosd]

I'm having some hard times trying to handle errors related to missing translations for languages with non 'germanic' pluralization forms (for example Polish, Russian, etc).

The problem / current behaviour

The problem comes up when the following scenario is true :

# Given the locale is `ru`
I18n.locale = :ru

# And the pluralization key for `2` is :few
pluralizer.call(2) # => :few

# And the ru messages are missing (ru.po)
#
# msgid "singular text"
# msgid_plural "plural text"
# msgstr[0] ""
# msgstr[1] ""
# msgstr[2] ""

# Asking for translations with count => 2 raises exception
n_('singular text', 'plural text', 2) # => raises InvalidPluralizationData

Wanted behaviour

Instead I would like the following :

# Asking for translations with count => 2 should return the default plural form
n_('singular text', 'plural text', 2) # => 'plural text'

Current (hack) workaround

To actually make this work in our own applications I have made the following monkey patch on the pluralization module :

module I18n
  module Backend
    module Pluralization
      # Overriding the pluralization method so if the proper plural form is missing we will try
      # to fallback to the default gettext plural form (which is the `germanic` one).
      def pluralize(locale, entry, count)
        return entry unless entry.is_a?(Hash) and count

        pluralizer = pluralizer(locale)
        if pluralizer.respond_to?(:call)
          return entry[:zero] if count == 0 && entry.has_key?(:zero)

          plural_key = pluralizer.call(count)
          return entry[plural_key] if entry.has_key?(plural_key)

          # fallback to the default gettext plural forms if real entry is missing (for example :few)
          default_gettext_key = count == 1 ? :one : :other
          return entry[default_gettext_key] if entry.has_key?(default_gettext_key)

          # If nothing is found throw the classic exception
          raise InvalidPluralizationData.new(entry, count)
        else
          super
        end
      end
    end
  end
end

but this is not the right way to do it since it's way too opinionated.

Possible approches / Ideal solutions

Ideally, any of the following would be good solutions :

A. Using the generic I18n.exception_handler

# config/initializers/i18n.rb
module I18n
  def self.handle_invalid_pluralization_data(*args)
    exception = args.first
    if exception.is_a?(InvalidPluralizationData)
      # do your thing here ...
    else
      super
    end
  end
end

I18n.exception_handler = :handle_invalid_pluralization_data

to make this work something InvalidPluralizationData should become a MissingTranslation which makes sense to me also in terms of semantics, since it's not the case that the localization data are invalid, it's that a specialized localization datum is missing.

alternatively :

B. Using a specialized I18n::Backend::Pluralization.invalid_pluralization_handler

# config/initializers/i18n.rb
I18n::Backend::Pluralization.invalid_pluralization_handler = Proc.new do |locale, key, count|
  # to the handling here
end

which means that the pluralization backend should be changed to something like :

module I18n
  module Backend
    module Pluralization
      # ...
      def pluralize(locale, entry, count)
        return entry unless entry.is_a?(Hash) and count

        pluralizer = pluralizer(locale)
        if pluralizer.respond_to?(:call)
          key = count == 0 && entry.has_key?(:zero) ? :zero : pluralizer.call(count)
          entry.has_key?(key) ? entry[key] : pluralization_error_proc.call(locale, entry, count)
        else
          super
        end
      end

      # ...
    end
  end
end

[nikosd]

For the record this is how C# Gettext is handling this issue by default (yikes) :

    public virtual String GetPluralString (String msgid, String msgidPlural, long n) {
      Object value = GetObject(msgid);
      if (value == null || value is String)
        return (String)value;
      else if (value is String[]) {
        String[] choices = (String[]) value;
        long index = PluralEval(n);
        return choices[index >= 0 && index < choices.Length ? index : 0];
      } else
        throw new InvalidOperationException("resource for \""+msgid+"\" in "+GetType().FullName+" is not a string");
    }

[svenfuchs]

Interesting.

Lemme first see if I get the issue right.

So, you have pluralization data that does not fit the current pluralization algorithm and you want to pick a default, but there's no way to handle that.

Does the default not depend on the current locale as well? I could imagine it's different for different locales?

What exactly would you do in handle_invalid_pluralization_data where you say "do your thing here"?

[nikosd]

A simple example of a graceful fallback :

# config/initializers/i18n.rb
module I18n
  def self.handle_invalid_pluralization_data(*args)
    exception = args.first
    if exception.is_a?(InvalidPluralizationData)
      key = exception.count == 1 ? :one : :other
      return exception.entry[key] if exception.entry.has_key?(key)
    else
      super
    end
  end
end

I18n.exception_handler = :handle_invalid_pluralization_data

where the super actually means call the default exception handler (I'm not sure that this example would work)

Does this make it any clearer or should add an exact scenario with english default values (singular/plural) and polish as the requested language (I18n.locale)?

[marcusg]

+1

any updates here?

[merlagautham]

+1

In our case if the translator does not give all the plural form I just want to fall back to the default language (English in our case) instead of showing an exception to end users.

[tigrish]

The way we handle this when we export data on Locale is to use the :other form. AFAIK, this form is defined by all pluralization rules.

[mixmix]

I think i'm striking an issue similar to this with some asian languages (japanese, indonesian, vietnamese) where there is no difference between singular and plural. In this case Transifex records only generate the :other translation, and our app is crashing because it is unnecessarily looking for the singular translation :one

Is there a tidy solution to this?

[jsilland]

+1 to resolving this issue. As it currently stands, there are basically no ways to properly recover from a missing plural form. An acceptable fall back would be to switch back the locale to English in this case, as is the case for when a translation is simply missing.

[Startouf]

I also run into frequent translations errors for French, mainly related to using :many instead of :other, etc. I believe there are already some merge requests to fix this issue,

[radar]

I would accept a PR here to fix this issue.

[sandstrom]

How widespread is gettext for translation keys? Anyone know what the most popular methods for storing translations are these days?

My guess would be that most are yml, json or some database (redis, sql, etc). If anyone know I'd be very happy to hear!

[azyzio]

Hi, has there been any progress here? The lack of fallbacks requires lot of code repetition in the YML translation files

[sandstrom]

@azyzio Would you be willing to put together a PR to solve this?

A simple start could be a brief outline of the suggested changes + public api changes. Just a few paragraphs and some code.