search

zopefoundation / zope.i18n

APIs and data related to internationalization and localization.
Other
2 stars 13 forks source link

CLDR XML data is from 2004-ish #40

Open agroszer opened 5 years ago

agroszer commented 5 years ago

putting CLDR 34 data in of course creates failures

Tests with errors:
   test_getDateTimeFormatter (zope.i18n.locales.tests.test_locales.TestLocaleAndProvider)
   test_getTimeFormatter (zope.i18n.locales.tests.test_locales.TestLocaleAndProvider)
   test_dateFormatter (zope.i18n.locales.tests.test_locales.TestRootLocale)

Tests with failures:
   test_getDateFormatter (zope.i18n.locales.tests.test_locales.TestLocaleAndProvider)
   test_getNumberFormatter (zope.i18n.locales.tests.test_locales.TestLocaleAndProvider)
   test_getLocale (zope.i18n.locales.tests.test_locales.TestLocaleProvider)
agroszer commented 5 years ago

also zope.i18n 4.6.2 release size is 800K, this would boast it to over 6M

agroszer commented 5 years ago

well CLDR data structure changed a LOT between 2004 and version 34, it's not just a simple copy-over-the-new-xmls

jensens commented 4 years ago

IMO it would be better to depend on PyICU instead of doing all the work on our own. PyICU uses the ICU C-Implementation which is the well-maintained standard implementation for CLDR locales used by many of the IT giants.

Doing this on out own is non-trivial and probably not our main focus.

jamadden commented 4 years ago

I completely agree that it would be nice to drop the problem of keeping the locale data on someone else. However,PyICU looks like a big dependency. They don't ship any binaries, only an sdist, and "Before building PyICU the ICU libraries must be built and installed. Refer to each system's instructions for more information." It's not clear to me that the ICU libraries are a standard part of every system we would like to support.

I personally had no problems building them on my mac, but I have ICU installed via a third-party package manager. It wound up linking to three libraries, /opt/local/lib/libicui18n.65.dylib, /opt/local/lib/libicuuc.65.dylib, and /opt/local/lib/libicudata.65.dylib. The system library directory only has /usr/lib/libicucore.A.dylib; does Apple provide a full implementation of ICU? I don't know, but I do know that libicudata.65.dylib is 27M, whereas libicucore is only 3M, so if they do they keep the data somewhere else.

jensens commented 4 years ago

Yes, no bdist or wheels, bad enough. It seems the author dislikes binary releases https://github.com/ovalhub/pyicu/issues/119

It's at least an idea. I did not look if there are other solutions around yet.

I just stumbled over this, because the decimal parser here (used by z3c.form) behaves with a de_CH locale to parse "1.234" but fails with "1.2345" and the fix is to edit the line here https://github.com/zopefoundation/zope.i18n/blob/90506d35e131d72adcb8bb92d43d4bd3af9bb1bc/src/zope/i18n/locales/data/de_CH.xml#L43 to have more '#' at the end.

agroszer commented 1 month ago

bites again... <territory type="SP">Serbia</territory> vs. Serbia ISO3166 RS

:-(