faulty NumberDataConverter for Decimal?

[agitator]

Plone 5.2 on Py 3.7

I have a behavior with a decimal field

    item_net = schema.Decimal(
        title=_(u"label_item_net", default=u"Item net price"),
        required=False,
        defaultFactory=default_item_net,
        min=Decimal(0),
    )

Enter 1.7641597028783658 View 19000000000000000.00 Reedit 17.641.597.028.783.658

Stepping into NumberDataConverter

[18] > /Users/peter/workspace/project/buildout/eggs/z3c.form-3.7.0-py3.7.egg/z3c/form/converter.py(123)toFieldValue()
-> if value == u'':
(Pdb++) value
'1.7641597028783658'
(Pdb++) ll
 120         def toFieldValue(self, value):                                                                                                                                                                                                 
 121             import pdb; pdb.set_trace()                                                                                                                                                                                                
 122             """See interfaces.IDataConverter"""                                                                                                                                                                                        
 123  ->         if value == u'':                                                                                                                                                                                                           
 124                 return self.field.missing_value                                                                                                                                                                                        
 125             try:                                                                                                                                                                                                                       
 126                 return self.formatter.parse(value)                                                                                                                                                                                     
 127             except zope.i18n.format.NumberParseError:                                                                                                                                                                                  
 128                 raise FormatterValidationError(self.errorMessage, value)                                                                                                                                                               
(Pdb++) n
[18] > /Users/peter/workspace/project/buildout/eggs/z3c.form-3.7.0-py3.7.egg/z3c/form/converter.py(125)toFieldValue()
-> try:
(Pdb++) self.formatter.parse(value)
Decimal('17641597028783658')

Can someone give me a hint why this is happening? Or any pointer on how to fix?

[jamadden]

self.formatter is defined as

locale = self.widget.request.locale
self.formatter = locale.numbers.getFormatter('decimal')

So it might probably be helpful to know what locale you're in and what the formatter winds up being.

[agitator]

(Pdb++) locale.getLocaleID()
'de'

so the fix should be in a po-file?

[agitator]

hmm but either way it should convert it the same way

[jamadden]

zope.i18n's 'de' locale uses '.' as the group separator, so that much appears consistent:

>>> from zope.i18n import locales
>>> de = locales.locales.getLocale('de')
>>> de.numbers.getFormatter('decimal').getPattern()
'#,##0.###;-#,##0.###'
>>> de_f = de.numbers.getFormatter('decimal')
>>> de_f.symbols
{'decimal': ',',
 'exponential': 'E',
 'group': '.',
 'infinity': '∞',
 'list': ';',
 'minusSign': '-',
 'nan': '�',
 'nativeZeroDigit': '0',
 'patternDigit': '#',
 'perMille': '‰',
 'percentSign': '%',
 'plusSign': '+'}
>>> de.numbers.getFormatter('decimal').parse('1.7641597028783658')
17641597028783658
>>> de.numbers.getFormatter('decimal').format(17641597028783658)
'17.641.597.028.783.658'

I don't know where 19000000000000000.00 is coming from, it looks like rounding.

The English locale appears to share the same format string:

>>> de.numbers.getFormatter('decimal').getPattern()
'#,##0.###;-#,##0.###'
>>> en = locales.locales.getLocale('en')
>>> en.numbers.getFormatter("decimal").getPattern()
'#,##0.###;-#,##0.###'

But they differ in their symbols:

>>> de_f.symbols['decimal']
','
>>> en_f.symbols['decimal']
'.'

The english locale won't parse the number; it won't parse it because of too many decimal digits (too much precision) as 3 digits appears to be the most the pattern allows:

>>> en.numbers.getFormatter('decimal').parse('1.7641597028783658')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "//python3.8/site-packages/zope/i18n/format.py", line 315, in parse
    raise NumberParseError('Not a valid number for this pattern %r.'
zope.i18n.format.NumberParseError: Not a valid number for this pattern '#,##0.###;-#,##0.###'.
>>> en.numbers.getFormatter("decimal").parse('17641597028783.658')
17641597028783.656

It does format it as I would expect, in the same way the de format does:

>>> en.numbers.getFormatter('decimal').format(17641597028783658)
'17,641,597,028,783,658'

[agitator]

Yes, 19000000000000000.00is from rounding and some taxes in the view

Copying over a value osx (locale en) calculator to plone site (locale de)

toFieldValue: 1.5
FormatterParseToFieldValue: 15

toFieldValue: 1,5
FormatterParseToFieldValue: 1.5

So it seems to be the wrong decimal delimiter... Everything works then as expected... WOW!

Thanx @jamadden for the enlightenment 👍

[agitator]

Maybe it would be an idea to simplify that code? Get the first nonnumerical character from the right side as delimiter and delete all grouping characters?

[icemac]

That's not always what you want:

1.234.567 should become Decimal('1234567') with locale='de', not Decimal('1.234567') as this would be more than surprising for German natives.

I'd suggest to close this issue and leave the code as it is. Okay?

[agitator]

@icemac you're right Thx again for all the feedback!