Traceback billing to city with name outside ASCII

[GoogleCodeExporter]

Trying GetPaid buildout from trunk, rev 1225, I get a traceback if I enter
a city with a name that has non-ASCII characters, for example my place of
residence, "Linköping". Here's the traceback:

2008-02-03 13:01:49 ERROR Zope.SiteErrorLog
http://uterus:8080/Plone/@@getpaid-checkout-wizard
Traceback (innermost last):
  Module ZPublisher.Publish, line 119, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module Products.PDBDebugMode.pdbzope.runcall, line 60, in pdb_runcall
  Module ZPublisher.Publish, line 42, in call_object
  Module Products.PloneGetPaid.browser.checkout, line 298, in __call__
  Module zope.formlib.form, line 770, in __call__
  Module Products.PloneGetPaid.browser.checkout, line 176, in render
  Module zope.formlib.form, line 764, in render
  Module Shared.DC.Scripts.Bindings, line 313, in __call__
  Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
  Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
  Module Products.PageTemplates.PageTemplate, line 89, in pt_render
  Module zope.pagetemplate.pagetemplate, line 117, in pt_render
  Module zope.tal.talinterpreter, line 271, in __call__
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 891, in do_useMacro
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 521, in do_optTag
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 891, in do_useMacro
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 521, in do_optTag
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 949, in do_defineSlot
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 949, in do_defineSlot
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 521, in do_optTag
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 745, in do_insertStructure_tal
  Module Products.PageTemplates.Expressions, line 221, in evaluateStructure
  Module zope.tales.tales, line 696, in evaluate
   - URL: checkout-review-pay
   - Line 11, Column 4
   - Expression: <PathExpr standard:'view/hidden_inputs'>
   - Names:
      {'container': <PloneSite at /Plone>,
       'context': <PloneSite at /Plone>,
       'default': <object object at 0xb7d58530>,
       'here': <PloneSite at /Plone>,
       'loop': {},
       'nothing': None,
       'options': {'args': ()},
       'repeat': <Products.PageTemplates.Expressions.SafeMapping object at
0xdd4b68c>,
       'request': <HTTPRequest,
URL=http://uterus:8080/Plone/@@getpaid-checkout-wizard>,
       'root': <Application at >,
       'template': <ImplicitAcquirerWrapper object at 0xdd4be0c>,
       'traverse_subpath': [],
       'user': <PloneUser 'forsberg'>,
       'view': <Products.Five.metaclass.CheckoutReviewAndPay object at
0xecce6ac>,
       'views': <zope.app.pagetemplate.viewpagetemplatefile.ViewMapper
object at 0xdd4b80c>}
  Module zope.tales.expressions, line 217, in __call__
  Module Products.PageTemplates.Expressions, line 153, in _eval
  Module zope.tales.expressions, line 124, in _eval
  Module Products.PageTemplates.Expressions, line 103, in
trustedBoboAwareZopeTraverse
  Module zope.traversing.adapters, line 164, in traversePathElement
   - __traceback_info__: (<Products.Five.metaclass.CheckoutReviewAndPay
object at 0xecce6ac>, 'hidden_inputs')
  Module zope.traversing.adapters, line 44, in traverse
   - __traceback_info__: (<Products.Five.metaclass.CheckoutReviewAndPay
object at 0xecce6ac>, 'hidden_inputs', [])
  Module Products.PloneGetPaid.browser.checkout, line 100, in hidden_inputs
  Module ZTUtils.Zope, line 207, in make_hidden_input
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in
position 4: ordinal not in range(128)
*** NameError: name 'pdb' is not defined
>
/home/forsberg/dev/plone/getpaid.buildout/parts/zope2/lib/python/ZTUtils/Zope.py
(207)make_hidden_input()
-> qlist[i] = ('<input type="hidden" name="%s%s" value="%s">'

Original issue reported on code.google.com by forsber...@gmail.com on 3 Feb 2008 at 12:05

[GoogleCodeExporter]

It also happens for accented names, or any non-ascii character entered in the 
address collection form.
It seems to me that replacing the line in 
Products/PloneGetPaid/browser/checkout.py 
that reads:

v = unicode(v, encoding='utf8')

with:
if type(v) is not unicode:
    v = unicode(v, encoding='utf8')

solves the problem.

Original comment by silviot on 11 Apr 2009 at 3:51

Attachments:

• getpaid#175.patch

[GoogleCodeExporter]

As Maurits van Rees explained it on his blog in Search forms with zope formlib 
and
batching

There is a difference between getting a value from the request or from 
request.form
when unicode is involved:

>>> request.form.get('SearchableText')
u'\xff'
>>> request.get('SearchableText')
'\xc3\xbf'
In the class BaseFormView of base.py, adding the 5 lines between the #
    def setUpWidgets( self, ignore_request=False ):
        self.adapters = self.adapters is not None and self.adapters or {}
        # request unicode matters
        fieldnames = [x.__name__ for x in self.form_fields]
        for key in fieldnames:
            value = self.request.form.get(key)
            if value is not None and not value == u'':
                self.request[key] = value
        #
        ...

solves the problem ...
Thanks Maurits

Original comment by danielle...@gmail.com on 17 Apr 2010 at 11:54

[GoogleCodeExporter]

It seems that it doesn't now  happened anymore
with the lines added in browser.checkout 
        try:
            # try to convert to unicode first, because str() of the unicode string may fail
            v = unicode(v)
        except UnicodeDecodeError:
            v = str(v)
but the comment doesn't give a clue of the "why" side... so there is a risk 
that this try, except will be stay for ever even if there is no more problem at 
the root, IMHO

Original comment by danielle...@gmail.com on 1 Jan 2011 at 12:28

• Changed state: Started