Doc write exception with unicode characters

GoogleCodeExporter commented 8 years ago

What steps will reproduce the problem?

The following test script fails with 0.9, but works fine with 0.8:

import json
import couchdb

db = couchdb.Server(u"https://<username>:<password>@<server>")['<database>']
docs = [{u'Type':"TestDoc", u'Test':True, u'val':unichr(40960)}]
ret = db.update( docs )
print ret

What is the expected output? What do you see instead?

The write should succeed.

The exception is:

  File "couch_encoding_test.py", line 10, in <module>
    ret = db.update( docs )
  File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.9-py2.7.egg/couchdb/client.py", line 764, in update
    _, _, data = self.resource.post_json('_bulk_docs', body=content)
  File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.9-py2.7.egg/couchdb/http.py", line 527, in post_json
    **params)
  File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.9-py2.7.egg/couchdb/http.py", line 546, in _request_json
    headers=headers, **params)
  File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.9-py2.7.egg/couchdb/http.py", line 542, in _request
    credentials=self.credentials)
  File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.9-py2.7.egg/couchdb/http.py", line 327, in request
    resp = _try_request_with_retries(iter(self.retry_delays))
  File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.9-py2.7.egg/couchdb/http.py", line 285, in _try_request_with_retries
    return _try_request()
  File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.9-py2.7.egg/couchdb/http.py", line 307, in _try_request
    conn.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 812, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xea in position 52: 
ordinal not in range(128)

What version of the product are you using? On what operating system?

couchdb-python 0.9, ubuntu 12.04

Please provide any additional information below.

If the parameter to couchdb.Server is a str instead of a unicode object, the 
script executes correctly.

Original issue reported on code.google.com by hhmw56...@gmail.com on 28 Apr 2014 at 9:42

GoogleCodeExporter commented 8 years ago

I tried this test case and couldn't get it to fail on the current tip of the 
default branch:

diff --git a/couchdb/tests/client.py b/couchdb/tests/client.py
--- a/couchdb/tests/client.py
+++ b/couchdb/tests/client.py
@@ -185,6 +185,10 @@
         self.assertEqual(u'Iñtërnâtiônàlizætiøn', self.db[u'føø'][u'bår'])
         self.assertEqual(u'ASCII', self.db[u'føø'][u'baz'])

+    def test_unicode_235(self):
+        doc = {u'type': 'test', u'val': unichr(40960)}
+        self.db.update([doc])
+
     def test_disallow_nan(self):
         try:
             self.db['foo'] = {'number': float('nan')}

This makes me think this has been fixed already. If you can still reproduce it 
with a more recent changeset, feel free to reopen.

Original comment by djc.ochtman on 6 Jul 2014 at 11:02

Changed state: WorksForMe

GoogleCodeExporter commented 8 years ago

"If the parameter to couchdb.Server is a str instead of a unicode object, the 
script executes correctly."

In other words, the behaviour changes depending on whether the couchdb.Server 
object is initialized with a string or a unicode string.

Original comment by mikebell...@gmail.com on 15 Jul 2014 at 6:26

GoogleCodeExporter commented 8 years ago

Right. Have you tried reproducing with current tip? Because I still can't.

Original comment by djc.ochtman on 15 Jul 2014 at 6:57

GoogleCodeExporter commented 8 years ago

I have reproduced the problem in 0.10 and the latest couchdb-python from 
github. 

Looking at your test code it is not obvious to me that the Server object is 
being initialized with a unicode string, which is necessary to trigger the 
behavior I'm describing.

Original comment by hhmw56...@gmail.com on 16 Jul 2014 at 6:13

GoogleCodeExporter commented 8 years ago

This issue has been migrated to GitHub. Please continue discussion here:

https://github.com/djc/couchdb-python/issues/235

Original comment by djc.ochtman on 26 Jul 2014 at 6:28

charveasna / couchdb-python

Doc write exception with unicode characters #235