couchdb-python doesn't support locales

[GoogleCodeExporter]

The couchdb-python cache system is based on the RFC 2822 Date system for object 
expiration. When the locale changes, strptime() is no more able to parse this 
kind of date.

What steps will reproduce the problem?
1. Run this (or see attachment):

import locale
locale.setlocale(locale.LC_ALL, '')

import couchdb
import couchdb.http

# disable cache. 0 means force cache validation
# on every request.
couchdb.http.CACHE_SIZE = 10, 0

couch = couchdb.Server()
try:
    db = couch.create('test')
    db['foo'] = {'bar': 'test'}
    print db['foo']
finally:
    couch.delete('test')

What is the expected output? What do you see instead?

Expected:
<Document u'foo'@u'1-1d1b1ba13d97badec02a57adb942cd42' {u'bar': u'test'}>

Instead:
Traceback (most recent call last):
  File "plop.py", line 15, in <module>
    print db['foo']
  File "c:\Pyramid\lib\site-packages\couchdb\client.py", line 322, in __getitem__
    _, _, data = self.resource.get_json(id)
  File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 393, in get_json
    status, headers, data = self.get(*a, **k)
  File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 374, in get
    return self._request('GET', path, headers=headers, **params)
  File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 419, in _request
    credentials=self.credentials)
  File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 316, in request
    self._clean_cache()
  File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 324, in _clean_cache
    ls = sorted(self.cache.iteritems(), key=cache_sort)
  File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 84, in cache_sort
    t = time.mktime(time.strptime(i[1][1]['Date'][5:-4], '%d %b %Y %H:%M:%S'))
  File "c:\Python27\Lib\_strptime.py", line 454, in _strptime_time
    return _strptime(data_string, format)[0]
  File "c:\Python27\Lib\_strptime.py", line 325, in _strptime
    (data_string, format))
ValueError: time data '21 Feb 2012 15:18:38' does not match format '%d %b %Y 
%H:%M:%S'

What version of the product are you using? On what operating system?
- CouchDB-0.8-py2.7
- WinXP32 SP3

Please provide any additional information below.

IMHO the best solution is to remove the month part of the date (so as to only 
have numbers which should be locale independant) before passing it to 
strptime() (and to adjust the parse pattern accordingly)

Original issue reported on code.google.com by emmanuel...@gmail.com on 21 Feb 2012 at 3:26

Attachments:

• plop.py

[GoogleCodeExporter]

Looks like windows-only bug or I couldn't reproduce it on Linux system, but 
could on Windows one.

There are two options: wait for @Matt http module and cache system refactoring 
or quick fix current behavior. I'm for first one.

Original comment by kxepal on 21 Feb 2012 at 3:46

[GoogleCodeExporter]

Something is missing: the default locale, or the choosen locale shouldn't be in 
english. Since the locale name is OS dependant (fr_FR on Linux, French_France 
on Windows), I didn't include it in the example.

Original comment by emmanuel...@gmail.com on 21 Feb 2012 at 4:00

[GoogleCodeExporter]

This is important note, thanks!

# Quick fix for your example:
import time
from email.Utils import parsedate
from datetime import datetime
def cache_sort(i):
    t = time.mktime(parsedate(i[1][1]['Date']))
    return datetime.fromtimestamp(t)
# And monkey-patch cache_sort function:
couchdb.http.cache_sort = cache_sort

This should work! At least for me with ('ru_RU', 'cp1251') locale info. I'll 
prepare patch and tests later.

Original comment by kxepal on 21 Feb 2012 at 4:20

[GoogleCodeExporter]

Original comment by kxepal on 5 Oct 2012 at 8:58

• Changed state: Accepted
• Added labels: Milestone-0.9

[GoogleCodeExporter]

Anyone has ideas how to write test case that depended on some non en_US locale? 
If I'll set some ru_RU this test will pass for me, but not for those who hasn't 
this locale enabled. Same thing is about fr_FR one that is missed for me.

Original comment by kxepal on 10 Oct 2012 at 8:44

[GoogleCodeExporter]

Maybe do some trickery with unittest.skip?  Check for the presence of the 
locale at module import time, and conditionally skip the test?

Original comment by wickedg...@gmail.com on 11 Oct 2012 at 6:02

[GoogleCodeExporter]

unittest.skip will bring unittest2 dependency which couldn't help with prevent 
problem in future. Also it's impossible to check locale because it could not be 
available - as for me I have ('ru_RU', 'cp1251') while 'fr_FR' is missed for me.

There are two solutions:
1. Make tests for Windows only against ANSI encoding since it brings a problem, 
while Linux long ago has *.UTF-8 system encoding by default.

2. Since this bug dependent on OS environment AND processed Date value follows 
well known rules[1][2] it will be right decision to use right ways[3][4][4] to 
parse such value. Tests are redundant in this case since there is actually 
nothing to test - Python functions are well tested by his community and bug 
occurs not because of couchdb-python logic code.  

As for me, second variant is better because:
- it follows RFC like parsed value does;
- it is not locale dependent;
- I don't see any portable and stable ways to test locale specific things 
without providing some dependency overhead (babel and mock may help there);

Patch attached. Thoughts?

[1]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.18
[2]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.3.1
[3]: http://docs.python.org/library/email.util.html#email.utils.parsedate
[4]: 
http://stackoverflow.com/questions/1471987/how-do-i-parse-an-http-date-string-in
-python
[5]: 
http://stackoverflow.com/questions/225086/rfc-1123-date-representation-in-python

Original comment by kxepal on 11 Oct 2012 at 11:35

Attachments:

• couchdb-python-209-r1.patch

[GoogleCodeExporter]

This issue was closed by revision b7ada64cd590.

Original comment by kxepal on 12 Oct 2012 at 8:02

• Changed state: Fixed