Problems with http cache in multitheaded environment.

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Create multitheaded script
2. Share Database instance between threads 
3. Make a lot of requests until crush happens (see exception.log attachment)

There are two different exceptions, but I suppose that they have same nature.

I'll make test script and patch a little bit later, just have created issue to 
not forget about this problem.

Original issue reported on code.google.com by kxepal on 26 Nov 2011 at 4:36

Attachments:

• exception.log

[GoogleCodeExporter]

Attached patch and test case

Original comment by kxepal on 27 Nov 2011 at 12:11

Attachments:

• 02-issue-208-http.patch

[GoogleCodeExporter]

Forgot to add test case to suite, also named it well.

Original comment by kxepal on 27 Nov 2011 at 1:02

Attachments:

• 01-issue-208-test_http.patch

[GoogleCodeExporter]

That looks like it should work, thanks!

I extracted the cache code from http.Session recently to make it more testable 
but haven't pushed it yet. My fault entirely, but do you mind if I rework your 
patch on top on my changes? Should I post the new diff back here for you to 
check before I commit with your name against it?

Original comment by matt.goo...@gmail.com on 28 Nov 2011 at 3:29

[GoogleCodeExporter]

There is no problem if you make patch for my patch - finally, we all works for 
the same project and I always like to see what things I've missed(:

Original comment by kxepal on 28 Nov 2011 at 5:57

[GoogleCodeExporter]

Matt, a ping on this?

Original comment by djc.ochtman on 16 Feb 2012 at 10:31

[GoogleCodeExporter]

I've just pushed some code including r6ab0ed59cf1d to extract a Cache class 
from the middle of the http.Session code. It doesn't actually fix the issue but 
I wanted to do this to make testing easier/possible.

Original comment by matt.goo...@gmail.com on 20 Sep 2012 at 6:32

[GoogleCodeExporter]

It looks like I may have a heavily modified version of kxepal's test that works 
against the new Cache class and fails consistently. I'll check it out more - 
it's a little insane ;) - and see if kxepal's fix works ok.

Original comment by matt.goo...@gmail.com on 20 Sep 2012 at 6:47

[GoogleCodeExporter]

Nice refactoring! However, I have a little scepsis about tests, since now they 
would proof the fact that dict iterators are not threadsafe. It still requires 
to change `.iteritems()` to `.items()` or wrap `_cleanup` method body with lock 
to let it work in multithreaded environment [1]. In perspective, if we'd like 
to add Python3 support, we still need to use lock or alien for <Python-2.7 
`.itemsview()` method since behavior of `.items()` is changed, so lock-variant 
might be more better and optimal.

[1] 
http://code.google.com/p/couchdb-python/source/browse/couchdb/http.py?spec=svn6a
b0ed59cf1d013ed7a2fcaf3eaf32ac8130965a&name=6ab0ed59cf1d&r=6ab0ed59cf1d013ed7a2f
caf3eaf32ac8130965a#431

Original comment by kxepal on 20 Sep 2012 at 7:00

[GoogleCodeExporter]

This issue has been migrated to GitHub. Please continue discussion here:

https://github.com/djc/couchdb-python/issues/208

Original comment by djc.ochtman on 15 Jul 2014 at 7:22