Fix Voldemort

[GoogleCodeExporter]

If a key-value pair is saved to the database, it currently cannot be read
from other nodes. One example using the dbtest app saves the data to box1.
Here, it can be read from box1 but not box2 (saved key is 'a67', val is 'bar'):

root@appscale-image0:~/tests# curl http://box2/get -d 'key=a67'
key didn't exist

root@appscale-image0:~/tests# curl http://box1/get -d 'key=a67'
bar

Suspecting that the individual databases are forming Voldemort groups by
themselves (that is, 4 distinct databases instead of 1 shared database).

Original issue reported on code.google.com by shattere...@gmail.com on 22 Jan 2010 at 5:05

[GoogleCodeExporter]

Currently unable to replicate this problem. Will keep an eye out for it 
throughout
this weekend's testing however, and if cannot repro it by then, will close.

Original comment by shattere...@gmail.com on 30 Jan 2010 at 8:17

• Changed state: Started

[GoogleCodeExporter]

Can consistently reproduce now. Works fine in 1 and 2 node deployments but not 
in 4
node deployments. Will run numbers for these deployments first and then compare
configuration files generated to try to find problems.

Original comment by shattere...@gmail.com on 30 Jan 2010 at 9:01

[GoogleCodeExporter]

Is not a problem with Voldemort itself but a problem with our interface to it.
Accessing the Voldemort shell reveals that all nodes have the data in question, 
but
since App Engine can't see the value, there must be a problem in the 
py_voldemort
interface. Will examine further.

Original comment by shattere...@gmail.com on 16 Feb 2010 at 4:20

[GoogleCodeExporter]

Fixed. If a persistent database connection is used, then under high traffic the
connection can get corrupted and cause the library to not see new updates. 
Changed to
not use a persistent connection.

Upgrading to new version of Voldemort may allow us to use a persistent 
connection.

Original comment by shattere...@gmail.com on 17 Feb 2010 at 1:32

• Changed state: Fixed

[GoogleCodeExporter]

Re-opened, as we're now experiencing this problem again with the latest code 
changes.
Trying to not use connection pooling to see if it fixes the problem.

Original comment by shattere...@gmail.com on 7 May 2010 at 11:51

• Added labels: Milestone-Release1.4

[GoogleCodeExporter]

Original comment by shattere...@gmail.com on 7 May 2010 at 11:51

• Changed state: Started

[GoogleCodeExporter]

Original comment by shattere...@gmail.com on 22 Sep 2010 at 4:16

• Changed state: Fixed