SDB item.save() never times out

[GoogleCodeExporter]

What steps will reproduce the problem?
1.Set up an SDB connection and create/get a domain. In a loop, do something 
like:
    print 'Saving %s' % (item_num)
    item = domain.new_item(item_num)
    item['v1'] = 'foo'
    item.save()
    item_num += 1

2. Wait.

What is the expected output? What do you see instead?

Eventually, one of the item.save() calls will block forever (I've let it sit 
for several hours). It should time out after some reasonable amount of time.

The stack trace when it's blocked is:

  File "./boto-sdb-test.py", line 72, in <module>
    item.save()
  File "/Library/Python/2.6/site-packages/boto-2.0b2-py2.6.egg/boto/sdb/item.py", line 90, in save
    self.domain.put_attributes(self.name, self, replace)
  File "/Library/Python/2.6/site-packages/boto-2.0b2-py2.6.egg/boto/sdb/domain.py", line 89, in put_attributes
    replace, expected_values)
  File "/Library/Python/2.6/site-packages/boto-2.0b2-py2.6.egg/boto/sdb/connection.py", line 290, in put_attributes
    return self.get_status('PutAttributes', params)
  File "/Library/Python/2.6/site-packages/boto-2.0b2-py2.6.egg/boto/connection.py", line 654, in get_status
    body = response.read()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 517, in read
    return self._read_chunked(amt)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 553, in _read_chunked
    line = self.fp.readline()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py", line 395, in readline
    data = recv(1)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ssl.py", line 96, in <lambda>
    self.recv = lambda buflen=1024, flags=0: SSLSocket.recv(self, buflen, flags)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ssl.py", line 217, in recv
    return self.read(buflen)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ssl.py", line 136, in read
    return self._sslobj.read(len)

What version of the product are you using? On what operating system? Boto 
2.0b2, Python 2.6.1, Mac OS X 10.6.4.

Please provide any additional information below.

It seems highly variable how long it takes for this to happen, but I've been 
able to reproduce it every time I've tried. Once it happened after a couple 
thousand calls, most recently it happened after about 46k calls. It could be 
that SDB is throttling me somehow; I'm more concerned with boto's handling of 
the error condition than that it happens in the first place.

Original issue reported on code.google.com by sch9...@gmail.com on 22 Aug 2010 at 5:31

[GoogleCodeExporter]

I've never seen this behavior before and I don't have an explanation yet.  
There are no explicit timeouts managed in boto but the underlying libraries 
like httplib and socket should be handling that layer.  From your example code, 
it looks like this is single-threaded.  Is that correct?

Original comment by Mitch.Ga...@gmail.com on 27 Aug 2010 at 10:41

• Changed state: Accepted
• Added labels: Component-SDB, Type-Defect

[GoogleCodeExporter]

Yes, I was surprised it didn't timeout in httplib or socket... My code is 
single threaded.

Original comment by sch9...@gmail.com on 27 Aug 2010 at 3:12

[GoogleCodeExporter]

it's curious to see that this is happening outside of the _mexe() code, so that 
should mean that the request succeeded, but it's getting hung up reading the 
response back. For that to happen, something very strange must be going on. Can 
you turn on the boto debugging and post the output that you see right when the 
process is hung?

Have you waited > 2 minutes and it still doesn't give up? At some point the TCP 
connection itself should time-out, so I can't see how it's hanging unless SDB 
is sending back a keep-alive signal.

Original comment by koper...@gmail.com on 27 Aug 2010 at 7:55

[GoogleCodeExporter]

I've waited hours and it never gives up. I'll try to repro tonight and also 
capture a tcpdump so we can see if packets are still going back and forth.

Original comment by sch9...@gmail.com on 27 Aug 2010 at 8:25

[GoogleCodeExporter]

Sorry if I'm being dense, but I'm not seeing how to turn on debugging for SDB. 
conn.debug to a value higher than 1, I get output that shows the HTTP stuff, 
but nothing internal to boto (and if I set it to 1, I get no debug output)...

In that case, the last thing I see before it hangs is:

reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: text/xml
header: Transfer-Encoding: chunked
header: Date: Sat, 28 Aug 2010 01:35:11 GMT
header: Server: Amazon SimpleDB

Original comment by sch9...@gmail.com on 28 Aug 2010 at 1:43

[GoogleCodeExporter]

I wonder if the occasional hang is dependent on platform.  I found this issue 
via a Google search because a similar issue is happening to me; in both my case 
and the original author's case, our platform is Mac (his was 10.6.4, I'm 
10.6.8).  I'm running his proposed code on a Linux box right now, doing an 
item.save() every .1 seconds; I've gone through about 91,000 saves so far (I 
started earlier today) with no hangs.  So:

Mac:  Intermittent hangs
Linux: No hangs so far.

Original comment by royrapop...@gmail.com on 17 Jul 2011 at 9:36