SDB BoxUsage returned by get_usage misses most operations

[GoogleCodeExporter]

What steps will reproduce the problem?
run the attached program.

What is the expected output? What do you see instead?
I see:

$ python sdb_usage_error.py 
boto version: 2.0b2
get_usage after connect returns 0.0
get_usage after create_domain returns 0.0
get_usage after put_attributes returns 0.00439818
get_usage after select returns 0.00439818

and I expect to see that the value returned by the get_usage() method
would be nonzero and incremented after each operation.

What version of the product are you using? On what operating system?
boto version 2.0b2 on Ubuntu 8.04 with Python 2.5.2.

Please provide any additional information below.
The program prints the value returned by get_usage() after each of
several SDB operations; only the put_attributes operation causes the
value to change.

Original issue reported on code.google.com by s...@acm.org on 14 Dec 2010 at 10:42

Attachments:

• sdb_usage_error.py

[GoogleCodeExporter]

The problem is that boxusage accumulation is being done on the connection 
object (by the sax parser), but for most operations, it's the domain rather 
than the connection that is being passed to the parser. So it tries to 
accumulate on the domain, and throws a passed exception. 

The simplest fix is to add a box_usage=0.0 to the sdb.domain.Domain 
constructor, and give it a print_usage() method like 
sdb.connection.Connection's. I have modifications to fix accumulation on the 
connection as well, but am unsure they are fit for posting because they involve 
the ResultSet, and might affect more than just sdb.

Original comment by welch.qu...@gmail.com on 14 Feb 2011 at 7:53

[GoogleCodeExporter]

Thanks for your workaround suggestion which did the trick for me.

Original comment by s...@acm.org on 23 Feb 2011 at 4:07

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I've been looking at this same issue, and a few things come to mind.

in boto/connection.py the get_list function has a parent param.  What is this 
used for?
The only use I see of it is passing it into the XmlHnadler constructor.
THe only thing that XmlHandler does with parent is store it in self.connection 
(implying this should be a connection instance) and then pass it to every 
startElement and endElement function.
for the most part startElement and endElement never ever use it.  The only 
exception is to increment connection.box_usage.
(I may have missed another use of it, but I did search and looked at a whole 
lot of start and endElements)

boto.sdb.connection.select(...)
   boto.connection.get_list(..., parent=domain)
     handler.xmlHandler(rs, connection=parent)
         ResultSet.endElement(..., connection)
             ...
             connection.box_usage += float(value) #connection is actually a domain raise exception

My naive conclusion is that we could either always pass self int the 
constructor for XmlHandler to ensure that connection is in fact always an 
instance of connection, which would allow box_usage to increment properly.
boto.sdb.connection.select(...)
   boto.connection.get_list(..., parent=domain)
     handler.xmlHandler(rs, connection=self)  #this is called from connection, so self is a connection
         ResultSet.endElement(..., connection)
             ...
             connection.box_usage += float(value)  #connection is now an instance of 'connection'

Or
removed 'parent' param from get_list, remove 'connection' param from 
XmlHandeler constructor, remove 'connection' param from  
startElement/endElement and ResultSet could gain a box_usage member variable.  
get_list could then use rs.box_usage to increment self.box_usage.  It's also 
useful to be able to get the box_usage for a single call from ResultSet.  Maybe 
it would be a good idea to extend ResultSet to SDBResultSet to add box_usage 
since it's used outside of SDB. 

boto.sdb.connection.select(...)
   boto.connection.get_list(...)
     handler.xmlHandler(rs)
         ResultSet.endElement(...)
             ...
             self.box_usage = float(value)  #store box_usage in the result set  
     self.box_usage += rs.box_usage         #increment connections total box usage

Original comment by bencorn...@gmail.com on 8 Sep 2011 at 3:29