Session could be improved for more performance

joerussbowman commented 15 years ago

There's a lot of serialization that happens with session. Except for Models, anything going in as a session data item is pickled on write. So reads are unpickled, which in appengine can be a little heavy.

A few ideas are

Catch any items that can be stored as string/unicode, and store them as that in a TextProperty, rather than the BlobProperty. I believe in most cases, this plus the change mentioned above for Models should cover most uses of session. This means pickle, in most applications, may not be used at all.
To completely optmize, session can include a WSGI Middleware which would handle all the serialization at the end of the request. Using the internal per request cache (cache attribute on the Session object) to handle the data until the request is over and everything is written to the datastore. May even be able to speed up the overall write speed of the datastore by doing one db.put for all items.
Drop using the db.key() as the unique identifier and switch to generating uuids for this purpose. Reason being that db.key() has proven to be an inconsistent solution and fails occasionally. At this point I don't believe it's something can be depended on for application level unique identifiers, and Google has proven to be unresponsive (months) in addressing the issue I posted and others have contributed on.

Note: I'm not accepting this at this time, though this is fair game for anyone to take on AFTER the 1.3 release.

joerussbowman commented 14 years ago

I've thought about this a little, and think maybe there's an alternative to the WSGI approach. That's to make session a decorator. That would allow you to insert the session object to any function, and handle saving it at the end. Also seems to be the most pythonic approach.

Still have no idea when I'll get to this unfortunately. I might make a side project out of it. This however would be a pretty big change and will impact exist code. This would be a version 2 release for gaeutilities most likely.

joerussbowman commented 14 years ago

Guido would agree I think - http://groups.google.com/group/google-appengine-python/browse_thread/thread/f2794adaac5723d9/3ba1f13156642f39?lnk=gst&q=middleware#3ba1f13156642f39

lol

joerussbowman / gaeutilities

Session could be improved for more performance #2