Store modified sessions only

[magro]

From martin.grotzke on February 16, 2010 23:41:32

Right now sessions are sent to memcached for each request, even if the session was not modified by this request.

To improve performance sessions should only be sent to memcached if they were modified, that is they differ from the session that is already stored in memcached.

As the session can contain complex objects modifications of such object graphs cannot be detected by the memcached session manager. To detect a modification of the data stored in the session a hash over the serialized session can be used: the session is stored with this hash in memcached (and in the tomcats session map) and when the session shall be sent again to memcached the hash can be computed again and compared with the previous hash. The hash of course must not be included in the serialized byte array but has to be stored separately.

One thing to consider is the expiration in memcached. Possible solutions:

• Increase the expiration significantly
• Use the (requested) touch operation to extend the expiration, unfortunately this is not yet possible - https://code.google.com/p/memcached/issues/detail?id=110
• Send the session following some algorithm even if it was not modified to extend the expiration (e.g. after a certain time period) Still need to figure out what's the best solution for this.

Original issue: http://code.google.com/p/memcached-session-manager/issues/detail?id=36

[magro]

From martin.grotzke on February 17, 2010 10:28:51

After having a closer look I realized, that the session itself will be modified even if it was just accessed: it has the fields lastAccessedTime and thisAccessedTime.

Therefore the hash that's used to determine if the session was modified must not include these attributes.

One solution would be to create the hash only over the session attributes and to serialize the attributes only using the provided transcoder factory - the session internal fields could be serialized separately and stored in separate, dedicated bytes in the byte array sent to memcached.

Another solution would be to serialize the session attributes, create the hash over the serialized session attributes and if it's detected that the session was modified then the session would be serialized again, but this time completely and this would be sent to memcached. Serialization would be done more often then, the question therefore would be, how big the impact of the serialization is regarding performance, in relation to the time spent with communication with memcached.

[magro]

From martin.grotzke on February 20, 2010 13:58:34

Just as a reminder for myself regarding the expiration in memcached: another possibility would be to store the lastBackup timestamp as part of the session and have a dedicated thread checking sessions that need to be updated in memcached. This would allow to send sessions only to memcached if they are either modified or are reaching expiration in memcached.

[magro]

From martin.grotzke on February 22, 2010 16:01:26

A first draft is implemented:

• Session internal fields and session attributes are serialized separately
• The serializer strategy (transcoder) now is only responsible for serializing session attributes
• The session manager tracks modifications of session attributes with a hash over the serialized attributes, both when sending data to memcached and when loading a session from memcached.

This is implemented for java serialization and xml/javolution based serialization.

Still missing:

• XStream implementation (easy)
• Expiration update in memcached
• More tests
• Documentation/Javadoc

Status: Started

[magro]

From martin.grotzke on February 24, 2010 07:17:52

Just implemented: Before the session attribute serialization and hash calculation is performed, it's checked if the session was accessed at all. If this is not the case we don't even have to serialize session attributes and calculate the hash but save this power for a greener world :-)

[magro]

From martin.grotzke on February 27, 2010 16:51:12

As a reminder: session.principal also needs to be (de)serialized.

[magro]

From martin.grotzke on February 27, 2010 16:58:04

Need to check if the session.notes also need to be serialized to support container managed auth (see http://old.nabble.com/Is-it-possible-to-replace-the-HttpSession- implementation-in-Tomcat--td27596813.html).

[magro]

From martin.grotzke on March 01, 2010 01:21:59

Implemented expiration update in memcached: The expiration time in memcached is set to the time that the session will be valid in tomcat. The expiration update is being done by a new session backup, once the memcached touch operation is available (https://code.google.com/p/memcached/issues/detail?id=110), this might also be used. The tomcat internal thread for background processes is used for periodically updating the expiration time (by overwriting Manager.backgroundProcess, by default this is invoked every 10 seconds). Expiration update will be performed, if the session would timeout in <= 20 seconds (2 * tomcats backgroundProcessorDelay). Details to this commit: http://github.com/magro/memcached-session-manager/commit/12285f935768042c0a112fee2d093048ec887863

[magro]

From martin.grotzke on March 03, 2010 15:23:32

Just added support for container managed authentication (session.authType and session.principal), so that authorized users/sessions can be used by other tomcat instances without the need to login again.

What's not supported is serialization of the SavedRequest for form-based login (when the unauthenticated user tries to access a protected resource and is presented a login form): When the tomcat that presented the login form (and saved the original request) dies, this information (SavedRequest) is not serialized and therefore cannot be picked up by another tomcat. This is the same for tomcats own session replication which does not support this: https://issues.apache.org/bugzilla/show_bug.cgi?id=40531 However, it should be possible to achieve this, even if the SavedRequest (and associated Cookies) are not serializable. For non-sticky applications using container-managed, form-based authentication this would be required.

[magro]

From martin.grotzke on March 06, 2010 14:06:59

• Implemented versioning: the first 2 byte (short) are reserved for the version of the serialization format, starting with version 1. Handling of different versions has to be implemented as soon as the format changes the next time.
• Implemented upgrade of sessions that were stored using the previous serialization format (full session was serialized by the serialization strategy). Upgrade implemented for java serialization, javolution and xstream. The TranscoderFactory interface had to be changed for this.
• Upgraded httpclient from 3.1 to 4.1-alpha1 (this was required due to I don't remember exactly what it was, 4.1-alpha1 is used as 4.0.1 didn't do preemptive basic auth correctly)

[magro]

From martin.grotzke on March 08, 2010 08:03:53

Added missing javadoc, some final cleanup, and just merged this into the develop branch. Will be released with 1.2

Status: Fixed

[magro]

From martin.grotzke on March 20, 2010 13:53:33

Just added a check if session attributes were accessed via getAttribute/setAttribute before testing if attributes were modified via serialization/hash calculation