Store modified sessions only

[GoogleCodeExporter]

Right now sessions are sent to memcached for each request, even if the 
session was not modified by this request.

To improve performance sessions should only be sent to memcached if they 
were modified, that is they differ from the session that is already stored 
in memcached.

As the session can contain complex objects modifications of such object 
graphs cannot be detected by the memcached session manager.
To detect a modification of the data stored in the session a hash over the 
serialized session can be used: the session is stored with this hash in 
memcached (and in the tomcats session map) and when the session shall be 
sent again to memcached the hash can be computed again and compared with the 
previous hash. The hash of course must not be included in the serialized 
byte array but has to be stored separately.

One thing to consider is the expiration in memcached. Possible solutions:
- Increase the expiration significantly
- Use the (requested) touch operation to extend the expiration, 
unfortunately this is not yet possible - 
http://code.google.com/p/memcached/issues/detail?id=110
- Send the session following some algorithm even if it was not modified to 
extend the expiration (e.g. after a certain time period)
Still need to figure out what's the best solution for this.

Original issue reported on code.google.com by martin.grotzke on 16 Feb 2010 at 10:41

[GoogleCodeExporter]

After having a closer look I realized, that the session itself will be modified 
even 
if it was just accessed: it has the fields lastAccessedTime and 
thisAccessedTime.

Therefore the hash that's used to determine if the session was modified must 
not 
include these attributes.

One solution would be to create the hash only over the session attributes and 
to 
serialize the attributes only using the provided transcoder factory - the 
session 
internal fields could be serialized separately and stored in separate, 
dedicated bytes 
in the byte array sent to memcached.

Another solution would be to serialize the session attributes, create the hash 
over 
the serialized session attributes and if it's detected that the session was 
modified 
then the session would be serialized again, but this time completely and this 
would be 
sent to memcached. Serialization would be done more often then, the question 
therefore 
would be, how big the impact of the serialization is regarding performance, in 
relation to the time spent with communication with memcached.

Original comment by martin.grotzke on 17 Feb 2010 at 6:28

[GoogleCodeExporter]

Just as a reminder for myself regarding the expiration in memcached: another 
possibility would be to store the lastBackup timestamp as part of the session 
and have 
a dedicated thread checking sessions that need to be updated in memcached. This 
would 
allow to send sessions only to memcached if they are either modified or are 
reaching 
expiration in memcached.

Original comment by martin.grotzke on 20 Feb 2010 at 9:58

[GoogleCodeExporter]

A first draft is implemented:
- Session internal fields and session attributes are serialized separately
- The serializer strategy (transcoder) now is only responsible for serializing 
session attributes
- The session manager tracks modifications of session attributes with a hash 
over the serialized 
attributes, both when sending data to memcached and when loading a session from 
memcached.

This is implemented for java serialization and xml/javolution based 
serialization.

Still missing:
- XStream implementation (easy)
- Expiration update in memcached
- More tests
- Documentation/Javadoc

Original comment by martin.grotzke on 23 Feb 2010 at 12:01

• Changed state: Started

[GoogleCodeExporter]

Just implemented: Before the session attribute serialization and hash 
calculation is 
performed, it's checked if the session was accessed at all. If this is not the 
case we 
don't even have to serialize session attributes and calculate the hash but save 
this 
power for a greener world :-)

Original comment by martin.grotzke on 24 Feb 2010 at 3:17

[GoogleCodeExporter]

As a reminder: session.principal also needs to be (de)serialized.

Original comment by martin.grotzke on 28 Feb 2010 at 12:51

[GoogleCodeExporter]

Need to check if the session.notes also need to be serialized to support 
container 
managed auth (see 
http://old.nabble.com/Is-it-possible-to-replace-the-HttpSession-
implementation-in-Tomcat--td27596813.html).

Original comment by martin.grotzke on 28 Feb 2010 at 12:58

[GoogleCodeExporter]

Implemented expiration update in memcached: The expiration time in memcached is 
set to the time that the session 
will be valid in tomcat. The expiration update is being done by a new session 
backup, once the memcached touch 
operation is available 
(http://code.google.com/p/memcached/issues/detail?id=110), this might also be 
used.  
The tomcat internal thread for background processes is used for periodically 
updating the expiration time (by 
overwriting Manager.backgroundProcess, by default this is invoked every 10 
seconds). Expiration update will be 
performed, if the session would timeout in <= 20 seconds (2 * tomcats 
backgroundProcessorDelay).

Details to this commit:
http://github.com/magro/memcached-session-manager/commit/12285f935768042c0a112fe
e2d093048ec887863

Original comment by martin.grotzke on 1 Mar 2010 at 9:21

[GoogleCodeExporter]

Just added support for container managed authentication (session.authType and 
session.principal), so that authorized users/sessions can be used by other 
tomcat 
instances without the need to login again.

What's not supported is serialization of the SavedRequest for form-based login 
(when 
the unauthenticated user tries to access a protected resource and is presented 
a 
login form):
When the tomcat that presented the login form (and saved the original request) 
dies, 
this information (SavedRequest) is not serialized and therefore cannot be 
picked up 
by another tomcat.
This is the same for tomcats own session replication which does not support 
this: 
https://issues.apache.org/bugzilla/show_bug.cgi?id=40531

However, it should be possible to achieve this, even if the SavedRequest (and 
associated Cookies) are not serializable. For non-sticky applications using 
container-managed, form-based authentication this would be required.

Original comment by martin.grotzke on 3 Mar 2010 at 11:23

[GoogleCodeExporter]

- Implemented versioning: the first 2 byte (short) are reserved for the version 
of 
the serialization format, starting with version 1. Handling of different 
versions has 
to be implemented as soon as the format changes the next time.

- Implemented upgrade of sessions that were stored using the previous 
serialization 
format (full session was serialized by the serialization strategy). Upgrade 
implemented for java serialization, javolution and xstream. The 
TranscoderFactory 
interface had to be changed for this.

- Upgraded httpclient from 3.1 to 4.1-alpha1 (this was required due to I don't 
remember exactly what it was, 4.1-alpha1 is used as 4.0.1 didn't do preemptive 
basic 
auth correctly)

Original comment by martin.grotzke on 6 Mar 2010 at 10:06

[GoogleCodeExporter]

Added missing javadoc, some final cleanup, and just merged this into the 
develop 
branch. Will be released with 1.2

Original comment by martin.grotzke on 8 Mar 2010 at 4:03

• Changed state: Fixed

[GoogleCodeExporter]

Just added a check if session attributes were accessed via 
getAttribute/setAttribute 
before testing if attributes were modified via serialization/hash calculation

Original comment by martin.grotzke on 20 Mar 2010 at 8:53