Treating memcache data

[blreams]

What to do when a database put invalidates memcache data?

1. Invalidate the corresponding memcache entry(s) by popping or deleting the associated key?
2. Update memcache with new data?
3. Flush the entire memcache?

Obviously, 3. is ridiculous.

[jbholden]

I think option 2 is the best one. This would prevent an end user from having to wait on a memcache miss. I was intending for the code/update.py class to contain the functions necessary to update the memcache.

For example, after the game scores are updated for a week then a call to update_week_results with the week and year number would update the memcache.

There are also in functions in code/database.py that can update the memcache.

A couple of things I have been thinking about

• is there a limit to the size of the memcache? I have been using it pretty liberally so far and I didn't know if there was an upper limit.
• I'm not sure if my code will currently handle a week with games and no picks yet. I need to consider this.
• Some of the tests I wrote flush the entire memcache and try and load the memcache with all available data. They take quite a while to run. I don't know if we should consider running a subset of these tests when submitting new code (running all the tests might take too long)

[blreams]

I agree, we'll go with option 2. I put some info on the wiki concerning current memcache usage, though it is likely to get out of date quickly.

I did a little reading about the app engine implementation of memcache. Looks like the following limits apply:

1. Individual cache entries may not exceed 1 MB.
2. Cache keys should not exceed 250 bytes, but if they do, they are automatically hashed.
3. There was something about a 32 MB data limit...wasn't sure if that was the cache size (I don't think we will be anywhere close to that).

Also, when I was writing code to update memcache when a pick sheet is created, I didn't spend too much time thinking about race conditions...but memcache has a compare and set algorithm to address that. Need to look into that some more.