Yelp / doloop

Task loop for keeping things updated
Other
10 stars 10 forks source link

doloop.stats() clock skew #19

Closed coyotemarin closed 12 years ago

coyotemarin commented 12 years ago

doloop.stats() runs queries in READ_UNCOMMITTED mode. This means we can see changes that happened to the relevant table after the query started running.

UNIX_TIMESTAMP() in the doloop.stats() queries is set to when they started running, but the UNIX_TIMESTAMP() for any queries that took effect while a doloop.stats() query is running may be sometime later.

This introduces a form of clock skew that can give us negative min_update_time, max_lock_time slightly larger than the lock_for used by doloop.get(), and other silliness.

The fix is to run a final query that just selects UNIX_TIMESTAMP(). All the other queries should just select times rather than offsets from UNIX_TIMESTAMP(), and doloop.stats() should handle the subtraction itself.

This would introduce an even more subtle form of clock skew, but I can't think of a case where it matters; doloop doesn't have any requirements that something must happen by a certain time.