Closed daveschaefer closed 10 years ago
After some discussion on the list: if we don't update the previous cert at all we get cases where sites that change their certificates frequently (e.g., every observation) end up with a number of observations with one-second lengths. To avoid this but still prevent data corruption we will start by putting a limit on how far data can be backfilled.
If the previous observation ended within 48 hours, it will be updated. Otherwise it is left untouched and a new observation record is created. This allows for normal behaviour in the cases where site scans are not performed exactly every 24 hours (e.g. the scan starts at the same time but sites are scanned in a random order).
Fixed in 3.2
Note that this bug combined with #25 has left the data for the default notaries in a bad state. The default notaries are running a very old version of the server software (v2.0). Because their logs would fill the entire hard disk, they would be out of commission for long periods of time, and then backfill large swaths of observation data when they came back online.
I fortunate to be able to chat with both @danwent and David Andersen, who both agreed that if we can get the default notaries back online it would be best to simply start over from scratch with fresh data. I can upload a copy of the current v2 notary data for historical purposed in case anyone wants to take a look.
Currently if a notary scans a site and finds a different certificate than it saw last time, it does the following:
We should not perform the second step - it invents data that we can't vouch for (e.g. the cert could have changed 1 second after we last observed it). It also opens the opportunity for a bad bug: if you don't run your scans regularly, or run scans after a very long absence, the notary would backfill a large time range, claiming the certificate had been seen throughout that time period.