search

AustralianAntarcticDataCentre / raadsync

Other
2 stars 0 forks source link

investigate timestamping on oceancolor data #18

Closed raymondben closed 8 years ago

raymondben commented 8 years ago

Is timestamping wise/needed? Looks like we have some files where the local checksum differs from the server, but the server timestamp is no newer than the local copy and so it doesn't download. Needs investigation.

mdsumner commented 8 years ago

Assuming you've seen this? http://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?pid=12520;hl=timestamping

Just checking, haven't explored myself

raymondben commented 8 years ago

Hm, looks like the issue actually stems from broken checksums at the oceancolor end. e.g. the file T20000322000060.L3m_MO_SST_9.bz2 is reported by their tool to have a checksum of "0" (clearly an error). So our script recognizes that the checksum differs from our copy, and tries to re-download. But because we also have timestamping on, it then aborts the download because the file on the server is no newer than ours. We'll leave timestamping in place.

TL,DR: all good, our system works.