jgallagher59701
commented
2 years ago As a slight change to #3, the client is lenient in that it can accept checksums, but just ignores them. Since the checksum value appears in the DMR, I think that's pretty straightforward. Then, the compatibility issues with old servers that always include checksums goes away and there's no overhead for clients that don't care about the checksums (not cost associated with the CRC computation).
NB: The notion for checksums is not data integrity for the transmission. It's for determining if the data values changed over time (which they do, in some cases, as datasets are reprocessed).
ndp-opendap
The spec allows optional checksumming of variable data but as far as I can tell, we never provided a way for a client to tell a server if checksumming should be used on a request.
So I propose the following change to Volume 2, section 5.1 (Query String Parameters) to add the following new query parameter "dap4.checksum".
The possible cases are defined as follows:
Nathan noted:
I assume that there is no hope for fixing these servers. So I can see three ways of handling this:
A short term solution is to make the default be "dap4.checksum=true" when no dap4.checksum query is present (case 5 above). This will work until such time as clients start including specific dap4.checksum query parameters. At that point, accessing such a server will fail if the user specifies checksums are off.
We can identify the relevant servers and stick a hack in our libraries to force "dap4.checksum=true" when requests are sent to those servers. The DAP2 code we have already does things like this for non-compliant servers like Columbia. Its ugly, but it works. [Nathan, can you create a list of such servers?]
The client always computes the checksum as it reads the variable's data off the socket stream. Then it needs to see if there is an extra four bytes of data at the end of the variable's data. I do not know if this is unambiguously possible. And of course, it is a performance hit.