search

ehn-dcc-development / eu-dcc-hcert-spec

Electronic Health Certificates Specification
363 stars 40 forks source link

Compression header #8

Closed jschlyter closed 3 years ago

jschlyter commented 3 years ago

Verifiers MUST check of the presence of a valid ZLIB/Deflate header (0x78, 0xDA) - or proceed without this step when absent.

Do we require best compression, or does any compression level work? My examples has 0x78 0x9c

dirkx commented 3 years ago 
78 01 - No Compression
78 58 - Low compression
78 9C - Default Compression
78 DA - Best Compression

Normal codes. So the question is - do we make level-9/best mandatory - or do we say - should.

The CBOR (i.e. cose) header is 1 byte - Major (3 bits) and Short count (5 bits);

So 78 means 0111. 1000 => 011 = major =3, 1100 n0x18 for the short count. That means a string.

So I think it is fair to conclude that you cannot have a valid COSE (CWT) that starts with 0x78 ?

jschlyter commented 3 years ago

Yes, it makes sense to RECOMMEND best compression but the other should not be disallowed.

jschlyter commented 3 years ago

I'm not sure we want to allow non-compressed data without compression header though - that will most likely confuse validator implementors. Better to require the compression IMHO.

fredriklj commented 3 years ago

Problem would be solved if we could just conclude that we use compression for all cases in which the HCERT is optically encoded. I REALLY think it does not make sense to have this optional, as every reasonable HCERT payload I have ever seen shrinks at least 60-70 bytes using compression, even in the minimal case.

The processing using Zlib is also very cheap, so I fail to see the reason to make it optional.

dirkx commented 3 years ago

@fredriklj So during testing we found 1 or 2 cases where ZLIB made it longer. We can do two things - wait for the metadata spec to be final - then we know for sure if this still can happen. OR simply ignore this - as in those cases - the resuling ZLIB; while a few bytes longer was in the sub 200 bytes range. Above that we did not see it happen at all.

martin-lindstrom commented 3 years ago

The specification states that the recommended compression mechanism is ZLIB. And later the spec says: "If no compression signature bytes are detected, the implementation SHALL assume an uncompressed format.".

I think that the spec should state that compression SHOULD be performed, and if compression is done, ZLIB according to RFC1950/RFC1951 MUST be used.

Otherwise, a consumer may have problems looking for the compression signature bytes (i.e., if something else than ZLIB is used).

jschlyter commented 3 years ago

I would not assume that HCERT issuers understands the reasons for doing compression, hence I would recommend that we (as designers) require compression. Requiring compression will make it easier for all parties and have yet seen a use case where no compression is (except for a few bytes) better.

martin-lindstrom commented 3 years ago

Yes. I agree. But, even if compression is not a MUST, ZLIB for the compression mechanism should be a MUST. It is too vague as it is written now: "The specification states that the recommended compression mechanism is ZLIB". We can't just recommend ZLIB and later on require that ZLIB signature bytes should be checked for.

martin-lindstrom commented 3 years ago

However, for interoperability reasons the most efficient wording would to have a MUST for compression and ZLIB.

jschlyter commented 3 years ago

I propose we simply revert to the old wording:

To lower size and to improve speed and reliability in the reading process of the HCERT, the CWT SHALL be compressed using ZLIB (RFC 1950) and the Deflate compression mechanism in the format defined in (RFC 1951).