XAdES : Content Timestamp validation fails

[bsanchezb]

Probably related to https://github.com/lovele0107/signatures-conformance-checker/issues/1

The archive contains two signatures with the same AllDataObjectsTimeStamp, signing the same data. However, the ETSI Checker computes different message-imprint digest for these two files (both times invalid) on the same content timestamp.

Could you please investigate the case?

signatures.zip

[jccruellas]

Dear BSanchezB, I have been taken a look to your signatures, and I would say that I have identified the source of the problems. Your signatures have two ds:Reference elements. The first one contains a URI attribute which is AN EMPTY URI. Now look to what the XML Signature W3C Recommendation in its clause 4.4.3.3 says about the usage of empty URIs as a way for referencing what is being signed:

4.4.3.3 Same-Document URI-References Dereferencing a same-document reference must result in an XPath node-set suitable for use by Canonical XML [XML-C14N]. Specifically, dereferencing a null URI (URI="") must result in an XPath node-set that includes every non-comment node of the XML document containing the URI attribute.

Please note that what is saying is that if you include an empty URI then the retrieval process will result in the whole document that contains the URI attribute. In your case this implies that in validation, regardless of how you have computed the message imprint of your AllDataObjectsTimeStamp, the relying party will follow the requirements of XML Signature 1.1 (in fact the XAdESCC is using the xmlsec from Apache, which is the one that retrieves the data objects pointed by the ds:Reference elements), and in this case IT WILL RETRIEVE THE WHOLE XML DOCUMENT.

I would say that this is the reason for the error raised by the XAdESCC. It is obvious that you, in origin, have not computed the same message imprint as you have not created yet the AllDataObjectsTimeStamp. However, as your signature includes the empty URI, your signature is instructing the relying party to take the whole document (AllDataObjectsTimeStamp included) as the input for the message imprint computation.

I would say that this is not an error in the XAdESCC but a bad generation in the XAdES signatures. You should use a same-document URI based in fragments, for instance in your firs ds:Reference.

Having said that, what it seems to me an error in the XAdESCC behaviour is that it does not show the message imprint computations contributions for AllDataObjectsTimeStamp If you look at the page "XML Raw Report" you will see this contribution, but somehow the transform that should present this data in the other page does not do it..this is an error that I will try to fix.

Could you please let me know your opinion on what I have said?

Hope this helps

Best regards Juan Carlos

[bsanchezb]

Dear Juan Carlos,

I agree that a dereferencing of a reference with an empty URI ("") shall result to the whole document. This is the expected behavior. However, based on ETSI EN 319 132-1 and XMLDSIG I expect another result of the complete reference processing (please, do not confuse with "dereferencing").

ETSI EN 319 132-1, ch. "5.2.8.1 The AllDataObjectsTimeStamp qualifying property" says:

The input to the computation of the message imprint shall be the result of processing all the ds:Reference elements within the ds:SignedInfo except the one referencing the SignedProperties element, in their order of appearance, as follows: 1) process the retrieved ds:Reference element according to the reference-processing model of XMLDSIG [1] clause 4.4.3.2; 2) if the result is a XML node set, canonicalize it as specified in clause 4.5; and 3) concatenate the resulting octets to those resulting from previously processed ds:Reference elements in ds:SignedInfo.

Based on XMLDSIG 4.4.3.2 I assume the following algorithm shall be executed for a reference processing: 1) Dereference the reference (here we do it by URI, which is empty in our case); 2) Apply transforms in the order of occurrence (here we have only one, the XPath enveloped transform).

Therefore, I expect to get the whole document excluding all elements as the reference processing result (step 1) of the ETSI algorithm), the result after dereferencing and transforms processing.

Moreover, the same algorithm (XMLDSIG 4.4.3.2) is used for a validation of element itself, what does not fail in a validation of XADESCC. By my opinion, there is a conflict between processing of the same reference for different needs.

I hope this clarifies my point.

Best regards, Aleksandr.

[jccruellas]

Dear BSanchezB, As a continuation of my former comment, I have managed to achieve that my local version now reports both the input to the message imprint computation and the details of the content of the AllDataObjectsTimeStamp.

I attach here a zip file containing the folders tree that contain the result of the checking of your firstSignature. Please note that the input to the message imprint computation is the whole document, as shown in the corresponding page.

Juan Carlos. firstSignature.zip

[jccruellas]

Hi again Alex. I have re-tested the two signatures: the CC does not raise any error now....I have noticed that the ds:Reference with the empty URI had a transformation which I had overlooked when did my former comments. Sorry for that. Indeed this transform makes that the retrieved bytes are not the whole document....the change in the xml sec apache method from Reference class worked. Thanks a lot. If you agree, I will close the issue.

Regards Juan Carlos.

[bsanchezb]

Dear Juan Carlos,

Thank you again for your time. It will help a lot in our algorithm improvement as well.

I also wish to take this opportunity to ask you one more question.

As I previously pointed to the algorithm defined in "5.2.8.1 The AllDataObjectsTimeStamp qualifying property", the steps 2) says:

2. if the result is a XML node set, canonicalize it as specified in clause 4.5; and

I just want to clarify, what is "the result is a XML node set"? Is it based on outputs defined in XMLDSIG "6.6 Transform Algorithms"?

If yes, then I assume when the last transform in a ds:Reference transforms list is a canonicalization algorithm, then the step two shall be skipped (the canonicalization for a ds:Reference can differ from one defined for a timestamp)? Because XMLDSIG specifies an output of all canonicalizations as an octet-stream.

How do you validate if the output is a node-set?

Best regards, Aleksandr.

P.S. Besides the question, the issue can be closed.

[bsanchezb]

Hello @jccruellas ,

Any update on my last question?

In particular I want to know what is "XML node set" in the context of a message-imprint computation algorithm defined 5.2.8.1?

Regards, Aleksandr.

[jccruellas]

Hi Aleksandr,

Sorry I did not react before. Will take a close look beginning next week and let you know.

[jccruellas]

Good afternoon Alexandr,

I have reacted to your question in another issue this morning, so I copy the answer here. Apologies.

I have been re-reading both clause 4.4.3.2 of XML Signature 1.1 and clause 5.5.2.2 of ETSI EN 319 132-1 and I would say that we have an issue in there. The thing would go like this:

Clause 4.4.3.2 of XML Signature 1.1 (The Reference Processing Model) specifies how to retrieve the data object referenced by the URI of the reference, how to process the different transforms, and HOW TO GET THE BYTES TO BE DIGESTED. This last thing is important: this clause actually says that the processing always ends with an octet string, either because the last transform in the chain is a canonicalization or because if this is not the case and the result of the last transform is a node-set, then by default the application must canonicalize it.

Now, what happens is that ETSI TS 319 132-1 is not completely right in its wording in clause 5.5.2.2. Look the text, when it is specified how to incorporate the contribution of the ds:Reference:

Take all the ds:Reference elements in their order of appearance within ds:SignedInfo referencing whatever the signer wants to sign including the SignedProperties element. Process each one as indicated below:

Process the retrieved ds:Reference element according to the reference processing model of XMLDSIG [1], clause 4.4.3.2. If the result is a XML node set, canonicalize it as specified in clause 4.5. And Concatenate the resulting octets to the final octet stream. The problem is in the first bullet: "Process the retrieved ds:Reference element according to the reference processing model of XMLDSIG [1], clause 4.4.3.2.". If one interprets "Process" as the complete processing, i.e. the one whose final result is an octet stream because it incorporates a canonicalization (either explicit in the chain of transforms or implicit, as indicated in XML Signature), then the two next bullets do not make sense!.... However so far, my reading was that "process" was restricted to two steps: 1) Retrieve the data referenced by the URI and 2) Process the. chain of transforms. If this is the "process", then the two last bullets replace the last step in clause 4.4.3.2 of XMLSig, and then if the final transformation is an explicit canonicalization, the canonicalization in the ArchiveTimeStamp does not play any role, as the result of that canonicalization is an octet stream, but if the result of the last transform is a node-set (and this can be seen from the transformation itself -at least the registered transforms in XML Sig), then this node-set is canonicalized using the algorithm indicated in ArchiveTimeStamp instead of the regular canonical algorithm.

After all these disgressions, I propose the following:

Agree in that the right reading of this clause is the last one, ie, that "processing" refers only to the retrieval of the data object referenced by the URI and the computation of all the transforms.

Propose to ESI to ammend clause 5.5.2.2 of ETSI 319 132-1 (we are in a phase of reviewing of this series of ENs) so that the wording makes it clear that only retrieval of data object referenced by the URI and the computation of the transforms constitute the "processing" mentioned in the first bullet.

Rework the AdESCC to align to this interpretation. This would require to take your signatures and see how the AdESCC should behave.

Hope this helps.

Juan Carlos.

[bsanchezb]

Dear Juan Carlos,

I have prepared some test cases for the issue we have discussed within the tickets #1 and #2 regarding to the timestamp message-imprint computation (canonicalizing or not a reference processing output). Please, confirm if XAdES CC is able to validate the signatures (with the applied fix how it was discussed earlier within the tickets), and if not, please say what file fails and we will try to find out the issue.

Best regards, Aleksandr Beliakov.

xades-test-cases.zip

[jccruellas]

Thanks for this Aleksandr,

I am busy with other stuff the rest of the week. Hope to be able to come back with a report next week Regards Juan Carlos.