EU validation tools and invalid xades signature format

[sangar82]

HI!

However it is likely users will not have an encyclopaedic knowledge of XAdES. So the other aspect is to allow less expert users provide a minimum amount of information and ...

I'm a user of this type ;) . We are learning about certifications but is a new world for us! It's hard! Thanks for your package, makes life easier for users like us.

I've found a great validator for ades signatures from EU https://ec.europa.eu/cefdigital/DSS/webapp-demo/validation

If I upload your test xml http://www.xbrlquery.com/xades/hashes%20for%20nba%20with%20signature.xml the validator says that the signature format is: XADES-BASELINE-B

But if I upload my xml signed (with deferred option= false) with my certificates, it says that the signature hasn't a xades format (XML-NOT-ETSI) Reason: The structure of the signature is not valid!

What is happening? Can you help me?

[bseddon]

I'll try my best to provide helpful information but I think I'm someone who is blind leading the blind. In a moment I will refer to 'OCSP' which is a protocol certificate authorities can use to confirm to a client (such as a browser of a XAdES checker) that a certificate is valid.

Validators

I can't really comment on how the EU validator works. As I mention in the README I use the validator created by ETSI who are the ESO that created the specification. My recommendation is that you follow the link in the README and ask for a username and password. The wonderful thing about the ETSI validator is that it provides element by element validation of the signature.

You will see in the README a link to a response by Juan Carlos Cruellas to my question about the versions of the specification. This repository is managed by staff within ETSI who are responsible for the specification and related tools. They may be able to help with information about the checker you are using (they may have created it).

Response to my signature

So, to the responses you have received. The first one looks OK in the sense that the checker has identified the signature is an XAdES baseline form. The failure is also correct. The signature you used from my site stores both the certificate used by me and the certificate of the issuing CA. However, the issuing CA is also me but there is no OCSP responder for the CA at the moment. This means when a validator tries to verify the certificate chain it will fail because my CA will not respond to confirm the root certificate is currently valid. As a result, the signature can be validated but the certificate chain verification problem means the signature status is indeterminate. Remember, any CA might fail to respond correctly because of an network error and this is why, I think, the status is reported as 'indeterminate' not 'error'. They will expect a validator to try again later and then make their own judgment call about the safety of the signature.

By the way, this is why the 'C' and 'A' forms described by the specification exist. For a signature to be rock solid it should also include a timestamp and information such as a valid OCSP response from a CA that the signing certificate was valid at one point. This OCSP response should also be timestamped. This way, when a signature is verified in the future, long after the certificate expired, a verifier will be able to review the timestamps and OCSP responses to confirm the signing certificate was valid at the time the signature was created. As mentioned in the README the code provides theoretical support for these 'C' and 'A' forms but it is completely untested at the moment.

Response to your signature

Your example is failing for a different reason. As you can see at the end of the screenshot, the checker confirms the signature is valid. However, the certificate you have used is not good enough for it. Depending upon the policy being used, an XAdES signature may need to be created using a certificate generated by a Secure Signature Generation Device. This is the reference to QSCD (Qualified Electronic Signature Creation Device). Sounds ominous but its really just a web site though it could be something else.

The problem QSCD is trying to solve is to answer the question: how do you know the person creating the signature has the necessary authority to create the signature? I could sign the Microsoft year end financials (reported to the SEC using an XML-based format called XBRL) using my own certificate but it would mean nothing. So ETSI and the other ESOs have come up with the idea that certificates used to create signatures must have been created by an appropriate QCSD if the policy demands it.

It seems that EU checker does demand that for a signature to be valid, the signing certificate must have been created by a QSCD. As your certificate is not from a QSCD this part of the signature verification fails.

A QSCD is just a site owned and managed by a company that is on a list of valid companies. To get on the list, companies must agree to abide by a set of rules defined in an EU directive and agree to have their operations audited on request. These companies are expected to undertake background checks on individuals requesting a certificate to verify they are who they say they are. For example, in order for an auditor to be able to sign the financials for a Netherlands company, their certificate must have been generated by a QSCD recognised by the NL government which has verified the auditor is a member of the Netherlands institute of chartered accountants.

[sangar82]

I've a solution. Is about how the public key is stored in the xml.

I use a .p12 key to sign

        openssl_pkcs12_read(
            file_get_contents($sign_certificate),
            $certData,
            $sign_certificate_password
        );

     XAdES::signDocument(
            new InputResourceInfo(
                $path_xml, // The source document
                ResourceInfo::file, // The source is a url
                $file['dirname'], // The location to save the signed document
                $file['filename'].'_sign', // The name of the file to save the signed document in,
                null,
                false
            ),
            new CertificateResourceInfo( $certData['cert'], ResourceInfo::string ),
            new KeyResourceInfo( $certData['pkey'], ResourceInfo::string ),

My xml signed look like this

<ds:KeyInfo><ds:X509Data><ds:X509Certificate>-----BEGIN CERTIFICATE-----
MIIIhzCCB2+gAwIBAgIQElPSyUiEif1gsNsmOSEY/jANBgkqhkiG9w0BAQsFADBN
MQswCQYDVQQGEwJFUzERMA8GA1UECgwIRk5NVC1SQ00xDjAMBgNVBAsMBUNFUkVT
...
...
...
qd/n2u37sco1W6n8MA3NodDKKLpob+hJskY4
-----END CERTIFICATE-----
</ds:X509Certificate></ds:X509Data></ds:KeyInfo>

If I delete manually in the xml '-----BEGIN CERTIFICATE-----', '-----END CERTIFICATE-----' and delete all the breaklines (having the certificate in a single line) the validator show me that the xml have now Xades format

I've tried to delete the strings and the breaklines before sent to the signDocument function, but I got an exception

        $certData['cert'] = str_replace("\n",'', $certData['cert']);
        $certData['cert'] = str_replace('-----BEGIN CERTIFICATE-----','', $certData['cert']);
        $certData['cert'] = str_replace('-----END CERTIFICATE-----','', $certData['cert']);

"message":"Unable to find the file 
MIIIhzCCB2+gAwIBAgIQElPSyUiEif1gsNsmOSEY/jANBgkqhkiG9w0BAQsFADBN
MQswCQYDVQQGEwJFUzERMA8GA1UECgwIRk5NVC1SQ00xDjAMBgNVBAsMBUNFUkVT ...
....
raUCwSqXFBsqmFbTVht+qnmwQmguJvjlOEO6bJhsOw5062+u/d67vdT1z7Dx+dcR
qd/n2u37sco1W6n8MA3NodDKKLpob+hJskY4

","error_trace":"#0 /home/vagrant/laravel/vendor/lyquidity/requester/src/OCSP/CertificateLoader.php(46): lyquidity\\Asn1\\Exception\\Asn1DecodingException::create()
#1 /home/vagrant/laravel/vendor/lyquidity/xml-signer/src/XAdES.php(880): lyquidity\\OCSP\\CertificateLoader->fromFile()
#2 /home/vagrant/laravel/vendor/lyquidity/xml-signer/src/XAdES.php(380): lyquidity\\xmldsig\\XAdES->createQualifyingProperties()
#3 /home/vagrant/laravel/vendor/lyquidity/xml-signer/src/XAdES.php(142): lyquidity\\xmldsig\\XAdES->signXAdESFile()

Can you write in the XML the public key without the 'begin end' strings and in a single line? Is this the solution?

Thank you!!

[sangar82]

Thanks for your comment! I'm writing the last comment at the same time. I will request a user/password to try too! About the QSCD I´m waiting for a new company private key provided by a company, I will ask if is signed with a QSCD

[bseddon]

You need to let the signer know it is being passed a certificate in a PEM format (with the ----BEGIN CERTIFICATE----) stuff. By default when a string resource info type is used its assumed the string is just the base64 encoding (like after your manual change). To let the signer know the format of the certificate is PEM 'or' this flag with the string flag:

new CertificateResourceInfo( $certData['cert'], ResourceInfo::string | ResourceInfo::pem )

When a file is used, a PEM type is assumed so the pem flag is implicit.

Your question lets me know its time to add more information about some of these features of the code base.

[bseddon]

PS Technically, the same is true of the resource info for the private key. However since this is never stored and the openssl function s know how to handle a key in any format, its not important. A reason for including the pem flag in the key resource info is to let future developers know.

[sangar82]

Using new CertificateResourceInfo( $certData['cert'], ResourceInfo::pem ) get the following error:

"message":"The resource supplied representing the certificate to be recorded in the signature is not valid.","error_trace":"#0 /home/vagrant/laravel/vendor/lyquidity/xml-signer/src/XAdES.php(142): lyquidity\\xmldsig\\XAdES->signXAdESFile()

Only if I use ResourceInfo::string works.

The $certData['cert'] value is a string :

"-----BEGIN CERTIFICATE-----\nMIIIhzCCB2+gAwIBAgIQElPSyUiEif1gsNsmOSEY/jANBgkqhkiG9w0BAQsFADBN\nMQs ... hJskY4\n-----END CERTIFICATE-----\n"

Could the error be from using p12 certificate and get private key and public from it?

Edit: With ResourceInfo::string the process works well, but isnt't a desired output (include begin and end certificate strings and breaklines)

[bseddon]

The issue is that you have gone from ResourceInfo::string to ResourceInfo::pem These values need to be 'or'd together as I show in the previous post. If you leave out ResourceInfo::string the signer will assume you are providing a path to a file. However you are providing a PEM so when the signer tries to read the file it fails and reports an error. Again, you need to use:

new CertificateResourceInfo( $certData['cert'], ResourceInfo::string | ResourceInfo::pem )

By using ResourceInfo::string | ResourceInfo::pem you are declaring that the resource is a string which contains a certificate in a PEM format. If you only use ResourceInfo::string (which you did earlier) you are declaring the resource is a string which contains a certificate in a base 64 format (the PEM format is a base 64 encoded certificate chunked at 76 characters and with ----BEGIN xxx---- header and -----END xxx ----- footer).

There is nothing wrong with your .p12. You just need to use the flags shown in this message,

[sangar82]

OK! I didn't understand you, sorry!

It's works! Excellent!!

I am impressed with the way you set the options, good job! Another time, thanks for your help and your time! Now, we will try to timestamp the document

Thank you again @bseddon

[bseddon]

If you try to create a timestamp using the default timestamp authority you may need to provide a certificate bundle. The TSA request may fail if you do not provide one. The easiest way to provide a bundle is to set a global to the path to your bundle:

global $certificateBundlePath;
$certificateBundlePath = "...path to my bundle file...";