Open vokac opened 2 years ago
mlassnig
commented
2 years ago Hi, this blocks us from using the default CERN CA provided host certificates when dealing with commercial Amazon cloud storage. As a workaround for now, is there something we can tell sites to do, e.g. some configuration parameter on dCache? Or do we have to slice'n'dice the host certificate that we get from CERN CA so dCache would accept it? Thanks.
paulmillar
commented
2 years ago Hi guys,
Sorry for the delay ... I was off on holiday.
WOW! There's a lot to say about this issue.
I've broken this down into sections, to make it more digestible.
The information, as presented, isn't terribly easy to read and process.
In this section, I'm reworking only the information (as provided) to make it easier to read. This is partly to explain my thinking and partly to help anyone else who is reading this issue.
First, here is dCache's error message slightly edited to make it more readable:
The peer's certificate with subject's DN
CN=atlas-amazon-cloud.cern.ch,OU=computers,DC=cern,DC=ch
was rejected. The peer's certificate status is: FAILED
The following validation errors were found:
* error at position 4 in chain, problematic certificate subject:
CN=CERN Certification Authority,DC=cern,DC=ch
The certificate subject is not accepted by any rule of the
relevant namespace policies. Policies which matches the issuer
are:
/etc/grid-security/certificates/c2a48ab6.namespaces:1
/etc/grid-security/certificates/c2a48ab6.namespaces:3
* error at position 4 in chain, problematic certificate subject:
CN=CERN Certification Authority,DC=cern,DC=ch
The certificate subject is not accepted by any rule of the
relevant namespace policies. Policies which matches the issuer
are:
/etc/grid-security/certificates/c2a48ab6.signing_policy
/etc/grid-security/certificates/c2a48ab6.signing_policy
* error at position 5 in chain, problematic certificate subject:
CN=CERN Certification Authority,DC=cern,DC=ch
The certificate subject is not accepted by any rule of the
relevant namespace policies. Policies which matches the issuer
are:
/etc/grid-security/certificates/c2a48ab6.namespaces:1
/etc/grid-security/certificates/c2a48ab6.namespaces:3
* error at position 5 in chain, problematic certificate subject:
CN=CERN Certification Authority,DC=cern,DC=ch
The certificate subject is not accepted by any rule of the
relevant namespace policies. Policies which matches the issuer
are:
/etc/grid-security/certificates/c2a48ab6.signing_policy
/etc/grid-security/certificates/c2a48ab6.signing_policyHere is a summary of the certificates presented, based on the information Petr provided:
0 s:DC = ch, DC = cern, OU = computers, CN = atlas-amazon-cloud.cern.ch
i:DC = ch, DC = cern, CN = CERN Grid Certification Authority
1 s:DC = ch, DC = cern, CN = CERN Root CA
i:DC = ch, DC = cern, CN = CERN Root CA
2 s:C = ch, O = CERN, CN = CERN Root Certification Authority 2
i:C = ch, O = CERN, CN = CERN Root Certification Authority 2
3 s:DC = ch, DC = cern, CN = CERN Grid Certification Authority
i:C = ch, O = CERN, CN = CERN Root Certification Authority 2
4 s:DC = ch, DC = cern, CN = CERN Certification Authority
i:C = ch, O = CERN, CN = CERN Root Certification Authority 2
5 s:DC = ch, DC = cern, CN = CERN Certification Authority
i:C = ch, O = CERN, CN = CERN Root Certification Authority 2
6 s:O = IPAdev CERN, CN = Certificate Authority
i:O = IPAdev CERN, CN = Certificate AuthorityIn an effort to make this information more understandable, I've reformulated this in terms of the certificates roles: EEC, Intermediate CA and Root CA. I've left off the certificate's subject (for the EEC and Root CA certificates), and included the issuer as "by". I've also changed any Certificate Authority in the CN to CA, to make this more readable.
0. EEC (by CERN Grid CA)
1. Root (by CERN Root CA, self-signed)
2. Root (by CERN Root CA 2, self-signed)
3. Intermediate: CERN Grid CA (by CERN Root CA 2)
4. Intermediate: CERN CA (by CERN Root CA 2)
5. Intermediate: CERN CA (by CERN Root CA 2)
6. Root (by IPAdev CERN, self-signed)In the following section, I've reworking this further. This shows the certificate chains contained within the certificates returned by the server. The number underneath each phrase indicates the certificate that identifies that subject. The arrow (<---) indicates a "signed by" relationship; i.e., the certificate's issuer.
EEC <--- CERN Grid CA <--- CERN Root CA 2 (self)
0. 3. 2.
CERN Root CA (self)
1.
CERN CA <--- CERN Root CA 2 (self)
4. 5. 2.
IPAdev CERN (self)
6.I have a few comments about the certificates presented by the server.
Including any root CA certificates in the certificate chain is pointless. Either a root CA is trusted by the peer or it isn't, including the root certificate doesn't change this.
Rather, including a root certificate just makes the TLS handshake take (slightly) longer. More data will need to be transferred, possibly over multiple TLS frames.
Best practice is to avoid including any root CA certificates in TLS handshakes.
The server presented many certificates that are not helpful in establishing the trust chain.
Both IPAdev and CERN Root CA certificates are not at all useful in establishing the trust chain. These two are also self-signed root CA certificates, so (first point again) shouldn't be included anyway.
The "CERN Certification Authority" certificate (signed by "CERN Root Certification Authority 2") is also useless in establishing the trust chain. This should have no effect, but including these certificates would anyway make the TLS handshake take longer.
Strangely enough, this "CERN Certification Authority" certificate is included twice. Why do this? Duplicating certificates is pointless and just makes the TLS handshake take longer.
The EEC is signed by "CERN Grid Certification Authority", which is part of the standard IGTF trust bundle.
In theory, you should not need to include any intermediate or root CAs as the EEC should be sufficient if the client accepts the IGTF trust bundle.
The problem (as reported) is that dCache rejected the connection because the server presented two certificates (certificates 4. and 5.) that failed the naming policies of the issuing CA.
Certificates 4. and 5. are actually the same certificate (duplicated), so this is really one problem.
The standard IGTF trust chain does include "CERN Root CA 2". The policy files for this CA (at least, for IGTF v1.116) allow this CA to issue only two certificates:
DC=ch, DC=cern, CN=CERN Grid Certification Authority
DC=ch, DC=cern, CN=CERN LCG IOTA Certification AuthoritySo, per the signing policies / namespace rules for "Root CA 2", this CA should never have signed "DC=ch, DC=cern, CN=CERN Certification Authority".
Therefore, dCache is operating correctly when it rejects the "Root CA 2" certificate.
One might argue that the "Root CA 2" certificate is irrelevant in establishing the trust for the remote server's DN.
The fact that the EEC is issued by a trusted CA ("CERN Grid Certification Authority" ) and follows the signing policy/namespace for that CA should be sufficient to establish the trust relationship and for dCache to accept the server's DN.
Following this argument, the server should be free to include whatever garbage (expired, self-signed, policy-violating) certificates it wants in the set it returns to the client. It is sufficient for the client to establish a trust chain for the peer's identity to be accepted. The client should simply ignore any garbage certificates.
I believe this is the thrust of Petr's comment.
I'm sympathetic to that point-of-view; however, I'm also not sure of all the security trade-offs involved, which are often far from obvious or trivial.
Moreover, dCache does not implement this behaviour itself. Instead, it relies on an external library (CaNL) to handle certificate validation. dCache rejected the server because CaNL said the presented certificates were unacceptable.
The behaviour you described should probably be opened as an issue against CaNL, so it can either be fixed or the development team can explain why the current policy is correct.
The signing policy is something introduced by Globus / IGTF to protect identities; specifically, it protects that no two (different) CAs will issue the same DN to two different entities.
In practice, the problem (this policy protects against) only affects certificates issued to people or robots.
The problem doesn't affect host certificates. This is for two main reasons:
The rational for ubiquitous namespace checking is that it is often hard for software to know when a certificate comes from a user or from a remote server: host certificates are sometimes used as client certificates. IGTF tried (and failed) to address this problem.
However, when dCache is using a built-in client (e.g., as the active party in an HTTP-TPC transfer), the pool knows the remote party is an HTTP server. Therefore, these namespace checks actually make no sense for the active party when establishing the data-bearing HTTP transfer.
dCache configuration is quite flexible. It contains the configuration property pool.mover.http-tpc.authn.namespace-mode that controls how stringently the namespace is verified. In current dCache versions, the pool will (through various defaults) enforce namespace policies on HTTP-TPC transfers. Setting this property to IGNORE will switch off namespace verification for HTTP-TPC transfers.
This particular problem is not the first time we've encountered such problems. There is even a patch (see commit 718e3af4df) that updates the defaults so any transfer where dCache is using a client will ignore namespace checks (the pool.authn.namespace-mode configuration property). This commit is "master only". We have no plans to back-port it.
There's basically three independent approaches here:
My suggestion would be to go with 1. to solve the problem. This is likely the quickest way to get transfers working, as it only requires changes to the server involved.
Option 3. is also an option. We (dCache.org) want to go in this direction, but we're also conscious of potential disruption, so do not want to back-port this change. If a dCache instance is operating within a specific VO and that VO is willing to accept the risk then this process could (easily) be accelerated.
Option 2. should (probably) also be explored; however, this will also be the route that will (likely) take the longest before transfers are successful again.
thdesy
commented
2 years ago Hi all,
just adding my own observations. Last month we received complaints about problems with our HTCondorCEs after we had updated the CEs' host certificates from the previous GridKa CA to new EANT TCS/Sectigo multi-domain host certificates. The problem seems to occur with CE clients failing to correctly parse the pem file bags, which consists of the actual host certificate, the GEANT CA cert and the USERTrust CA Cert signing each other in the chain (with both, GEANT & USERTrust, certs available in the usual grid securtiy certificate dir under their DN symlinks). The issue seems to affect primarily python2.7 clients with older evrsions of gfal/globus, i.e., after updating their python2 DIRAC to the current python3 DIRAC (+libs) Belle2 & ILC could submit again.
In the meantime, we have updated also all of our dCache doors to Sectigo multi-domain certs as well and received again complaints by NAF users, who had issues reading files via Grid protocols including gFTP (where the users had some established workflows, where they are apparently using gFTP due to reasons) - which again seemed to be related for ATLAS users to Athena releases with older clients.
No idea, if it is 100% related, but a bit more complex pem cert bags might cause problems with some client - SEs as well as CEs.
Cheers, Thomas
msalle
commented
2 years ago Hi all,
just adding my own observations. Last month we received complaints about problems with our HTCondorCEs after we had updated the CEs' host certificates from the previous GridKa CA to new EANT TCS/Sectigo multi-domain host certificates. The problem seems to occur with CE clients failing to correctly parse the pem file bags, which consists of the actual host certificate, the GEANT CA cert and the USERTrust CA Cert signing each other in the chain (with both, GEANT & USERTrust, certs available in the usual grid securtiy certificate dir under their DN symlinks). The issue seems to affect primarily python2.7 clients with older evrsions of gfal/globus, i.e., after updating their python2 DIRAC to the current python3 DIRAC (+libs) Belle2 & ILC could submit again.
In the meantime, we have updated also all of our dCache doors to Sectigo multi-domain certs as well and received again complaints by NAF users, who had issues reading files via Grid protocols including gFTP (where the users had some established workflows, where they are apparently using gFTP due to reasons) - which again seemed to be related for ATLAS users to Athena releases with older clients.
No idea, if it is 100% related, but a bit more complex pem cert bags might cause problems with some client - SEs as well as CEs.
Cheers, Thomas
Hi all, just to point out that the issue from @thdesy is the same Sectigo issue (but then for hostcerts) as mentioned in https://github.com/eu-emi/canl-java/issues/105 which is not really the same issue as the one reported here. (@dlgroep send a long reply to the wlcg-resource-trust-evolution which might be useful to include here but I don't want to pollute this ticket too much). In short, users need to clean up the chain they get from Sectigo since it contains the wrong CA certs, in particular it should contain the self-signed USERTrust (instead of the subCA version).
mlassnig
commented
2 years ago I can confirm that the third party copy works when the certificate chain on AWS is sliced'n'diced to "only the relevant issuers".
vokac
commented
1 year ago What is the progress with this issue? I can see again fancy certificate chain for DESY dCache that should be fine, but dCache TPC fails
$ export SRC=https://lcg-se0.ifh.de:2880/pnfs/ifh.de/data/atlas/atlasdatadisk/rucio/mc16_13TeV/45/7f/HITS.31660887._173089.pool.root.1
$ export DST=https://lcg-se0.ifh.de:2880/pnfs/ifh.de/data/atlas/atlasdatadisk/SAM/x
$ export TSRC=$(curl --silent --cert /tmp/x509up_u$(id -u) --key /tmp/x509up_u$(id -u) --cacert /tmp/x509up_u$(id -u) --capath /etc/grid-security/certificates -X POST -H 'Content-Type: application/macaroon-request' -d '{"caveats": ["activity:DOWNLOAD"], "validity": "PT30M"}' "$SRC" | jq -r '.macaroon')
$ export TDST=$(curl --silent --cert /tmp/x509up_u$(id -u) --key /tmp/x509up_u$(id -u) --cacert /tmp/x509up_u$(id -u) --capath /etc/grid-security/certificates -X POST -H 'Content-Type: application/macaroon-request' -d '{"caveats": ["activity:UPLOAD,DELETE,LIST"], "validity": "PT30M"}' "$DST" | jq -r '.macaroon')
$
$ curl -v --capath /etc/grid-security/certificates -L -X COPY -H 'Credentials: none' -H "Authorization: Bearer $TDST" -H "TransferHeaderAuthorization: Bearer $TSRC" -H "Source: $SRC" "$DST"
* About to connect() to lcg-se0.ifh.de port 2880 (#0)
* Trying 2001:638:700:f0c8::1:10...
* Connected to lcg-se0.ifh.de (2001:638:700:f0c8::1:10) port 2880 (#0)
...
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* subject: CN=lcg-se0.ifh.de,O=Deutsches Elektronen-Synchrotron DESY,ST=Hamburg,C=DE,DC=tcs,DC=terena,DC=org
* start date: Jan 02 00:00:00 2023 GMT
* expire date: Feb 01 23:59:59 2024 GMT
* common name: lcg-se0.ifh.de
* issuer: CN=GEANT eScience SSL CA 4,O=GEANT Vereniging,C=NL
> COPY /pnfs/ifh.de/data/atlas/atlasdatadisk/SAM/x HTTP/1.1
> User-Agent: curl/7.29.0
> Host: lcg-se0.ifh.de:2880
> Accept: */*
> Credentials: none
> Authorization: Bearer SECRET1
> TransferHeaderAuthorization: Bearer SECRET2
> Source: https://lcg-se0.ifh.de:2880/pnfs/ifh.de/data/atlas/atlasdatadisk/rucio/mc16_13TeV/45/7f/HITS.31660887._173089.pool.root.1
>
< HTTP/1.1 202 Accepted
< Date: Mon, 02 Jan 2023 22:26:57 GMT
< Server: dCache/7.2.26
< Content-Type: text/perf-marker-stream
< Transfer-Encoding: chunked
<
Perf Marker
Timestamp: 1672698417
State: 9
State description: Requesting mover
Stripe Index: 0
Total Stripe Count: 1
End
failure: The peer's certificate with subject's DN CN=lcg-se0.ifh.de,O=Deutsches Elektronen-Synchrotron DESY,ST=Hamburg,C=DE,DC=tcs,DC=terena,DC=org was rejected. The peer's certificate status is: FAILED The following validation errors were found:;error at position 2 in chain, problematic certificate subject: CN=USERTrust RSA Certification Authority,O=The USERTRUST Network,L=Jersey City,ST=New Jersey,C=US (category: NAMESPACE): Namespace definition for the certificate issuer (CN=AAA Certificate Services,O=Comodo CA Limited,L=Salford,ST=Greater Manchester,C=GB) is not defined, and namespaces are configured to be required.
* Connection #0 to host lcg-se0.ifh.de left intactProblematic is the last (2 in chain) certificate with issuer /C=GB/ST=Greater Manchester/L=Salford/O=Comodo CA Limited/CN=AAA Certificate Services that's not defined in the IGTF namespaces
$ echo "" | openssl s_client -connect lcg-se0.ifh.de:2880 -showcerts
CONNECTED(00000003)
depth=3 C = GB, ST = Greater Manchester, L = Salford, O = Comodo CA Limited, CN = AAA Certificate Services
verify return:1
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
verify return:1
depth=1 C = NL, O = GEANT Vereniging, CN = GEANT eScience SSL CA 4
verify return:1
depth=0 DC = org, DC = terena, DC = tcs, C = DE, ST = Hamburg, O = Deutsches Elektronen-Synchrotron DESY, CN = lcg-se0.ifh.de
verify return:1
---
Certificate chain
0 s:/DC=org/DC=terena/DC=tcs/C=DE/ST=Hamburg/O=Deutsches Elektronen-Synchrotron DESY/CN=lcg-se0.ifh.de
i:/C=NL/O=GEANT Vereniging/CN=GEANT eScience SSL CA 4
-----BEGIN CERTIFICATE-----
MIII2zCCBsOgAwIBAgIQWIooUzhsgK/O8DiGskB/TjANBgkqhkiG9w0BAQwFADBK
MQswCQYDVQQGEwJOTDEZMBcGA1UEChMQR0VBTlQgVmVyZW5pZ2luZzEgMB4GA1UE
AxMXR0VBTlQgZVNjaWVuY2UgU1NMIENBIDQwHhcNMjMwMTAyMDAwMDAwWhcNMjQw
MjAxMjM1OTU5WjCBqjETMBEGCgmSJomT8ixkARkWA29yZzEWMBQGCgmSJomT8ixk
ARkWBnRlcmVuYTETMBEGCgmSJomT8ixkARkWA3RjczELMAkGA1UEBhMCREUxEDAO
BgNVBAgTB0hhbWJ1cmcxLjAsBgNVBAoTJURldXRzY2hlcyBFbGVrdHJvbmVuLVN5
bmNocm90cm9uIERFU1kxFzAVBgNVBAMTDmxjZy1zZTAuaWZoLmRlMIICIjANBgkq
hkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwIIyijPI1+WV/gj3K+7lNF77+30DMYm3
V/Swu63DAgr6MQYSRpg6MWUcMz6+sU3L8N5k5Ri9+I0WUYqNKd+qK+POWz3T6FTL
wTFedIyRfpFWbe1Y8zLwvjNKfttppE+julygqbJnJnmNfHpSwMX5zDBeyjx8c5f/
iqOAhW+QdK5CvFhRlWG2cQjjj7CBi4gn/5aZP1Bak0jgPzy1ELOeSSSI+VIY6Kh6
BWJF+jV4vDvffhhEw66V96fUSB6G1nOQa+Jc+6MaLNr/3y8wAGfsOGmT+w32sva/
Omh1wXn0NEe+h1NkUGzcBtalfQ9bLszVYXr1lbgcGrPmWUSumjSua22dDbGENJLW
6qx5UskZPv2Z1hNA1qZAM1LFzAPzo1hVbxiwWjQbPUAprLTvkXyOA38sw9kZ+wfl
hvopJcUvhUL3CukirWd6XThyUCEMZzY7sVh72yXYe1vf5kaTAQ1T7yv8AymrL7Ua
VL+KlSnPmuP1PlnQCDgb29MLe7+hhugy3txGtTjZutvBFjCFJFoSw5wEW+SuDZ7a
7lJPPqzpnEc0H2HYPs2TLNpxbyEtnM0ljaXRyvJbskjbgpBPuIrCtR3ZLIXc8GTQ
2TFnTenjEd6pbltdl5IhQFUr20b17Bjpr/F+46Ig1jLuhBcIhIPym7pw6TZ3uSrY
KNSIQJPSEhkCAwEAAaOCA1owggNWMB8GA1UdIwQYMBaAFJoriiLWjQzAKqVvZDM/
lmBnFR2yMB0GA1UdDgQWBBQyPqF89Wi2JGMkXq1YI9SeSGmEaDAOBgNVHQ8BAf8E
BAMCBaAwDAYDVR0TAQH/BAIwADAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUH
AwIwdQYDVR0gBG4wbDAMBgoqhkiG90wFAgIBMA0GCyqGSIb3TAUCAwMCMA0GCyqG
SIb3TAUCAwECMDQGCysGAQQBsjEBAgJPMCUwIwYIKwYBBQUHAgEWF2h0dHBzOi8v
c2VjdGlnby5jb20vQ1BTMAgGBmeBDAECAjBFBgNVHR8EPjA8MDqgOKA2hjRodHRw
Oi8vR0VBTlQuY3JsLnNlY3RpZ28uY29tL0dFQU5UZVNjaWVuY2VTU0xDQTQuY3Js
MHsGCCsGAQUFBwEBBG8wbTBABggrBgEFBQcwAoY0aHR0cDovL0dFQU5ULmNydC5z
ZWN0aWdvLmNvbS9HRUFOVGVTY2llbmNlU1NMQ0E0LmNydDApBggrBgEFBQcwAYYd
aHR0cDovL0dFQU5ULm9jc3Auc2VjdGlnby5jb20wggF/BgorBgEEAdZ5AgQCBIIB
bwSCAWsBaQB2AHb/iD8KtvuVUcJhzPWHujS0pM27KdxoQgqf5mdMWjp0AAABhXG0
bsMAAAQDAEcwRQIhAKD/w5uRWRua3nodivewmdDHVZEAJSMSSoRYBTbZF0clAiBO
94k+KA0FB5zfoCNtWaM4v1upqmOrPHubPq0kobMX/AB2ANq2v2s/tbYin5vCu1xr
6HCRcWy7UYSFNL2kPTBI1/urAAABhXG0bs4AAAQDAEcwRQIgVIlQoPKai0HYQW95
0gjK/wKYnfJ14UIDZxqk9wio2NACIQC9WI0iecsD4ySsUj3dzehsRpkvrN37MTm3
uHcwnkue7QB3AO7N0GTV2xrOxVy3nbTNE6Iyh0Z8vOzew1FIWUZxH7WbAAABhXG0
bqYAAAQDAEgwRgIhAP5BpZBXkwIIBGaGgV7ZVMUUCgnB3V03zb0FDf4mxZ31AiEA
zz0ADsVDu+X2v5yQk4LEFb4leRn+PDMyAc6KFHQYW9wwGQYDVR0RBBIwEIIObGNn
LXNlMC5pZmguZGUwDQYJKoZIhvcNAQEMBQADggIBAF8wTbQHnvSlngH+GMFfGgji
PEktUz8hDb6g9mqvJOv3dVIB+g7lUshzbS3uo6FAtT32joIB6wMms9PRQh2greug
SV3OWOcjsQYeLE6zG+c5yHB5obK4OGhV0dVP/D+8rBj+QcYndkfvfUv4n5VEN6Nm
+grVmCxpppN1NdDMS3QHysqWLhOJz6v3r62y4Zm2mwSirasUY9hUmYZ57nC/rYlC
q1BCqKxav18cbfsnWQJNvxladgJYyn+o9umulRh0k37WytDUu8FU03AaszFFqJfJ
iBSe18X3D+9BCsQ30QM+bXHDXsUhPas+vMYb3wvq6mGQYTrBYT8khxg946tHF/Jp
RlDXnskYmNcecVzpNi2kPtLFMh+Mo2P+U2GP7rjPEOgIWkOxOocCk2wzaiR1vT/r
/jRIOcKuIgqVRRSBOarTOs/GYZxV22mcQvfWMjw52Ibsd5Oo/SSmd/jPrzu4ZJXB
HW2V5vtDcoo4EUXxzz4pzRSqMJMN1yu+z86NLxaiG9JgJVY4m6A82ksMazRk4njV
wqm61DDUGXa9RkjBnMghLXaystK0+agd1qn775LFLMqmOQz/U3blPlmeQAG0fUB0
Bkfdal+ZYwgRfU8fhgnzsXJxMfSXqqqA8PJ1xEaosP9TnRNBdv9TwW5bacRQvbm7
aP/vWk3lO/6kxfgFPcJ7
-----END CERTIFICATE-----
1 s:/C=NL/O=GEANT Vereniging/CN=GEANT eScience SSL CA 4
i:/C=US/ST=New Jersey/L=Jersey City/O=The USERTRUST Network/CN=USERTrust RSA Certification Authority
-----BEGIN CERTIFICATE-----
MIIG6jCCBNKgAwIBAgIQN9uaOAbXCsc+ONHf725RKjANBgkqhkiG9w0BAQwFADCB
iDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0pl
cnNleSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNV
BAMTJVVTRVJUcnVzdCBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMjAw
MjE4MDAwMDAwWhcNMzMwNTAxMjM1OTU5WjBKMQswCQYDVQQGEwJOTDEZMBcGA1UE
ChMQR0VBTlQgVmVyZW5pZ2luZzEgMB4GA1UEAxMXR0VBTlQgZVNjaWVuY2UgU1NM
IENBIDQwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCUtBhby5HZ4dxN
stR0OaxF/QrC0FENJm59Na5LZzPwXqwTdaq1NJUZwMIrHTXavExQpGqzdqwIKNS0
liyLkTT54vTs5VPh5SYRFtVAX9NLkY+fUIu2icOEGfwlPR6VlYp+kiR2Vj4OwYhy
aM0tj66uaTCaAj3Rz99dN1s64Z1jw/wjckCLat/MGHL4crKuQErYyPfoK1s39BR/
gXis0E2anvfmuyAZ3YDOsAyGKpPhEj3Dxuiz5+qpqJIePs3Vgylyx/ZIoJEH/jAq
v7GbdVTsSfqyMlnyceF0pyGQ6Yzp1DkLw4Dg26zoOSckLYeICtRziSwzrMu3YLYW
VB6h1r0Nw/hnuXMYt6WgLYekfxuwapwbntbw5gAJpK63geBp3Hj7bWnrPb8aLjBz
eutA2St+Y6YodQ8PjbhUpaWVMGzz+c2YJK/xbxTAjWUPOdd8cNrM7ilaOrvbSaLz
e7aPBFWZMpg1kOva5J91qSdOmS45DHyyfT4Xk2X/3HhFRLkKA/cdJaEYE1I/p0P5
A2qFlMdr+QDx7Uar5Zvsbvrm0gn+MER1uq3xugoweC3iegHwQQh4YupAEamo1vPy
n04r3WPn86rLjpe0OLFvlM8CQtvTWV6wOXy3HTnIh8gFGxe/0Wu3XYBtPcJXlBMO
aV7NDi69LE9ZiYNF+wBZ6iUgVw3ZRQIDAQABo4IBizCCAYcwHwYDVR0jBBgwFoAU
U3m/WqorSs9UgOHYm8Cd8rIDZsswHQYDVR0OBBYEFJoriiLWjQzAKqVvZDM/lmBn
FR2yMA4GA1UdDwEB/wQEAwIBhjASBgNVHRMBAf8ECDAGAQH/AgEAMB0GA1UdJQQW
MBQGCCsGAQUFBwMBBggrBgEFBQcDAjA4BgNVHSAEMTAvMC0GBFUdIAAwJTAjBggr
BgEFBQcCARYXaHR0cHM6Ly9zZWN0aWdvLmNvbS9DUFMwUAYDVR0fBEkwRzBFoEOg
QYY/aHR0cDovL2NybC51c2VydHJ1c3QuY29tL1VTRVJUcnVzdFJTQUNlcnRpZmlj
YXRpb25BdXRob3JpdHkuY3JsMHYGCCsGAQUFBwEBBGowaDA/BggrBgEFBQcwAoYz
aHR0cDovL2NydC51c2VydHJ1c3QuY29tL1VTRVJUcnVzdFJTQUFkZFRydXN0Q0Eu
Y3J0MCUGCCsGAQUFBzABhhlodHRwOi8vb2NzcC51c2VydHJ1c3QuY29tMA0GCSqG
SIb3DQEBDAUAA4ICAQAmpZcT3x9evRVo7pAAsdyTx3mmtbOGWIAQFQRiW4po1Jx9
9szSmymPrzDq0s1Lh7EL+sworQnNKEApLYpcFZ/ZeyjWPt+7D/SKGdjhFE9owf71
hNiwWBWzMHZWE/1lYpISw2gbclDk6Mp1GkDQhRVXQL5O6uXeouxNiKpAXZj8QFx7
Sk1kkuXewca4dFRTs9Oadec8ARnR+YzxhTE0p3m2rBkWL3zpNCtl+7zpPJm58eCc
8Oxt7ib/XwoZKE/tKKZ6rcA+kBL6X6xeim+DwVqToJW24yiqVAI2JYQw1741fqm8
lklFlkAoOTJTprMd3jIK2DKYvAXkZiAeY8eD85AVCJgC+DpD/0Oc3OpsP5kljGzF
2rUYGLutL1a9Uxd2YCGkqnAym2f2LsBo5u2+Znk5YzfD5xPSWBrs9K5UtE2G+5B0
iygvf+yl/+hcv4aN3zetNXbfsSArAR0ecdNkNtehXeVaFE7rIZUXFahzHn2cSOLO
n4vjBWKRHJc4lvlqhfz1jaPrcxS0BxaJq0VNrX3skPeajbVSsPAjXXVDh9I5qjdG
yMvAe5TUyUf752JKBN56Zi5mvz5ufbvULGcV4Frnwl2c2ySICF7KGIB+KGhGbgbf
tN6PKXHwC3gvRw9yzFSq6Rdwh7U+3AkmDel0rhdZnMTx2JryjQF/AeBKwkfrUQ==
-----END CERTIFICATE-----
2 s:/C=US/ST=New Jersey/L=Jersey City/O=The USERTRUST Network/CN=USERTrust RSA Certification Authority
i:/C=GB/ST=Greater Manchester/L=Salford/O=Comodo CA Limited/CN=AAA Certificate Services
-----BEGIN CERTIFICATE-----
MIIFgTCCBGmgAwIBAgIQOXJEOvkit1HX02wQ3TE1lTANBgkqhkiG9w0BAQwFADB7
MQswCQYDVQQGEwJHQjEbMBkGA1UECAwSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYD
VQQHDAdTYWxmb3JkMRowGAYDVQQKDBFDb21vZG8gQ0EgTGltaXRlZDEhMB8GA1UE
AwwYQUFBIENlcnRpZmljYXRlIFNlcnZpY2VzMB4XDTE5MDMxMjAwMDAwMFoXDTI4
MTIzMTIzNTk1OVowgYgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpOZXcgSmVyc2V5
MRQwEgYDVQQHEwtKZXJzZXkgQ2l0eTEeMBwGA1UEChMVVGhlIFVTRVJUUlVTVCBO
ZXR3b3JrMS4wLAYDVQQDEyVVU0VSVHJ1c3QgUlNBIENlcnRpZmljYXRpb24gQXV0
aG9yaXR5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAgBJlFzYOw9sI
s9CsVw127c0n00ytUINh4qogTQktZAnczomfzD2p7PbPwdzx07HWezcoEStH2jnG
vDoZtF+mvX2do2NCtnbyqTsrkfjib9DsFiCQCT7i6HTJGLSR1GJk23+jBvGIGGqQ
Ijy8/hPwhxR79uQfjtTkUcYRZ0YIUcuGFFQ/vDP+fmyc/xadGL1RjjWmp2bIcmfb
IWax1Jt4A8BQOujM8Ny8nkz+rwWWNR9XWrf/zvk9tyy29lTdyOcSOk2uTIq3XJq0
tyA9yn8iNK5+O2hmAUTnAU5GU5szYPeUvlM3kHND8zLDU+/bqv50TmnHa4xgk97E
xwzf4TKuzJM7UXiVZ4vuPVb+DNBpDxsP8yUmazNt925H+nND5X4OpWaxKXwyhGNV
icQNwZNUMBkTrNN9N6frXTpsNVzbQdcS2qlJC9/YgIoJk2KOtWbPJYjNhLixP6Q5
D9kCnusSTJV882sFqV4Wg8y4Z+LoE53MW4LTTLPtW//e5XOsIzstAL81VXQJSdhJ
WBp/kjbmUZIO8yZ9HE0XvMnsQybQv0FfQKlERPSZ51eHnlAfV1SoPv10Yy+xUGUJ
5lhCLkMaTLTwJUdZ+gQek9QmRkpQgbLevni3/GcV4clXhB4PY9bpYrrWX1Uu6lzG
KAgEJTm4Diup8kyXHAc/DVL17e8vgg8CAwEAAaOB8jCB7zAfBgNVHSMEGDAWgBSg
EQojPpbxB+zirynvgqV/0DCktDAdBgNVHQ4EFgQUU3m/WqorSs9UgOHYm8Cd8rID
ZsswDgYDVR0PAQH/BAQDAgGGMA8GA1UdEwEB/wQFMAMBAf8wEQYDVR0gBAowCDAG
BgRVHSAAMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwuY29tb2RvY2EuY29t
L0FBQUNlcnRpZmljYXRlU2VydmljZXMuY3JsMDQGCCsGAQUFBwEBBCgwJjAkBggr
BgEFBQcwAYYYaHR0cDovL29jc3AuY29tb2RvY2EuY29tMA0GCSqGSIb3DQEBDAUA
A4IBAQAYh1HcdCE9nIrgJ7cz0C7M7PDmy14R3iJvm3WOnnL+5Nb+qh+cli3vA0p+
rvSNb3I8QzvAP+u431yqqcau8vzY7qN7Q/aGNnwU4M309z/+3ri0ivCRlv79Q2R+
/czSAaF9ffgZGclCKxO/WIu6pKJmBHaIkU4MiRTOok3JMrO66BQavHHxW/BBC5gA
CiIDEOUMsfnNkjcZ7Tvx5Dq2+UUTJnWvu6rvP3t3O9LEApE9GQDTF1w52z97GA1F
zZOFli9d31kWTz9RvdVFGD/tSo7oBmF0Ixa1DVBzJ0RHfxBdiSprhTEUxOipakyA
vGp4z7h/jnZymQyd/teRCBaho1+V
-----END CERTIFICATE-----
---
Server certificate
subject=/DC=org/DC=terena/DC=tcs/C=DE/ST=Hamburg/O=Deutsches Elektronen-Synchrotron DESY/CN=lcg-se0.ifh.de
issuer=/C=NL/O=GEANT Vereniging/CN=GEANT eScience SSL CA 4
---Progress report on the three ways for solving this issue:
Uploading data to the storage that during TLS handshake sends unused certificates fails if these superfluous certificate doesn't pass "globus policy validation", e.g.
Why dCache 7.2.16 tries to validate certificate that is not really part of the chain? Currently
atlas-amazon-cloud.cern.ch:443present these certificates:At the beginning of this output you can also see that openssl takes into account only certificates that are really participate in the host CA parent chain.