Closed kofemann closed 5 months ago
Hi, just to be sure, depending on the CAs you need to support, you may need to enable SHA-1 on your EL9 host, if it will have to support CentOS 7 peers:
update-crypto-policies --set DEFAULT:SHA1
Then reboot the host to ensure the change is picked up by everything that needs it.
Thanks for the prompt reply!
Indeed, by changing the crypto-policies g-u-c works as expected. We obviously can't force that on all clients or expect CAs to use different hash functions. So, I guess we have to change something on dCache side (or java security policy) to keep clients happy...
Hi again, for the record, the matter has been documented at least here:
https://twiki.cern.ch/twiki/bin/view/LCG/EL9vsSHA1CAs
Though CentOS 7 will have its EOL on June 30, there may still be jobs running in containers based on that OS for years to come...
You can do something about it on the server side indeed, enabling SHA-1 just for the dCache services, not the whole host.
I am still confused. Our test server have certs with sha256:
# openssl x509 -in dCacheORG.pem -noout -text | grep Signature
Signature Algorithm: sha256WithRSAEncryption
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
# openssl x509 -in dCacheORG-CA.pem -noout -text | grep Signature
Signature Algorithm: sha256WithRSAEncryption
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
The proxy uses sha256:
# arcproxy -I
Subject: /C=DE/ST=Hamburg/O=dCache.ORG/CN=Kermit the frog/CN=898777003
Issuer: /C=DE/ST=Hamburg/O=dCache.ORG/CN=Kermit the frog
Identity: /C=DE/ST=Hamburg/O=dCache.ORG/CN=Kermit the frog
Time left for proxy: 6 hours 28 minutes 1 second
Proxy path: /tmp/x509up_u0
Proxy type: X.509 Proxy Certificate Profile RFC compliant impersonation proxy - RFC inheritAll proxy
Proxy key length: 2048
Proxy signature: sha256
====== AC extension information for VO desy ======
VO : desy
subject : /C=DE/ST=Hamburg/O=dCache.ORG/CN=Kermit the frog
issuer : /DC=org/DC=terena/DC=tcs/C=DE/ST=Hamburg/O=Deutsches Elektronen-Synchrotron DESY/CN=grid-voms.desy.de
uri : grid-voms.desy.de:15104
attribute : /desy/Role=NULL/Capability=NULL
attribute : /desy/workshop/Role=NULL/Capability=NULL
Time left for AC: 6 hours 28 minutes 1 second
The TLS hanshake uses TLS1.3 with sha384:
Using TLSv1.3.
cred_usage=1
Cipher being used:
TLS_AES_256_GCM_SHA384 TLSv1.3 Kx=any Au=any Enc=AESGCM(256) Mac=AEAD
X509 subject after proxy : /C=DE/ST=Hamburg/O=dCache.ORG/CN=store-door-svc.dcache-build-105787.svc.cluster.local
So, this is not diretly related to GSI, but something different, I guess... 🤷🏼♂️
Interesting!
Might a side effect of enabling SHA-1 be that something else that is needed also gets enabled?
Could it be the signature on the self-signed root CA or one of the intermediates (latter one is less likely) ? That would still give this error I'd say and more likely than any of the other tools.
@msalle the dcache test CA signed with sha256WithRSAEncryption. Is there a way to debug at which point g-u-c tries to use sha1?
Are both server and client certs signed by the same test CA? Is it a rootCA and/or are there intermediates? Also, are you using frontend/backend nodes for dCache or is it all a single node?
In any case I would probably run an strace
and look for (succesfully) opened files in /etc/grid-security/certificates
. It probably might also help to add the -dbg
flag which should give control channel debug info.
But it's also possible it's actually not one of the CAs, although I'm not sure how.
ok, looks like this is related to proxy delegation:
globus_i_gsi_gss_put_token exiting: major_status=0
globus_gsi_proxy_inquire_req entering
****** START X509_REQ ******
Certificate Request:
Data:
Version: 1 (0x0)
Subject: C=DE, ST=Hamburg, O=dCache.ORG, CN=Kermit the frog, CN=2982459500, CN=1261921953
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:ca:90:83:56:6a:11:76:06:32:90:49:5a:a2:25:
c0:bb:d4:2d:c3:69:e7:f0:91:81:90:16:28:d4:e7:
34:8e:b8:7f:71:6d:8e:62:27:56:a5:b9:2a:62:ba:
6e:1a:c8:23:a9:3b:f5:fa:44:ec:b3:c8:4a:f4:6b:
c6:d4:a7:12:10:20:92:6e:9b:b0:f7:be:b5:f7:6c:
8c:fe:1f:33:45:2d:ee:03:db:4a:37:56:89:6a:46:
00:41:14:c0:08:b8:67:e9:e5:90:0d:67:25:3b:48:
3a:27:d2:cc:fa:90:f0:53:c2:72:fd:c1:a9:8e:b8:
96:dd:12:3d:68:31:2c:1a:7f:41:a9:41:26:68:01:
14:b9:8f:50:e8:bd:e6:18:77:88:6c:08:41:8d:50:
c0:40:57:39:0b:d5:5e:df:17:78:d5:82:8f:29:be:
d1:09:d9:60:39:47:3a:70:00:1e:39:7c:9e:d6:32:
d6:88:7c:46:10:1a:ad:95:f6:22:25:04:8a:6d:80:
d9:96:94:59:17:08:25:4c:5b:5a:1e:27:3d:87:a8:
b3:3f:71:64:98:8a:22:6d:18:77:b7:05:c2:20:4f:
48:e4:fb:fb:88:9f:af:57:b2:b7:cd:d0:e2:14:64:
83:72:2c:db:d7:b2:e5:72:89:81:c9:89:7b:60:a2:
8e:19
Exponent: 65537 (0x10001)
Attributes:
Signature Algorithm: sha1WithRSA
Signature Value:
24:d1:c7:c2:e5:f4:22:d5:46:3c:f0:db:80:48:25:4f:6e:6d:
f2:06:6d:ad:15:63:30:d1:d4:3b:b9:1d:2b:3d:d0:19:9a:68:
2f:bb:d0:3d:e2:41:01:a2:6f:59:28:4e:39:5b:30:6f:31:78:
60:4a:32:fe:d4:71:ca:66:b3:b2:45:8c:f4:a7:3a:f2:0e:2c:
6b:61:d6:09:ee:0d:b5:32:a7:1e:5c:1e:17:f7:d9:58:43:32:
34:71:ca:4b:83:d6:0d:af:91:65:44:0a:53:6c:97:82:45:cd:
93:90:f4:4b:ff:46:04:e1:a0:24:d2:34:e2:73:6a:23:41:be:
3b:f6:32:8e:b0:0c:e3:a1:37:86:36:43:ba:d2:43:b2:7a:c2:
b2:e6:0a:23:e6:13:55:7a:c0:09:d0:f6:bb:c7:14:fd:01:0c:
f1:c4:f3:20:dc:e4:40:0e:a0:60:42:9d:d6:72:e1:c4:94:bd:
d2:32:d7:f1:3a:d0:00:e3:88:e1:a1:cd:7c:b1:74:31:51:67:
8a:b7:2c:a0:c4:59:13:e5:1a:98:5b:db:9b:36:e4:68:41:dc:
ea:56:3c:5c:eb:2d:30:30:5e:8d:e3:b1:97:7a:b8:46:5c:17:
ff:47:6c:47:6e:93:31:19:34:ce:19:b1:be:c6:c4:d5:a4:43:
78:a1:6f:9d
****** END X509_REQ ******
is it related to https://github.com/gridcf/gct/issues/153 ? And how to control which signature is used by delegation?
Eureka!
canl-java
have hard-coded sha1.
magic all debug options by globus (just in case someone will hit similar debug issue):
for i in * /lib64/libglobus_*; do strings $i | grep DEBUG_LEVEL; done | sort -u | awk '{ printf("%s=99 \\\n", $1); }'
GLOBUS_CALLOUT_DEBUG_LEVEL=99 \
GLOBUS_FTP_CLIENT_DEBUG_LEVEL=99 \
GLOBUS_FTP_CONTROL_DEBUG_LEVEL=99 \
GLOBUS_GSI_CALLBACK_DEBUG_LEVEL=99 \
GLOBUS_GSI_CERT_UTILS_DEBUG_LEVEL=99 \
GLOBUS_GSI_CRED_DEBUG_LEVEL=99 \
GLOBUS_GSI_GSS_ASSIST_DEBUG_LEVEL=99 \
GLOBUS_GSI_OPENSSL_ERROR_DEBUG_LEVEL=99 \
GLOBUS_GSI_PROXY_DEBUG_LEVEL=99 \
GLOBUS_GSI_SYSCONFIG_DEBUG_LEVEL=99 \
GLOBUS_GSSAPI_DEBUG_LEVEL=99 \
Eureka!
canl-java
have hard-coded sha1.
Nice find! I.e. in the end it's server side (dCache) and not a client side/GCT issue.
Disabling delegation by default still makes sense.
you probably have a point there. Probably only for 3rd party copies right? I'll reopen and see what the others have to say. Personally I think we should focus on really breaking issues or potential security issues for the GCT. @maarten-litmaath @fscheiner what do you think?
Well, we never found a real conclusion for https://github.com/gridcf/gct/issues/153 back then, other then that we might not have the manpower to fix non-breaking issues.
But I think we wouldn't categorically say no to patches that don't break defaults in this regard, right @msalle, @maarten-litmaath, @ellert ?
Another thought: For gsissh
delegation can be disabled AFAIR. So maybe there is a hidden switch for guc to do that, too? But I started to wonder if delegation is not already needed for split-process GridFTP server configurations. But still if it can be disabled with an additional switch it shouldn't hurt.
Sure, there is always a cost/benefit factor that should be taken into account. From dCache point of view, it's less than 1% of transfers that use gridftp. This issue has popped up in our internal testing, where we use all kinds of clients to avoid all kinds of regressions.
Unfortunately, GSI will stay longer with us. Thus, not requesting delegation by default will speed up the handshake and utilize less CPU resources on both the client and server sides.
Well, we never found a real conclusion for #153 back then, other then that we might not have the manpower to fix non-breaking issues.
ah great. Means we should probably close this one again...
But I think we wouldn't categorically say no to patches that don't break defaults in this regard, right @msalle, @maarten-litmaath, @ellert ?
no, I think that's fine. But actively patching it ourselves is probably not what we should be doing.
Another thought: For
gsissh
delegation can be disabled AFAIR. So maybe there is a hidden switch for guc to do that, too? But I started to wonder if delegation is not already needed for split-process GridFTP server configurations. But still if it can be disabled with an additional switch it shouldn't hurt.
For split process gridftp isn't it all between client and a single server? Not between server and a second server?
For split process gridftp isn't it all between client and a single server? Not between server and a second server?
Only if IP and DTP(s) are on the same host. If DTPs are on separate hosts, they need to have the means to authenticate the client connection somehow, I believe.
yes but that goes back to the original client which already has a proxy?
True. I think I need to do some more research here. :-)
In any case I'm reclosing this one, suggest we continue in #153 We can discuss if there are other scenarios than 3rd party copies that need delegated creds.
Dear maintainers,
during dCache testing with java-17 and almalinux9 (probably all REHL9 clones) clients, we discovered that globus-url-copy fails to establish GSS context. The same command works without issues on CentOS7. In both cases, we use
globus-url-copy
10.13. I would assume, that difference comes from openssl version (openssl-1.0.2k-26.el7_9.x86_64
vs.openssl-3.0.7-25.el9_3.x86_64
) or globus+openssl integration.Almalinux 9
CentOS 7.9