search

rucio / containers

Containers for Rucio
Apache License 2.0
8 stars 54 forks source link

rucio-server container does not allow x509_proxy authentication #329

Open maxnoe opened 2 weeks ago

maxnoe commented 2 weeks ago

When trying to authenticate using an x509_proxy, the httpd responds with proxy certificates not allowed:

[Fri Jun 14 08:05:15.966306 2024] [ssl:info] [pid 34:tid 56] [client 172.21.0.8:34670] AH02276: Certificate Verification: Error (40): proxy certificates not allowed, please set the appropriate flag [subject: CN=1209913245,CN=DPPS User / issuer: CN=DPPS User / serial: 481DCF9D / notbefore: Jun 14 08:04:12 2024 GMT / notafter: Aug  4 23:04:12 2025 GMT]
bari12 commented 2 weeks ago

Did you enable RUCIO_HTTPD_GRID_SITE_ENABLED ?

maxnoe commented 2 weeks ago

No... even when checking the env variables the docker image uses I didn't make that connection...

Is that documented somewhere? What else does this option enable / change?

bari12 commented 2 weeks ago

I don't think it's documented, but we should perhaps be mentioning it here https://github.com/rucio/containers/tree/master/server

I am not 100% sure if this fixes the issue, but in principle you need to run mod_gridsite in apache to handle x509 proxies. So it is at least a prerequisite.

maxnoe commented 2 weeks ago

Trying it now....

maxnoe commented 2 weeks ago

Ok, one step further, I now get a different error:

[user@d242836a00af dpps-docker-compose]$ rm -rf /tmp/user/.rucio_user/
[user@d242836a00af dpps-docker-compose]$ RUCIO_ACCOUNT=root RUCIO_AUTH_TYPE=x509_proxy RUCIO_X509_CLIENT_PROXY=/tmp/x509up_u1000 rucio -vvv whoami
2024-06-14 14:59:02,018 DEBUG   baseclient.py   No trace_host passed. Using rucio_host instead
2024-06-14 14:59:02,019 DEBUG   baseclient.py   No creds passed. Trying to get it from the config file.
2024-06-14 14:59:02,019 DEBUG   baseclient.py   HTTPS is required, but no ca_cert was passed. Trying to get it from X509_CERT_DIR.
2024-06-14 14:59:02,019 DEBUG   baseclient.py   HTTPS is required, but no ca_cert was passed and X509_CERT_DIR is not defined. Trying to get it from the config file.
2024-06-14 14:59:02,019 DEBUG   baseclient.py   No account passed. Trying to get it from the RUCIO_ACCOUNT environment variable or the config file.
2024-06-14 14:59:02,019 DEBUG   baseclient.py   No VO passed. Trying to get it from environment variable RUCIO_VO.
2024-06-14 14:59:02,019 DEBUG   baseclient.py   No VO found. Trying to get it from the config file.
2024-06-14 14:59:02,019 DEBUG   baseclient.py   No VO found. Using default VO.
2024-06-14 14:59:02,019 DEBUG   baseclient.py   get a new token
2024-06-14 14:59:02,019 DEBUG   baseclient.py   HTTP request: GET https://rucio-server/auth/x509_proxy
2024-06-14 14:59:02,019 DEBUG   baseclient.py   HTTP header:  X-Rucio-Auth-Token: [hidden]
2024-06-14 14:59:02,019 DEBUG   baseclient.py   HTTP header:  X-Rucio-VO: def
2024-06-14 14:59:02,019 DEBUG   baseclient.py   HTTP header:  Connection: Keep-Alive
2024-06-14 14:59:02,020 DEBUG   baseclient.py   HTTP header:  User-Agent: rucio-clients/34.4.3
2024-06-14 14:59:02,020 DEBUG   baseclient.py   HTTP header:  X-Rucio-Script: rucio::-vvv
2024-06-14 14:59:02,020 DEBUG   baseclient.py   HTTP header:  X-Rucio-Account: root
2024-06-14 14:59:02,023 ERROR   ConnectionError: ('Connection aborted.', IsADirectoryError(21, 'Is a directory'))
2024-06-14 14:59:02,025 ERROR   ConnectionError: ('Connection aborted.', IsADirectoryError(21, 'Is a directory'))
2024-06-14 14:59:02,027 ERROR   ConnectionError: ('Connection aborted.', IsADirectoryError(21, 'Is a directory'))
2024-06-14 14:59:02,027 ERROR   Cannot connect to the Rucio server.
Completed in 0.0092 sec.
maxnoe commented 1 week ago

In case this might help narrow down the issue: it seems that via curl, the request goes through:

[user@5607e50d4eec dpps-docker-compose]$ curl -v -f \
    --cert /tmp/x509up_u1000 \
    --cacert /etc/grid-security/certificates/74df993b.0 \
    -H "X-Rucio-Account: root" \
    -H "X-Rucio-VO: def" https://rucio-server/auth/x509_proxy 
*   Trying 192.168.64.7:443...
* Connected to rucio-server (192.168.64.7) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/grid-security/certificates/74df993b.0
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=rucio-server
*  start date: Jun 13 16:49:55 2024 GMT
*  expire date: Feb  2 16:49:55 2049 GMT
*  subjectAltName: host "rucio-server" matched cert's "rucio-server"
*  issuer: CN=DPPS Development CA
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Unknown (23):
> GET /auth/x509_proxy HTTP/1.1
> Host: rucio-server
> User-Agent: curl/7.76.1
> Accept: */*
> X-Rucio-Account: root
> X-Rucio-VO: def
> 
* TLSv1.2 (IN), TLS header, Unknown (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Tue, 25 Jun 2024 08:47:10 GMT
< Server: Apache/2.4.57 (AlmaLinux) OpenSSL/3.0.7 mod_wsgi/4.7.1 Python/3.9 mod_gridsite/3.0.0
< Content-Length: 0
< Access-Control-Allow-Origin: None
< Access-Control-Allow-Headers: None
< Access-Control-Allow-Methods: *
< Access-Control-Allow-Credentials: true
< Access-Control-Expose-Headers: X-Rucio-Auth-Token, X-Rucio-Auth-Token-Expires, X-Rucio-Auth-Account, X-Rucio-Auth-Accounts
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< X-Rucio-Auth-Token: root-CN=DPPS User-unknown-9134466d1a09444b87e66e28c69167ab
< X-Rucio-Auth-Token-Expires: Tue, 25 Jun 2024 09:47:10 UTC
< X-Rucio-Auth-Account: root
< X-Rucio-Host: rucio-server
< Content-Type: application/octet-stream
< 
* Connection #0 to host rucio-server left intact
maxnoe commented 1 week ago

Debugging with changing log messages to show the full exceptions and adding some addtional logs, I found that for some reason, the client tries to use my current working directory as location for the client certificate:

2024-06-25 09:04:32,204 INFO    Using client certificate from path '/home/user'
2024-06-25 09:04:32,207 ERROR   ConnectionError: ('Connection aborted.', IsADirectoryError(21, 'Is a directory'))
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 418, in ssl_wrap_socket
    context.load_cert_chain(certfile, keyfile)
IsADirectoryError: [Errno 21] Is a directory
maxnoe commented 1 week ago

The reason for this is that my config, due to how the docker system is setup, contained an empty entry for the client proxy.

Which due to this code:

https://github.com/rucio/rucio/blob/0bbc069faf17f06eff1891aa90e634adea67b216/lib/rucio/client/baseclient.py#L292-L309

leads to the current working directory being inserted into the cert option

bari12 commented 1 week ago

Would you suggest to change something in this case? I am not sure how much this is a Rucio issue or how much this stems from your docker setup? :-)

bari12 commented 1 week ago

Ahh, should have read my next eMail first. I guess #6845 is the followup from this. Can we close this containers issue then?

maxnoe commented 1 week ago

A fix here could be to also not have the empty sections in the default config that gets merged