search

globus / globus-connect-server

Globus Connect Server
6 stars 10 forks source link

GCS configures wrong MyProxy CA cert when attempting to join second IO node to endpoint #7

Open danpowers opened 8 years ago

danpowers commented 8 years ago

GCS configures wrong MyProxy CA cert when attempting to join second IO node to endpoint configured to use MyProxy based auth.

Both nodes running CentOS 7 and GCS 4.0.36.

Both nodes are fresh instances of ami-1b4a032b, fully patched, and running fresh GCS installs.

Details for first node, configured to run GridFTP and MyProxy:

# grep -v "^$\|^;" /etc/globus-connect-server.conf
[Globus]
User = %(GLOBUS_USER)s
Password = %(GLOBUS_PASSWORD)s
[Endpoint]
Name = prod01
Public = True
DefaultDirectory = /~/
[Security]
FetchCredentialFromRelay = True
IdentityMethod = MyProxy
[GridFTP]
Server = %(HOSTNAME)s
RestrictPaths =
[MyProxy]
Server = %(HOSTNAME)s
[OAuth]

# cat /etc/gridftp.d/*
version_tag GCS-4.0.36
usage_stats_id GCS-4.0.36+centos-7.2.1511-64bit
port_range 50000,51000
data_interface 54.186.30.23
$GSI_AUTHZ_CONF "/etc/gridmap_verify_myproxy_callout-gsi_authz.conf"
$GRIDMAP "/etc/grid-security/grid-mapfile"
$GLOBUS_MYPROXY_CA_CERT "/var/lib/globus-connect-server/grid-security/certificates/6f1924ec.0"
$X509_USER_CERT "/var/lib/globus-connect-server/grid-security/hostcert.pem"
$X509_USER_KEY "/var/lib/globus-connect-server/grid-security/hostkey.pem"
log_single /var/log/gridftp.log
log_level ERROR,WARN
$X509_CERT_DIR "/var/lib/globus-connect-server/grid-security/certificates"

Note $GLOBUS_MYPROXY_CA_CERT "/var/lib/globus-connect-server/grid-security/certificates/6f1924ec.0". Here are the subject and issuer for that cert:

# openssl x509 -subject -issuer -noout -in /var/lib/globus-connect-server/grid-security/certificates/6f1924ec.0
subject= /C=US/O=Globus Consortium/OU=Globus Connect Service/CN=bf78630a-4ac8-11e6-8233-22000b97daec
issuer= /C=US/O=Globus Consortium/OU=Globus Connect Service/CN=bf78630a-4ac8-11e6-8233-22000b97daec

Details for second node, running GridFTP and configured to use first node for MyProxy services:

# grep -v "^$\|^;" /etc/globus-connect-server.conf
[Globus]
User = %(GLOBUS_USER)s
Password = %(GLOBUS_PASSWORD)s
[Endpoint]
Name = prod01
Public = True
DefaultDirectory = /~/
[Security]
FetchCredentialFromRelay = True
IdentityMethod = MyProxy
[GridFTP]
Server = %(HOSTNAME)s
RestrictPaths =
[MyProxy]
Server = ec2-54-186-30-23.us-west-2.compute.amazonaws.com
[OAuth]
# globus-connect-server-setup -v
Globus Id:  XXX
Password: 
ENTER: ID.setup()

...

ENTER: get_myproxy_ca_dn_from_server()
fetching myproxy ca dn from server
MyProxy CA DN is /C=US/O=Globus Consortium/CN=Globus Connect CA 3
EXIT: get_myproxy_ca_dn_from_server()
MyProxy CA DN is /C=US/O=Globus Consortium/CN=Globus Connect CA 3
CA dir is /var/lib/globus-connect-server/grid-security/certificates
Looking for MyProxy CA cert in /var/lib/globus-connect-server/grid-security/certificates
Checking to see if a059cd44.0 matches MyProxyDN
EXIT: configure_gridmap_verify_myproxy_callout()

...

Using Authentication Method MyProxy
Configured Endpoint prod01
EXIT: IO.setup()

Note MyProxy CA DN is /C=US/O=Globus Consortium/CN=Globus Connect CA 3 and Checking to see if a059cd44.0 matches MyProxyDN.

# cat /etc/gridftp.d/*
version_tag GCS-4.0.36
usage_stats_id GCS-4.0.36+centos-7.2.1511-64bit
port_range 50000,51000
data_interface 54.149.162.126
$GSI_AUTHZ_CONF "/etc/gridmap_verify_myproxy_callout-gsi_authz.conf"
$GRIDMAP "/etc/grid-security/grid-mapfile"
$GLOBUS_MYPROXY_CA_CERT "/var/lib/globus-connect-server/grid-security/certificates/a059cd44.0"
$X509_USER_CERT "/var/lib/globus-connect-server/grid-security/hostcert.pem"
$X509_USER_KEY "/var/lib/globus-connect-server/grid-security/hostkey.pem"
log_single /var/log/gridftp.log
log_level ERROR,WARN
$X509_CERT_DIR "/var/lib/globus-connect-server/grid-security/certificates"

Note $GLOBUS_MYPROXY_CA_CERT "/var/lib/globus-connect-server/grid-security/certificates/a059cd44.0"

Attempts to access node 2 now generate errors like this:

Command Failed: Error (login) Endpoint: XXX#prod01 (c206fda2-4ac8-11e6-8233-22000b97daec) Server: ec2-54-149-162-126.us-west-2.compute.amazonaws.com:2811 Message: Login Failed --- 530-Login incorrect. : globus_gss_assist: Error invoking callout\r\n530-globus_callout_module: The callout returned an error\r\n530-globus_gridmap_callout_error: Gridmap lookup failure: Could not map /C=US/O=Globus Consortium/OU=Globus Connect Service/CN=bf78630a-4ac8-11e6-8233-22000b97daec/CN=testuser\r\n530-\r\n530 End.\r\n

The /var/lib/globus-connect-server/grid-security/certificates/6f1924ec.0 cert exists on node 2, and pointing $GLOBUS_MYPROXY_CA_CERT at it and restarting GridFTP fixes the issue - e.g.:

# cat /etc/gridftp.d/*
version_tag GCS-4.0.36
usage_stats_id GCS-4.0.36+centos-7.2.1511-64bit
port_range 50000,51000
data_interface 54.149.162.126
$GSI_AUTHZ_CONF "/etc/gridmap_verify_myproxy_callout-gsi_authz.conf"
$GRIDMAP "/etc/grid-security/grid-mapfile"
$GLOBUS_MYPROXY_CA_CERT "/var/lib/globus-connect-server/grid-security/certificates/6f1924ec.0"
$X509_USER_CERT "/var/lib/globus-connect-server/grid-security/hostcert.pem"
$X509_USER_KEY "/var/lib/globus-connect-server/grid-security/hostkey.pem"
log_single /var/log/gridftp.log
log_level ERROR,WARN
$X509_CERT_DIR "/var/lib/globus-connect-server/grid-security/certificates"

# systemctl restart globus-gridftp-server.service

Node 2 is now properly accessible.

danpowers commented 7 years ago

See also:

https://globusonline.zendesk.com/agent/tickets/306668 https://globusonline.zendesk.com/agent/tickets/306716 https://globusonline.zendesk.com/agent/tickets/307515 https://globusonline.zendesk.com/agent/tickets/307523

timeu commented 7 years ago

We had the same issue. Is there any fix planed for this ?

bester commented 7 years ago

I've put a new version of gcs (4.0.44) in the unstable repos which attempts to address this issue. Can you please test this with your configuration?

danpowers commented 7 years ago

Testing with GCS 4.0.44 produces the same results as previously reported.

From the node configured to run both GridFTP and the MyProxy service:

# grep -v "^$\|^;" /etc/globus-connect-server.conf
[Globus]
User = %(GLOBUS_USER)s
Password = %(GLOBUS_PASSWORD)s
[Endpoint]
Name = gcs_4.0.44
Public = True
DefaultDirectory = /~/
[Security]
FetchCredentialFromRelay = True
IdentityMethod = MyProxy
[GridFTP]
Server = %(HOSTNAME)s
RestrictPaths =
[MyProxy]
Server = %(HOSTNAME)s
[OAuth]

# cat /etc/gridftp.d/*
version_tag GCS-4.0.44
usage_stats_id GCS-4.0.44+centos-7.3.1611-64bit
port_range 50000,51000
data_interface 54.190.49.91
$GSI_AUTHZ_CONF "/etc/gridmap_verify_myproxy_callout-gsi_authz.conf"
$GRIDMAP "/etc/grid-security/grid-mapfile"
$GLOBUS_MYPROXY_CA_CERT "/var/lib/globus-connect-server/grid-security/certificates/53f883b8.0"
$X509_USER_CERT "/var/lib/globus-connect-server/grid-security/hostcert.pem"
$X509_USER_KEY "/var/lib/globus-connect-server/grid-security/hostkey.pem"
log_single /var/log/gridftp.log
log_level ERROR,WARN
$X509_CERT_DIR "/var/lib/globus-connect-server/grid-security/certificates"

From the second node configured to run only GridFTP locally and to point at the first node for the MyProxy service:

# grep -v "^$\|^;" /etc/globus-connect-server.conf
[Globus]
User = %(GLOBUS_USER)s
Password = %(GLOBUS_PASSWORD)s
[Endpoint]
Name = gcs_4.0.44
Public = True
DefaultDirectory = /~/
[Security]
FetchCredentialFromRelay = True
IdentityMethod = MyProxy
[GridFTP]
Server = %(HOSTNAME)s
RestrictPaths =
[MyProxy]
Server = ec2-54-190-49-91.us-west-2.compute.amazonaws.com
[OAuth]

# globus-connect-server-setup -v
Globus Id:  XXX
Password: 
ENTER: ID.setup()

...

ENTER: GCMU.configure_trust_roots()
Fetching MyProxy CA trust roots
ENTER: get_myproxy_dn_from_server()
fetching myproxy dn from server
MyProxy DN is /C=US/O=Globus Consortium/OU=Globus Connect Service/CN=56fee4f4-3049-11e7-bcae-22000b9a448b
EXIT: get_myproxy_dn_from_server()
fetching trust roots from myproxy server at ec2-54-190-49-91.us-west-2.compute.amazonaws.com
expecting dn /C=US/O=Globus Consortium/OU=Globus Connect Service/CN=56fee4f4-3049-11e7-bcae-22000b9a448b
expecting to put them in /var/lib/globus-connect-server/grid-security/certificates
Trust roots have been installed in /var/lib/globus-connect-server/grid-security/certificates/.

Updating CA hashes in /var/lib/globus-connect-server/grid-security/certificates
Checking files..
Nothing to do
EXIT: GCMU.configure_trust_roots()
Removing old trust roots configuration link
EXIT: IO.configure_sharing()
ENTER: configure_gridmap_verify_myproxy_callout()
ENTER: get_myproxy_ca_dn_from_server()
fetching myproxy ca dn from server
MyProxy CA DN is /C=US/O=Globus Consortium/CN=Globus Connect CA 3
EXIT: get_myproxy_ca_dn_from_server()
MyProxy CA DN is /C=US/O=Globus Consortium/CN=Globus Connect CA 3
CA dir is /var/lib/globus-connect-server/grid-security/certificates
Looking for MyProxy CA cert in /var/lib/globus-connect-server/grid-security/certificates
Checking to see if a059cd44.0 matches MyProxyDN
EXIT: configure_gridmap_verify_myproxy_callout()

...

Using Authentication Method MyProxy
Configured Endpoint gcs_4.0.44
EXIT: IO.setup()

# cat /etc/gridftp.d/*
version_tag GCS-4.0.44
usage_stats_id GCS-4.0.44+centos-7.3.1611-64bit
port_range 50000,51000
data_interface 34.209.48.42
$GSI_AUTHZ_CONF "/etc/gridmap_verify_myproxy_callout-gsi_authz.conf"
$GRIDMAP "/etc/grid-security/grid-mapfile"
$GLOBUS_MYPROXY_CA_CERT "/var/lib/globus-connect-server/grid-security/certificates/a059cd44.0"
$X509_USER_CERT "/var/lib/globus-connect-server/grid-security/hostcert.pem"
$X509_USER_KEY "/var/lib/globus-connect-server/grid-security/hostkey.pem"
log_single /var/log/gridftp.log
log_level ERROR,WARN
$X509_CERT_DIR "/var/lib/globus-connect-server/grid-security/certificates"

# ls -lah /var/lib/globus-connect-server/grid-security/certificates
total 28K
drwxr-xr-x 2 root root 4.0K May  3 21:44 .
drwxr-xr-x 3 root root 4.0K May  3 21:44 ..
-rw-r--r-- 1 root root 2.0K May  3 21:44 53f883b8.0
-rw-r--r-- 1 root root 1.3K May  3 21:44 53f883b8.signing_policy
-rw-r--r-- 1 root root 1.3K May  3 21:44 a059cd44.0
-rw-r--r-- 1 root root  461 May  3 21:44 a059cd44.signing_policy
-rw-r--r-- 1 root root  350 May  3 21:44 myproxy-install-log

Just as before, the MyProxy CA cert seems to be getting copied from node 1 to node 2, but it doesn't seem to be getting configured in the $GLOBUS_MYPROXY_CA_CERT value.