An exception when transferring files between some grid providers.

[thejkane]

My initial attempt was to transfer files between trestles and stampede. While trying to do this I got exception in [1]. File transfer is successful when I try to transfer files between two locations in trestles. But not between trestles and stampede.

Further debugging the issue we sought out that hash codes generated by Java and openssl is different. Further we figured out this behavior is common to in multiple signing_policy files.

Some of the files are;

e5cc84c2.signing_policy ef300431.signing_policy 01b5d333.signing_policy 081fefd0.signing_policy

In following I am pasting some code I used for testing;

public void testCertFileRead() throws Exception {

    GlobusResource globusResource1 = new GlobusResource("/Users/thejaka/development/apache/airavata/sandbox/grid-tools/gridftp-client/certificates/e5cc84c2.signing_policy");
    GlobusResource globusResource2 = new GlobusResource("/Users/thejaka/development/apache/airavata/sandbox/grid-tools/gridftp-client/certificates/ffc3d59b.signing_policy");
    GlobusResource globusResource3 = new GlobusResource("/Users/thejaka/development/apache/airavata/sandbox/grid-tools/gridftp-client/certificates/ef300431.signing_policy");
    GlobusResource globusResource4 = new GlobusResource("/Users/thejaka/development/apache/airavata/sandbox/grid-tools/gridftp-client/certificates/01b5d333.signing_policy");
    GlobusResource globusResource5 = new GlobusResource("/Users/thejaka/development/apache/airavata/sandbox/grid-tools/gridftp-client/certificates/081fefd0.signing_policy");
    //ResourceSigningPolicy resourceSigningPolicy = new ResourceSigningPolicy(globusResource);

    Assert.assertEquals(getHash(globusResource1), "e5cc84c2");
    Assert.assertEquals(getHash(globusResource2), "ffc3d59b");
    Assert.assertEquals(getHash(globusResource3), "ef300431");
    Assert.assertEquals(getHash(globusResource4), "01b5d333");
    Assert.assertEquals(getHash(globusResource5), "081fefd0");

}

private String getHash(GlobusResource globusResource) throws Exception {

    SigningPolicyParser parser = new SigningPolicyParser();

    Reader reader = new InputStreamReader(globusResource.getInputStream());

    Map<X500Principal, SigningPolicy> policies = parser.parse(reader);

    X500Principal principal = policies.keySet().iterator().next();

    System.out.println(principal.getName());

    return CertificateIOUtil.nameHash(principal);

}

Further debugging we figured out the encoding value returned by the certificate and the X500Principal (built using xxxx..signing_policy) is different.

For '/DC=EDU/DC=UTEXAS/DC=TACC/O=UT-AUSTIN/CN=TACC Classic CA' we see following encoding values.

From certificate -

[48,113,49,19,48,17,6,10,9,-110,38,-119,-109,-14,44,100,1,25,<<19>>,3,69,68,85,49,22,48,20,6,10,9,-110,38,-119,-109,-14,44,100,1,25,19,6,85,84,69,88,65,83,49,20,48,18,6,10,9,-110,38,-119,-109,-14,44,100,1,25,19,4,84,65,67,67,49,18,48,16,6,3,85,4,10,19,9,85,84,45,65,85,83,84,73,78,49,24,48,22,6,3,85,4,3,19,15,84,65,67,67,32,67,108,97,115,115,105,99,32,67,65]

From X500Principal -

[48,113,49,19,48,17,6,10,9,-110,38,-119,-109,-14,44,100,1,25,<<22>>,3,69,68,85,49,22,48,20,6,10,9,-110,38,-119,-109,-14,44,100,1,25,22,6,85,84,69,88,65,83,49,20,48,18,6,10,9,-110,38,-119,-109,-14,44,100,1,25,22,4,84,65,67,67,49,18,48,16,6,3,85,4,10,19,9,85,84,45,65,85,83,84,73,78,49,24,48,22,6,3,85,4,3,19,15,84,65,67,67,32,67,108,97,115,115,105,99,32,67,65]

Notice the 19th location. (19 and 22). There are few more differences like that. Therefore hash values generated are different.

Thanks Amila

[1] testTransferData(org.apache.airavata.filetransfer.FileTransferTest) Time elapsed: 1.232 sec <<< ERROR! java.lang.Exception: Cannot transfer file from GridFTP:gsiftp://trestles-dm.sdsc.xsede.org:2811//oasis/projects/nsf/sds128/ogce/file-transfer-tests/source/sample_wrfout.netcdf to gsiftp://gridftp.stampede.tacc.utexas.edu:2811//scratch/01437/ogce/file-transfer-tests/dest/xx at org.apache.airavata.filetransfer.utils.GridFtp.transfer(GridFtp.java:356) at org.apache.airavata.filetransfer.FileTransfer.transferData(FileTransfer.java:42) at org.apache.airavata.filetransfer.FileTransferTest.testTransferData(FileTransferTest.java:95) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74) Caused by: org.globus.common.ChainedIOException: Authentication failed [Caused by: Path validation failed. No signing policy for CN=TACC Classic CA, O=UT-AUSTIN, DC=TACC, DC=UTEXAS, DC=EDU] at org.globus.ftp.extended.GridFTPControlChannel.authenticate(GridFTPControlChannel.java:221) at org.globus.ftp.GridFTPClient.authenticate(GridFTPClient.java:127) at org.globus.ftp.GridFTPClient.authenticate(GridFTPClient.java:103) at org.apache.airavata.filetransfer.utils.GridFtp.transfer(GridFtp.java:322) ... 27 more Caused by: GSSException: Path validation failed. No signing policy for CN=TACC Classic CA, O=UT-AUSTIN, DC=TACC, DC=UTEXAS, DC=EDU at org.globus.gsi.gssapi.GlobusGSSContextImpl.initSecContext(GlobusGSSContextImpl.java:1148) at org.globus.ftp.extended.GridFTPControlChannel.authenticate(GridFTPControlChannel.java:209) ... 30 more Caused by: GSSException: Path validation failed. No signing policy for CN=TACC Classic CA, O=UT-AUSTIN, DC=TACC, DC=UTEXAS, DC=EDU at org.globus.gsi.gssapi.GlobusGSSContextImpl.sslProcessHandshake(GlobusGSSContextImpl.java:933) at org.globus.gsi.gssapi.GlobusGSSContextImpl.initSecContext(GlobusGSSContextImpl.java:1061) ... 31 more Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem at com.sun.net.ssl.internal.ssl.Handshaker.checkThrown(Handshaker.java:1015) at com.sun.net.ssl.internal.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:485) at com.sun.net.ssl.internal.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1108) at com.sun.net.ssl.internal.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1080) at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:452) at org.globus.gsi.gssapi.GlobusGSSContextImpl.sslProcessHandshake(GlobusGSSContextImpl.java:864) ... 32 more Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174) at com.sun.net.ssl.internal.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1508) at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:243) at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235) at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1209) at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:135) at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593) at com.sun.net.ssl.internal.ssl.Handshaker$1.run(Handshaker.java:533) at java.security.AccessController.doPrivileged(Native Method) at com.sun.net.ssl.internal.ssl.Handshaker$DelegatedTask.run(Handshaker.java:952) at org.globus.gsi.gssapi.GlobusGSSContextImpl.runDelegatedTasks(GlobusGSSContextImpl.java:412) at org.globus.gsi.gssapi.GlobusGSSContextImpl.sslProcessHandshake(GlobusGSSContextImpl.java:902) ... 32 more Caused by: java.security.cert.CertificateException: Path validation failed. No signing policy for CN=TACC Classic CA, O=UT-AUSTIN, DC=TACC, DC=UTEXAS, DC=EDU at org.globus.gsi.trustmanager.PKITrustManager.checkServerTrusted(PKITrustManager.java:115) at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1201) ... 39 more Caused by: java.security.cert.CertPathValidatorException: No signing policy for CN=TACC Classic CA, O=UT-AUSTIN, DC=TACC, DC=UTEXAS, DC=EDU at org.globus.gsi.trustmanager.SigningPolicyChecker.invoke(SigningPolicyChecker.java:61) at org.globus.gsi.trustmanager.X509ProxyCertPathValidator.checkCertificate(X509ProxyCertPathValidator.java:466) at org.globus.gsi.trustmanager.X509ProxyCertPathValidator.validate(X509ProxyCertPathValidator.java:172) at org.globus.gsi.trustmanager.X509ProxyCertPathValidator.engineValidate(X509ProxyCertPathValidator.java:111) at org.globus.gsi.trustmanager.PKITrustManager.checkServerTrusted(PKITrustManager.java:113) ... 40 more

[thejkane]

Further details; When encoding a X500Principal it encodes (DER) each of the components in it. (i.e. CN, DC, O etc ...) When we encode a principal extracted from a certificate, the type of DC is represented as a "printable string". But when we encode, X500Principal instantiated as in org.globus.gsi.util.CertificateUtil, we get "ASN1String" as the type of DC component.

Due to the type (tag) differences in DC components we get 2 different encoding types. Apparently this issue occurs only when there are DC components in the domain name.

[bbockelm]

Hi,

Can you contribute a test case for this? It'll be easier to debug based on that...

I'm suspicious this is only due to "DC=" components as many CAs have such components. There must be another factor here...

Brian

[thejkane]

Hi Brian,

Please find the attached test case. Please ignore the commented code. Also please change CERT_FILE_LOCATION to a suitable certificate directory. It seems like we cannot generate encoding for DN as different elements (such as DC's) can have different types (printable string, IA5String, UTF etc ...).

I was unable to attach test case to GIT issue. By any chance if attachment is not delivered to you please let me know.

Thank you Regards, Thejaka Amila

On Tue, Jun 11, 2013 at 8:53 AM, Brian Bockelman notifications@github.comwrote:

Hi,

Can you contribute a test case for this? It'll be easier to debug based on that...

I'm suspicious this is only due to "DC=" components as many CAs have such components. There must be another factor here...

Brian

— Reply to this email directly or view it on GitHubhttps://github.com/jglobus/JGlobus/issues/102#issuecomment-19259548 .

[bbockelm]

Hi Amila,

Can you send it as a pull request for the test? Github documentation is here:

https://help.github.com/articles/using-pull-requests

That's the preferred way of sharing code in git as opposed to attachments...

Brian

[smarru]

Amila, you will need to first "Fork" the repository to your space and switch to your repo and then create a pull request. -- Suresh

[smarru]

The certificates in question (ffc3d59b, e5cc84c2) are bundled in - https://software.xsede.org/security/xsede-certs.tar.gz

[thejkane]

Hi Brian,

The test case I created is outside of jglobus source code. Let me move test to jglobus and submit it through a pull request.

Thank you Regards, Amila

On Tue, Jun 11, 2013 at 9:31 AM, Brian Bockelman notifications@github.comwrote:

Hi Amila,

Can you send it as a pull request for the test? Github documentation is here:

https://help.github.com/articles/using-pull-requests

That's the preferred way of sharing code in git as opposed to attachments...

Brian

— Reply to this email directly or view it on GitHubhttps://github.com/jglobus/JGlobus/issues/102#issuecomment-19261576 .

[thejkane]

Hi Brian,

I added the patch and test case in pull request [1]. Please review the pull request and let me know if anything needs to be changed.

[1] https://github.com/jglobus/JGlobus/pull/103

Thank you Thejaka Amila

On Tue, Jun 11, 2013 at 9:50 AM, Amila Jayasekara thejaka.amila@gmail.comwrote:

Hi Brian,

The test case I created is outside of jglobus source code. Let me move test to jglobus and submit it through a pull request.

Thank you Regards, Amila

On Tue, Jun 11, 2013 at 9:31 AM, Brian Bockelman <notifications@github.com

wrote:

Hi Amila,

Can you send it as a pull request for the test? Github documentation is here:

https://help.github.com/articles/using-pull-requests

That's the preferred way of sharing code in git as opposed to attachments...

Brian

— Reply to this email directly or view it on GitHubhttps://github.com/jglobus/JGlobus/issues/102#issuecomment-19261576 .

[smarru]

Amila, I also cherry picked the commits from this PR into 2.0.6 branch.