search

knowm / XChange

XChange is a Java library providing a streamlined API for interacting with 60+ Bitcoin and Altcoin exchanges providing a consistent interface for trading and accessing market data.
http://knowm.org/open-source/xchange/
MIT License
3.87k stars 1.94k forks source link

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: #52

Closed timmolter closed 9 years ago

timmolter commented 11 years ago

I received the following Email:

I thought I would try out the Canadian VirtEx exchange ticker so I could setup some auto arbitrage alerts as well, however I'm getting the following exceptions (xchange-cavirtex-1.4.1-20130201.151127-4.jar):

Exchange virtEx = ExchangeFactory.INSTANCE.createExchange(VirtExExchange.class.getName()); mMarketDataVirtEx = virtEx.getPollingMarketDataService(); Ticker ticker = mMarketDataVirtEx.getTicker(Currencies.BTC, Currencies.CAD);

com.xeiam.xchange.rest.HttpException: Problem GETing (IO) at com.xeiam.xchange.rest.HttpTemplate.executeRequest(HttpTemplate.java:197) at com.xeiam.xchange.rest.HttpTemplate.executeRequest(HttpTemplate.java:97) at com.xeiam.xchange.rest.RestInvocationHandler.invokeHttp(RestInvocationHandler.java:61) at com.xeiam.xchange.rest.RestInvocationHandler.invoke(RestInvocationHandler.java:56) at sun.proxy.$Proxy13.getTicker(Unknown Source) at com.xeiam.xchange.virtex.service.marketdata.polling.VirtExPollingMarketDataService.getTicker(VirtExPollingMarketDataService.java:73) at main.Main.main(Main.java:147) Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.lang.reflect.Constructor.newInstance(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at sun.net.www.protocol.http.HttpURLConnection.getChainedException(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source) at com.xeiam.xchange.rest.HttpTemplate.executeRequest(HttpTemplate.java:186) ... 6 more Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Unknown Source) at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source) at sun.security.ssl.Handshaker.fatalSE(Unknown Source) at sun.security.ssl.Handshaker.fatalSE(Unknown Source) at sun.security.ssl.ClientHandshaker.serverCertificate(Unknown Source) at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source) at sun.security.ssl.Handshaker.processLoop(Unknown Source) at sun.security.ssl.Handshaker.process_record(Unknown Source) at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source) at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(Unknown Source) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getHeaderField(Unknown Source) at com.xeiam.xchange.rest.HttpTemplate.getResponseEncoding(HttpTemplate.java:264) at com.xeiam.xchange.rest.HttpTemplate.executeRequest(HttpTemplate.java:185) ... 6 more Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.validator.PKIXValidator.doBuild(Unknown Source) at sun.security.validator.PKIXValidator.engineValidate(Unknown Source) at sun.security.validator.Validator.validate(Unknown Source) at sun.security.ssl.X509TrustManagerImpl.validate(Unknown Source) at sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source) at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source) ... 21 more Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(Unknown Source) at java.security.cert.CertPathBuilder.build(Unknown Source) ... 27 more

timmolter commented 11 years ago

After a bit of Googling about this exception, I was let to these pages:

1) http://stackoverflow.com/questions/9619030/resolving-javax-net-ssl-sslhandshakeexception-sun-security-validator-validatore 2) http://www.jyothis.co.in/2011/11/12/javax-net-ssl-sslhandshakeexception/

It looks like this is out of the control of XChange. Can you try updating your JRE? What version do you have? I'm running: 1.6.0_37, and I don't get this error.

I also tried changing the virtex URL to HTTP from HTTPS, but that caused another problem, so we can't simply just make that change.

mmazi commented 11 years ago

This is the same error I mentioned in #24 and #31.

DougTanner commented 11 years ago

I'm using JRE 1.7.0_07, but I just downloaded and tested it with JDK/JRE 1.6.0.0_39. Same problem.

gary-rowe commented 11 years ago

It looks like CAVirtex is using a certificate signed by a root authority that is not present in the standard JRE distribution which is why it is getting rejected. In order to overcome this we'll need to provide a facility to allow users to add custom certificates into their JVMs via JSSE (requires write access to their JVM).

This is easily achieved (for a developer) by using the InstallCert.java code from MultiBit Merchant. We'd need to wrap it into a suitable API to make it a bit more friendly but it could be done.

timmolter commented 11 years ago

And here we go again, with Mt Gox now. From Jim:

"One of my users is not seeing the ticker information and in the logs is the error:

mtGox exchange = com.xeiam.xchange.mtgox.v1.MtGoxExchange@199de59 12:00:52.625 [Timer-22] ERROR o.multibit.exchange.TickerTimerTask - com.xeiam.xchange.HttpException Problem GETing (IO) 12:00:52.625 [Timer-22] ERROR o.multibit.exchange.TickerTimerTask - javax.net.ssl.SSLHandshakeException sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target This is for MtGox."

jim618 commented 11 years ago

The user that raised the bug with MultiBit found a workaround. He had two versions of MB on different machines - one worked, one didn't.

The difference was te Java version:

Windows 7 had Java 7 Update 10. (Worked ok). Windows XP had Java 6 Update 13. (Failed).

He updated Java on Windows XP. Now XP uses Java 7 Update 13 and now exchange ticker works correctly.

timmolter commented 11 years ago

From Michael (Bitcoinium): Android 2.2 is giving me a lot of problems. Some exchanges are flaky with previous versions of XChange on Android 2.2 but the latest SNAPSHOTs cause BTC-E, Bitstamp and Bitcoin24 to throw an invalid certificate exception at every single request.

Do I understand the only solution correctly? If the JRE doesn't have the certificate, then the user needs to manually obtain it and install it somehow?

timmolter commented 11 years ago

@gary-rowe "In order to overcome this we'll need to provide a facility to allow users to add custom certificates into their JVMs via JSSE (requires write access to their JVM)."

Do you think there may be a better way to do this, perhaps automagically? Manually adding a certificate on Android is probably also not very easy to do. Also, where would they get the certificates? From us? Or from the exchanges?

@veken0m Is there an app to let Android users install SSL certificates?

I'm trying to figure out the best solution and would like your feedback. Thanks!

timmolter commented 11 years ago

@veken0m See: http://nelenkov.blogspot.de/2011/12/ics-trust-store-implementation.html

To sum things up: users can now freely install and remove private keys and trusted certificates, as well as disable pre-installed CA certificates via the Settings app.

The article talks about these issues. I wonder if you can somehow catch this exception in your code, and have a pop-up appear that let's the user manually accept the CA certificate.

My conclusion to all of this is that XChange is not at "fault" here, and XChange isn't the appropriate place to manage CA certificates. I suppose XChange could add code to "bypass" the validation of certificates, that would be one option that I found, but that would leave a gaping security hole, and when dealing with people's money, that's a bad idea.

Please let me know what you think. I'm out of ideas.

veken0m commented 11 years ago

@timmolter

I've tried to manually add certificates to the TrustManager before but was not able to get things working. The Trust Store API you are referring to would be a great solution but it only seems to be available in Android 4.0+ while Android 2.2 is the one affected...

As for bypassing the validation, if you give the option to do this I could prompt the user when the certificate is invalid and ask if he wants to continue anyways, the Android 2.2 browser currently does this when accessing the affected websites. (proof that it's Android's issue and not XChange...)

I'll leave this up to you and I understand if you don't want to bypass validation of certificates for obvious security reasons. I am considering dropping support for Android 2.2 because only a very small percentage of Bitcoinium users use it and it's been a real pain to support it.

I appreciate the time you took to look into this issue. Thank you!

timmolter commented 11 years ago

I believe the only solution is to let the end user manage their CA certificates. For Android users, that means using the Trust Store. If they are using Android 3.x or less, it won't be so easy for them. For people running their own trader or MultiBit for example, they will need to manually run something like the code here. In most cases it seems like having the most up to date JRE solves this problem, but there are apparently some exceptions to this rule as Doug reported.

I'm still open to discuss this, but until a definitive answer comes to light, it's hard to know exactly what to do, and where to do it. I'd like to get V1.5.0 out by tomorrow, and a "fix" for this issue will unfortunately not be included.

timmolter commented 11 years ago

I'm going to assume everyone is OK with my conclusion, and I'm closing this. We can reopen it if we think there is a "fix" for this that can be integrated in to XChange.

sideburnie commented 11 years ago

I have that exact problem right now:

com.xeiam.xchange.examples.mtgox.v1.service.marketdata.polling.MtGoxMarketdataDemo 17:48:47.224 [default] [main] DEBUG com.xeiam.xchange.ExchangeFactory - Creating default exchange from class name 17:48:47.434 [default] [main] DEBUG com.xeiam.xchange.rest.HttpTemplate - Executing GET request at https://data.mtgox.com/api/1/BTCUSD/ticker?raw Exception in thread "main" com.xeiam.xchange.rest.HttpException: Problem GETing (IO) at com.xeiam.xchange.rest.HttpTemplate.executeRequest(HttpTemplate.java:198) at com.xeiam.xchange.rest.HttpTemplate.executeRequest(HttpTemplate.java:97) at com.xeiam.xchange.rest.RestInvocationHandler.invokeHttp(RestInvocationHandler.java:61) at com.xeiam.xchange.rest.RestInvocationHandler.invoke(RestInvocationHandler.java:56) at com.sun.proxy.$Proxy7.getTicker(Unknown Source) at com.xeiam.xchange.mtgox.v1.service.marketdata.polling.MtGoxPollingMarketDataService.getTicker(MtGoxPollingMarketDataService.java:72) at com.xeiam.xchange.examples.mtgox.v1.service.marketdata.polling.MtGoxMarketdataDemo.main(MtGoxMarketdataDemo.java:57) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) at sun.security.ssl.Handshaker.process_record(Handshaker.java:804) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323) at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:515) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1299) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338) at com.xeiam.xchange.rest.HttpTemplate.checkHttpStatusCode(HttpTemplate.java:212) at com.xeiam.xchange.rest.HttpTemplate.executeRequest(HttpTemplate.java:186) ... 11 more Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) at sun.security.validator.Validator.validate(Validator.java:260) at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231) at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:126) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1323) ... 25 more Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) ... 31 more

Process finished with exit code 1

I have added the mtgox cert (below) to /usr/lib/jvm/jdk1.7.0_17/jre/lib/security (using keytool) as well as adding the .pem file to /etc/ssl/certs and the .crt file to /usr/local/share/ca-certificates and running sudo update-ca-certificates... no love.

Alias name: mtgox.com Creation date: Apr 5, 2013 Entry type: trustedCertEntry

Owner: CN=mtgox.com, OU=Terms of use at www.verisign.com/rpa (c)05, O=K.K. Tibanne, L=Suginami, ST=Tokyo, C=JP, SERIALNUMBER=0110-01-069784, OID.2.5.4.15=Private Organization, OID.1.3.6.1.4.1.311.60.2.1.3=JP Issuer: CN=VeriSign Class 3 Extended Validation SSL SGC CA, OU=Terms of use at https://www.verisign.com/rpa (c)06, OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US Serial number: 205f724508592b0da9e6dcc3a63b7d16 Valid from: Tue Aug 30 17:00:00 PDT 2011 until: Fri Aug 30 16:59:59 PDT 2013 Certificate fingerprints: MD5: E8:C1:08:6E:98:92:6B:D8:8F:76:58:54:0D:95:E7:77 SHA1: 02:B3:83:77:92:32:5B:49:F4:FD:20:A5:3B:16:D7:5A:6F:AA:F9:38 Signature algorithm name: SHA1withRSA Version: 3

Extensions:

1: ObjectId: 1.3.6.1.5.5.7.1.1 Criticality=false

AuthorityInfoAccess [ [ accessMethod: ocsp accessLocation: URIName: http://ocsp.verisign.com, accessMethod: caIssuers accessLocation: URIName: http://EVIntl-aia.verisign.com/EVIntl2006.cer] ]

2: ObjectId: 1.3.6.1.5.5.7.1.12 Criticality=false

3: ObjectId: 2.5.29.15 Criticality=false

KeyUsage [ DigitalSignature Key_Encipherment ]

4: ObjectId: 2.5.29.19 Criticality=false

BasicConstraints:[ CA:false PathLen: undefined ]

5: ObjectId: 2.5.29.31 Criticality=false

CRLDistributionPoints [ [DistributionPoint: [URIName: http://EVIntl-crl.verisign.com/EVIntl2006.crl] ]]

6: ObjectId: 2.5.29.32 Criticality=false

CertificatePolicies [ [CertificatePolicyId: [2.16.840.1.113733.1.7.23.6] [PolicyQualifierInfo: [ qualifierID: 1.3.6.1.5.5.7.2.1 qualifier: 0000: 16 1C 68 74 74 70 73 3A 2F 2F 77 77 77 2E 76 65 ..https://www.ve 0010: 72 69 73 69 67 6E 2E 63 6F 6D 2F 63 70 73 risign.com/cps

]] ] ]

7: ObjectId: 2.5.29.35 Criticality=false

AuthorityKeyIdentifier [ KeyIdentifier [ 0000: 4E 43 C8 1D 76 EF 37 53 7A 4F F2 58 6F 94 F3 38 NC..v.7SzO.Xo..8 0010: E2 D5 BD DF .... ]

]

8: ObjectId: 2.5.29.37 Criticality=false

ExtendedKeyUsages [ serverAuth clientAuth 2.16.840.1.113730.4.1 ]

Any ideas? Guess I'll try jdk 1.6?.... the overrides should be compiling and running for 1.6 though no?

sideburnie commented 11 years ago

Somehow I get no SSL error on a version of MtGoxMarketdataDemo.java on a version of the code that I pulled down from github on March 16, but I get the above SSL error on the version I pulled down today....

sideburnie commented 11 years ago

WOW (smacks head)..... so, in your old version you query mtgox.com... which has a cert chain. in the new version you're querying data.mtgox.com which only does plaintext... I'm assuming that you'll be changing all the query and websocket code to plaintext?

java -cp . InstallCert data.mtgox.com:80 changeit Loading KeyStore /usr/lib/jvm/jdk1.7.0_17/jre/lib/security/jssecacerts... Opening connection to data.mtgox.com:80...

Starting SSL handshake...

PKIX errors detected. You need to import the certificate

javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection? at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:672) at sun.security.ssl.InputRecord.read(InputRecord.java:504) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:883) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1208) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1235) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1219) at InstallCert.main(InstallCert.java:68) Could not obtain server certificate chain

timmolter commented 11 years ago

@sideburnie Thanks for sleuthing this out for us. MtGox loves to change things unannounced and they have very poor API docs. We found out about the url change from mtgox... to data.mtgox... after people's apps suddenly stopped working. Only until you reported this, did we also realize that the market data is now over http and not https. I fixed this today in this commit: https://github.com/timmolter/XChange/commit/8b7a9c42ef017b12168bc223c57bf5f23bd11d47

Let me know if it still doesn't work...

sideburnie commented 11 years ago

Cool, thanks Tim. I'll pull it down and give it a try.

sideburnie commented 11 years ago

Everything's working. Thanks a lot Tim. I got some basic active plotting of depth (area plots) and ticker (line plots) working today. I started out plotting with XChart (wow, super simple calls, btw, nice), but was unable to figure out how to get the chart to repaint without what seemed like might be a lot of digging... thought I'd try JFreeChart as from what I was reading the chartPanels have a listener for changes to the underlying data... it works, but honestly it almost seems like there might be a bug in their paint code... the XChart plots look nicer... Maybe I was just missing something simple for doing a repaint in XChart after updating the underlying data? If you have something on the tip of your fingers, can you link me? If it's any non-trivial amount of time, don't worry about it, I'll keep digging.

timmolter commented 11 years ago

@sideburnie I wish I could say XChart has real-time updates right now, but it's still a feature request: https://github.com/timmolter/XChart/issues/25

I wish I had more time to implement it, but I'm working on too many other projects at the moment. I remember doing real-time charts with JFreeChart years ago and it was a huge pain. Back then was when I got the idea to create XChart actually, because JFreeChart was so not user friendly. Try out Flot is you are doing a webapp.

sideburnie commented 11 years ago

Found JChart2d. It's significantly more polished than jfree. Are you planning on implementing logging / playback functionality? I was thinking that's what I might do next (as I now have depth / ticker plotting working but would like to look at historical data in fast forward).

mmazi commented 11 years ago

I've looked into this briefly.

Here's my understanding of the problem:

I've checked this (at least generally) by checking out my JRE's trust store: keytool -list -keystore /usr/lib/jvm//jdk-1.7.0/jre/lib/security/cacerts

What I gather from this is that the solution is to somehow include StartCom certificate into the trust store used for creating the https connection. (As Tim said, "solving" this by ignoring the server certificate is probably not a good idea when dealing with people's money.)

This can be done in (at least) two ways:

(A) system-wide, using a command-line tool (haven't tried it though), or by other means on android; or

(B) Another option is to include the StartCom certificate somewhere in the XChange jars and use it to create https connections (as in https://github.com/gary-rowe/MultiBitMerchant/blob/develop/mbm/src/test/java/InstallCert.java). So, rescu would have to provide a facility for it's users (eg. XChange) to provide certificates (eg. rescu could scan the si.mazi.rescu.certs package in the classpath for any certificate files and include them if necessary).

One problem with (B) approach is that it won't work on older versions of Android, if I understand @veken0m's comments correctly.

If the certificate that needs to be included were issued by MtGox (eg. if MtGox used a self-signed certificate), I'd go for (B) since this probably shouldn't be system wide and also it's in the domain of XChange and not the user. But the certificate is StartCom's and it's already trusted by all browsers (that's why your browser doesn't complain when you visit https://data.mtgox.com or https://cavirtex.com), I think the correct solution is (A). Actually, I'd expect future versions of Java to include this certificate in their trust store at installation so this step won't even be necessary.

So maybe I just repeated what others have said before me :), but at least it's now quite clear in my head, and maybe this re-cap will help somebody else.

mmazi commented 11 years ago

Oh, if we go with (A), I think the StartCom certificate and installation instructions should be distributed with XChange. (Sorry, accidental issue close..)

jim618 commented 11 years ago

I think the MtGox certificate should be used but it would be better not to have the user run a command line operation which is a bit clunky.

There are a couple of other options available:

1) You can set up a hosts verifier for HTTPS and inspect the SSLSession and programmatically verify the MtGox certificate:

http://www.rgagnon.com/javadetails/java-fix-certificate-problem-in-HTTPS.html

2) You can specify a truststore that has the top level authority(StartSSL cert in this case) and use that trust store for MtGox:

http://stackoverflow.com/questions/7219989/java-and-ssl-certificates

This is the relevant comment:

Or, you could initialize your application with your own truststore using the -Djavax.net.ssl.trustStore= -Djavax.net.ssl.trustStorePassword= JVM startup flags, or execute the following code before initializing SSL connections:

System.setProperty("javax.net.ssl.trustStore",""); System.setProperty("javax.net.ssl.trustStorePassword","");

timmolter commented 11 years ago

Thanks for your comments, guys. I'll be able to look into this tomorrow.

mmazi commented 11 years ago

@jim618, I'm curious: why do you think it's better to import the MtGox certificate, as opposed to importing the StartCom CA certificate with which the MtGox certificate (and CaVirtEx, and possibly others) is signed? The effect would be the same: the MtGox certificate would be accepted as trusted, plus it would fix the problem with CaVirtEx's and all other potential certificates signed by StartCom.

Since Firefox, Opera, Microsoft Windows and OS X all ship with the StartCom certificate in their trust stores, I'd expect it to be trustworthy?

gary-rowe commented 11 years ago

I would go with option A) and make a nice easy interface onto the utility available through client applications. This is a JVM problem, not an XChange one. Essentially we are pushing the problem further downstream until a user interface can handle it.

We can offer guidance to application developers who encounter this in the form of a JVM compatibility table and a set of instructions of how to get the Startcom root certificate into the trust store. Normally this takes admin privileges and is something that could be done at application install time.

timmolter commented 11 years ago

@mmazi What password do I need to use for this to work? keytool -list -keystore /usr/lib/jvm//jdk-1.7.0/jre/lib/security/cacerts It tells me "invalid password".

Thanks!

timmolter commented 11 years ago

@mmazi nevermind. The answer is "no password". Just hit enter when it asks.

timmolter commented 11 years ago

OK. I reviewed all the info posted here, and I did some of my own research on the subject. Below I comment on the options.

A) Add StartCom cert to system-wide trust store using a command-line tool (link) or a custom Java utility program. - This option is horrible for application end users such as MB and Bitcoinium. For devs it's fine.

B) Include the StartCom certificate somewhere in the XChange jars (link) We'd just have to create a custom truststore (link) and load it dynamically during runtime from the classpath or from a known path on the system. - All end users would be happiest. More work for XChange. How to do this exactly is not clear.

C) Initialize your application with your own truststore containing StartCom cert (link - We'd still have to generate a trust store and have users install it somewhere on their machine. Using the sys properties and/or JVM startup flags doesn't let you link to a trust store on the classpath. Yuk!

D) Somehow convince all JVM providers to include the StartCom certificate in their next release. (Not gonna happen.)

E) ask the affected exchanges to change their CA certificate to one that is accepted more universally

StartCom certs are here.

My conclusion: Option A and C are fine for devs. Option B is best for end-user applications like MB and Bitcoinium, more work for XChange, and I'm not sure 100% possible to pull off.

veken0m commented 11 years ago

The StartCom CA has been included in Android since 2.2 so this is not an issue on my end. The only exchange that's causing problems for me at the moment is BTC-E on Android 2.2 (and that might just be an issue with the Android emulator).

Now I'm not really knowledgeable on the subject but would it be an option to ask the affected exchanges to change their CA certificate to one that is accepted more universally?

jim618 commented 11 years ago

Nice summary.

For option C is it possible to install the trust store at installation time and refer to it's location as a relative or absolute path, rather than classloading ? In MultiBit I work out the directories already. That might be simpler than dynamic class loading.

timmolter commented 11 years ago

@jim618 Yes, if you know where the trust store is on the filesystem, you could load it via B) or C).

(Note: I edited B) above to incluse the option of loading it from a known path on the system.)

timmolter commented 11 years ago

@veken0m That's also a good idea. I'll add that to the list above.

jim618 commented 11 years ago

@Mazi. You are right - better to import the parent cert rather than the child.

KaTXi commented 11 years ago

For App developers there is an easy solution that doesn't involve importing cert, etc: Ignore CERTs. I know, I know, you can't use that for a comercial software that you maintain and have customers, but if you just have a private trading bot or a website that retrieves data from Gox and DOES NOT allow trading it works. Just put this at the initialization of your App:

TrustManager[] trustAllCerts = new TrustManager[1];

trustAllCerts[0] = new TrustManager(); try { SSLContext sc = SSLContext.getInstance("SSL"); sc.init(null, trustAllCerts, new java.security.SecureRandom()); HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory()); } catch (Exception e) { }

Don't forget to:

import javax.net.ssl.HttpsURLConnection; import javax.net.ssl.SSLContext;

gary-rowe commented 11 years ago

@KaTXi While that is a workaround, I'd be surprised if that is allowed into XChange.

Trust all certs is basically an open invitation to allow MITM attacks and therefore you can no longer trust the data feed or XChange. That's the reason it's being served over HTTPS in the first place.

KaTXi commented 11 years ago

I have a very nice SWT App in front of me printing streaming live data like crazy, using this workarround and com.xeiam.xchange.mtgox.v2 ... Using 1.7.0-SNAPSHOT by the way. I was using 1.6.0, updated today to incorporate streaming data and started having this problem even for non streaming data, so found this workarround and it's working in my second monitor while I type this.

timmolter commented 11 years ago

@gary-rowe I don't think KaTXi meant it as a suggestion to make it into XChange, but rather as a way to quickly get around the issue for a personal non-trading (just marketdata) app.

I struggled to import the keys myself the other day, so what I ended up doing was to drop back down to Java 6 from Java 7.

KaTXi commented 11 years ago

Well of course! This can't go inside XChange, it's an UGLY hack. It's just a solution for "solo" developers or as a temp fix if you want to use the latest Snapshots to keep developing while a final solution is found.

gary-rowe commented 11 years ago

@KaTXi Apologies - I was tired and posted my response without thinking.

KaTXi commented 11 years ago

@gary-rowe no prob!!!

sslx commented 10 years ago

I started getting the same error on btc-e now. How can I add the exception to java on ubuntu?

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1884) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) at sun.security.ssl.Handshaker.process_record(Handshaker.java:804) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323) at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:563) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1091) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250) at si.mazi.rescu.HttpTemplate.executeRequest(HttpTemplate.java:111) at si.mazi.rescu.RestInvocationHandler.invokeHttp(RestInvocationHandler.java:66) at si.mazi.rescu.RestInvocationHandler.invoke(RestInvocationHandler.java:51) at com.sun.proxy.$Proxy10.getInfo(Unknown Source) at com.xeiam.xchange.btce.v3.service.polling.BTCEAccountServiceRaw.getBTCEAccountInfo(BTCEAccountServiceRaw.java:58) at com.xeiam.xchange.btce.v3.service.polling.BTCEAccountService.getAccountInfo(BTCEAccountService.java:52)

timmolter commented 10 years ago

I don't know how to properly import certs into Java/JVM. All the things I've tried have failed. As a workaround, I made the class CertHelper, which can be found in BTCChinaAccountDemo. If someone knows how to properly handle certs not in the shipped JVM, I'd be happy to work with you and put toegther a guide on XChange's Wiki...

jheusser commented 10 years ago

@sslx btc-e started throwing this exception for me as well.

I found a way to install a cert in the trust store, after many trials. @timmolter Is it ok if I add the steps to the wiki, as a new page?

jheusser commented 10 years ago

The steps are actually extremely simple. Maybe someone could verify if they work for them as well:

  1. Download and compile https://code.google.com/p/java-use-examples/source/browse/trunk/src/com/aw/ad/util/InstallCert.java
  2. Run java InstallCert <domain> (e.g. btc-e.com)
  3. Follow instructions on screen, make sure you understand the implications and verify what certificate you permanently trust
  4. A file jssecacerts has been created which is a Java KeyStore file. Start your application with -Djavax.net.ssl.trustStore=./jssecacerts
timmolter commented 10 years ago

Works! Thanks. I'll add that to the Wiki.

timmolter commented 10 years ago

https://github.com/timmolter/XChange/wiki/Installing-SSL-Certificates-into-TrustStore

jkolobok commented 10 years ago

This nit the best solution. Installing certificates for each jvm is not good. In version 2.0 in BTCEBasePolling service in line 71 there is line this.btce = RestProxyFactory.createProxy(btceType, exchangeSpecification.getSslUri()); Also RestProxyFactory.createProxy(..) method can also accept configuration which can contain SSLSocketFactory which could be dynamically configured with keystore which has certificates for a particular app like this

   KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());
    ks.load(null, "changeit".toCharArray());

    importCertificate(ks, "comodo", "certificates/COMODO Certification Authority.cer");

   private void importCertificate(KeyStore ks, String alias, String path) throws CertificateException, IOException, KeyStoreException {
    BufferedInputStream bis = new BufferedInputStream(getClass().getClassLoader().getResourceAsStream(path));

    CertificateFactory cf = CertificateFactory.getInstance("X.509");

    while (bis.available() > 0) {
        Certificate cert = cf.generateCertificate(bis);
        ks.setCertificateEntry(alias, cert);
    }
}

In this case the certificate will be used only with this app.
Something like this:

SSLContext sslContext = SSLContexts.custom() .useTLS() .loadTrustMaterial(ks) .build(); SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext);

PeterNSteinmetz commented 9 years ago

Perhaps also catching the exception and clarifying the nature of the problem could be helpful. The exception stack trace can be a bit hard to understand.

timmolter commented 9 years ago

@mmazi Do you think rescu would be a good place to catch this specific exception and enhance the error message with a brief hint towards the fix?