Bypassing Certificates and Proxies

[nabobalis]

This isn't a JHelioviewer problem and more of a question.

At LMSAL we are behind several layers of firewalls and proxies and that typically interfere with making secure connections at times.

Typically we disable SSL connections or by adding a new root certificate but I can't get this to work with JHelioviewer.

2024-02-13 15:51:30.710 [Update Checker] WARNING  org.helioviewer.jhv.io.UpdateChecker.lambda$check$1: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
2024-02-13 15:51:30.759 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.timelines.band.BandDataProvider$BandTypeDownloadCallback.onFailure: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

I am asking is there a way to disable the need for SSL via the command line or a configuration value?

On the other side I am working with IT and trying to fix it properly but in the meantime, I thought I would ask.

[bogdanni]

The first error is a problem (cannot reach ROB server to check version). The second one is due to the ongoing rebuild of the server, will be fixed when all services come back online.

On the topic of certificates: Java has its own certificate "truststore". JHV comes with a stripped down JRE which includes keytool which can be used to modify the "truststore". On macOS, those are located at: JHelioviewer.app/Contents/Resources/jre.bundle/Contents/Home/bin/keytool JHelioviewer.app/Contents/Resources/jre.bundle/Contents/Home/lib/security/cacerts

From StackOverflow it seems the command is something like: keytool -import -noprompt -trustcacerts -alias <AliasName> -file <certificate> -keystore <KeystoreFile> -storepass <Password>

Not sure if the modification of the "truststore" invalidates the signature of the entire app.

An alternative is to install the entire JDK from https://jdk.java.net/21/, alter its "truststore" with your certificate and run JHV from command line. The JARs are located at JHelioviewer.app/Contents/Resources/app/classes and the command to start JHV is something like: java --add-exports java.desktop/sun.awt=ALL-UNNAMED --add-exports java.desktop/sun.swing=ALL-UNNAMED -cp JHelioviewer.jar:natives-macos.jar org.helioviewer.jhv.JHelioviewer

If all that fails, I believe there is a programmatic way to disable entirely the certificate checking. This week I'm on vacation, maybe I can have a stab at it next week if you don't succeed.

[nabobalis]

Thanks for the detailed response @bogdanni.

I ended up adding the certificate to both the bundled JRE and the JAVA version that was already installed. That seemed to work!

❯ /Applications/JHelioviewer.app/Contents/MacOS/JavaApplicationStub
2024-02-15 11:46:56.773 [main] INFO  org.helioviewer.jhv.JHelioviewer.main - JHelioviewer started with command-line options:
2024-02-15 11:46:56.785 [main] INFO  org.helioviewer.jhv.JHVGlobals.getVersion - JHV/SWHV-4.6.1.10929 (x86_64 Mac OS X 12.7.3) Eclipse Adoptium JRE 21.0.2 8.0GB 8CPU
2024-02-15 11:46:58.259 [AWT-EventQueue-0] INFO  org.helioviewer.jhv.JHelioviewer.lambda$main$0 - Start main window
2024-02-15 11:46:58.860 [Location Checker] INFO  org.helioviewer.jhv.io.LocationChecker.lambda$setProximityServer$0 - Location: NA -> default server: GSFC
2024-02-15 11:46:59.858 [AWT-EventQueue-0] INFO  org.helioviewer.jhv.JHelioviewer.lambda$main$0 - Load bundled plugins
2024-02-15 11:47:00.064 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.plugins.swek.SWEKConfig.parseGroups: Cannot invoke "org.helioviewer.jhv.events.SWEK$Source.name()" because "this.source" is null
2024-02-15 11:47:00.394 [AWT-EventQueue-0] INFO  org.helioviewer.jhv.opengl.GLInfo.get - OpenGL 4.1 ATI-4.8.101
2024-02-15 11:47:01.135 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.io.LoadSources$Callback.onFailure - GSFC Beta: A JSONObject text must begin with '{' at 1 [character 2 line 1]
2024-02-15 11:47:01.386 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.timelines.band.BandDataProvider$BandTypeDownloadCallback.onFailure: Response{protocol=http/1.1, code=403, message=Forbidden, url=https://swhv.oma.be/datasets/index.php}
2024-02-15 12:12:38.512 [Worker] INFO  org.helioviewer.jhv.io.LoadLayer.requestAPI - {"frames":97,"message":null,"uri":"jpips://helioviewer.org:8090/movies/SDO_AIA_1700_F2024-02-13T18.33.09Z_T2024-02-15T18.33.09ZB1800L.jpx"}

I was unable to download jpips, I need to work out if thats our firewall.

[bogdanni]

The IAS server should be a full mirror of the GSFC server and the JPIP movie streaming connection with it is non-SSL. I expect it to work. The GSFC streaming connection is wrapped within SSL at the lowest level, and may encounter troubles in your context.

[nabobalis]

Thanks for your help

I will speak to our IT and see if we can fix this on our end.

[ebuchlin]

In the meantime @nabobalis could you confirm that the IAS server works from your network?

[bogdanni]

@ebuchlin One optimization we did on the new ROB server (still under re-build) is to assign a second network card and IP address to the VM. That way, we can run esajpip on port 80 (purely a server side configuration) and eliminate an entire class of problems due to restrictive firewalls which block "unknown" ports like 8080 or 8090.

[nabobalis]

In the meantime @nabobalis could you confirm that the IAS server works from your network?

What is the best way to check this?

[bogdanni]

Load a movie from IAS server.

[nabobalis]

Unfortunately, no:

java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 403 Forbidden": jpip://helioviewer.ias.u-psud.fr:8080//movies/SDO_AIA_171_F2024-02-19T17.33.09Z_T2024-02-21T17.33.09ZB1800L.jpx

[bogdanni]

Could you please try ROB server?

[nabobalis]

Same issue:

java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 403 Forbidden": jpip://jpip.swhv.oma.be/movies/SDO_AIA_193_F2024-02-19T17.33.09Z_T2024-02-21T17.33.09ZB1800L.jpx

I think I need to talk to my IT team about this.

[bogdanni]

Re-opening this following the email from Ralph.

@nabobalis So, by inserting that certificate you managed to make the JHV high-level HTTP client work with the proxy, which is actually big progress, it means that Java detects and uses your proxy settings.

The next step is to make the movie streaming work: that network connection is implemented at socket (lower) level. Because I have no access to such a network, the code for using sockets with proxy is implemented in the blind. The whole thing is also a bit voodoo to me, I suspect a SOCKS type of proxy connection should be attempted.

I have uploaded a build at https://swhv.oma.be/download_test/proxy-test/ which attempts to use first a SOCKS proxy type. As before, the proxy certificate has to be inserted in Java's "truststore". The command-line to start it is now simpler, something like java -cp JHelioviewer.jar:natives-macos.jar org.helioviewer.jhv.JHelioviewer. Please attempt to load a movie from the IAS or ROB servers (not GSFC for the time being, because that one uses TLS), and check the log (in the terminal or via Help->Show Log).

If that still doesn't work, please try with java -DsocksProxyHost=<yourProxyHost> -DsocksProxyPort=<yourProxyPort> -cp JHelioviewer.jar:natives-macos.jar org.helioviewer.jhv.JHelioviewer

Does that proxy server need a username and password?

[nabobalis]

Re-opening this following the email from Ralph.

@nabobalis So, by inserting that certificate you managed to make the JHV high-level HTTP client work with the proxy, which is actually big progress, it means that Java detects and uses your proxy settings.

The next step is to make the movie streaming work: that network connection is implemented at socket (lower) level. Because I have no access to such a network, the code for using sockets with proxy is implemented in the blind. The whole thing is also a bit voodoo to me, I suspect a SOCKS type of proxy connection should be attempted.

I have uploaded a build at swhv.oma.be/download_test/proxy-test which attempts to use first a SOCKS proxy type. As before, the proxy certificate has to be inserted in Java's "truststore". The command-line to start it is now simpler, something like java -cp JHelioviewer.jar:natives-macos.jar org.helioviewer.jhv.JHelioviewer. Please attempt to load a movie from the IAS or ROB servers (not GSFC for the time being, because that one uses TLS), and check the log (in the terminal or via Help->Show Log).

If that still doesn't work, please try with java -DsocksProxyHost=<yourProxyHost> -DsocksProxyPort=<yourProxyPort> -cp JHelioviewer.jar:natives-macos.jar org.helioviewer.jhv.JHelioviewer

Thanks, I will try this when I get into work later today

Does that proxy server need a username and password?

No, I have never had to provide it one before.

[nabobalis]

So this happened with the test version, I think I forgot some steps previously (which I forgot).

❯ /Applications/JHelioviewer.app/Contents/MacOS/JavaApplicationStub
2024-07-18 10:46:30.360 [main] INFO  org.helioviewer.jhv.JHelioviewer.main - JHelioviewer started with command-line options:
2024-07-18 10:46:30.370 [main] INFO  org.helioviewer.jhv.JHVGlobals.getVersion - JHV/SWHV-4.7.1.11270 (x86_64 Mac OS X 12.7.5) JetBrains s.r.o. JRE 21.0.3 8.0GB 8CPU
2024-07-18 10:46:31.193 [AWT-EventQueue-0] INFO  org.helioviewer.jhv.JHelioviewer.lambda$main$0 - Start main window
2024-07-18 10:46:33.024 [AWT-EventQueue-0] INFO  org.helioviewer.jhv.JHelioviewer.lambda$main$0 - Load bundled plugins
2024-07-18 10:46:33.218 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.plugins.swek.SWEKConfig.parseGroups: Cannot invoke "org.helioviewer.jhv.events.SWEK$Source.name()" because "this.source" is null
2024-07-18 10:46:33.573 [AWT-EventQueue-0] INFO  org.helioviewer.jhv.opengl.GLInfo.get - OpenGL 4.1 ATI-4.8.101
2024-07-18 10:47:01.198 [Location Checker] WARNING  org.helioviewer.jhv.io.LocationChecker.lambda$setProximityServer$0: Connect timed out
2024-07-18 10:47:03.101 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.timelines.band.BandReaderHapi$CallbackCatalog.onFailure - java.net.SocketTimeoutException: Connect timed out
    at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:546)
    at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:592)
    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
    at java.base/java.net.Socket.connect(Socket.java:751)
    at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:145)
    at okhttp3.internal.connection.ConnectPlan.connectSocket(ConnectPlan.kt:271)
    at okhttp3.internal.connection.ConnectPlan.connectTcp(ConnectPlan.kt:139)
    at okhttp3.internal.connection.FastFallbackExchangeFinder$launchTcpConnect$1.runOnce(FastFallbackExchangeFinder.kt:141)
    at okhttp3.internal.concurrent.TaskRunner.runTask(TaskRunner.kt:124)
    at okhttp3.internal.concurrent.TaskRunner.access$runTask(TaskRunner.kt:44)
    at okhttp3.internal.concurrent.TaskRunner$runnable$1.run(TaskRunner.kt:73)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1583)

2024-07-18 10:47:03.511 [Update Checker] WARNING  org.helioviewer.jhv.io.UpdateChecker.lambda$check$1: Connect timed out
2024-07-18 10:47:03.567 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.io.LoadSources$Callback.onFailure - ROB: Connect timed out
2024-07-18 10:47:03.567 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.io.LoadSources$Callback.onFailure - ESAC: Connect timed out
2024-07-18 10:47:03.567 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.io.LoadSources$Callback.onFailure - GSFC: Connect timed out
2024-07-18 10:47:03.568 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.io.LoadSources$Callback.onFailure - GSFC Beta: Connect timed out
2024-07-18 10:47:03.568 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.io.LoadSources$Callback.onFailure - IAS: Connect timed out

I also updated my own Mac OS java install and tried to run it directly:

Uncaught Exception in JHV/SWHV-4.7.1.11270 (x86_64 Mac OS X 12.7.5) Oracle Corporation JRE 22.0.2
Thread: Thread[#61,AWT-EventQueue-0,6,main]
Message: superclass access check failed: class com.jidesoft.plaf.basic.BasicRangeSliderUI$Actions (in unnamed module @0x2a40cd94) cannot access class sun.swing.UIAction (in module java.desktop) because module java.desktop does not export sun.swing to unnamed module @0x2a40cd94
Stacktrace:
java.lang.IllegalAccessError
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1023)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:862)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
at com.jidesoft.plaf.basic.BasicRangeSliderUI.installAdditionalKeyboardActions(BasicRangeSliderUI.java:58)
at com.jidesoft.plaf.basic.BasicRangeSliderUI.installKeyboardActions(BasicRangeSliderUI.java:51)
at java.desktop/javax.swing.plaf.basic.BasicSliderUI.installUI(BasicSliderUI.java:231)
at com.formdev.flatlaf.jideoss.ui.FlatRangeSliderUI.installUI(FlatRangeSliderUI.java:87)
at java.desktop/javax.swing.JComponent.setUI(JComponent.java:740)
at com.jidesoft.swing.RangeSlider.updateUI(RangeSlider.java:90)
at java.desktop/javax.swing.JSlider.<init>(JSlider.java:308)
at com.jidesoft.swing.RangeSlider.<init>(RangeSlider.java:68)
at org.helioviewer.jhv.gui.components.base.JHVRangeSlider.<init>(JHVRangeSlider.java:9)
at org.helioviewer.jhv.layers.filters.LevelsPanel.<init>(LevelsPanel.java:36)
at org.helioviewer.jhv.layers.ImageLayerOptions.<init>(ImageLayerOptions.java:30)
at org.helioviewer.jhv.layers.ImageLayer.<init>(ImageLayer.java:86)
at org.helioviewer.jhv.layers.ImageLayer.create(ImageLayer.java:44)
at org.helioviewer.jhv.JHelioviewer.lambda$main$0(JHelioviewer.java:90)
at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:318)
at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:773)
at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:720)
at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:714)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:400)
at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:87)
at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:742)
at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)

Log:
2024-07-18 10:57:03.680 [main] INFO  org.helioviewer.jhv.JHelioviewer.main - JHelioviewer started with command-line options: 
2024-07-18 10:57:03.702 [main] INFO  org.helioviewer.jhv.JHVGlobals.getVersion - JHV/SWHV-4.7.1.11270 (x86_64 Mac OS X 12.7.5) Oracle Corporation JRE 22.0.2 8.0GB 8CPU
2024-07-18 10:57:04.549 [AWT-EventQueue-0] INFO  org.helioviewer.jhv.JHelioviewer.lambda$main$0 - Start main window
2024-07-18 10:57:06.939 [AWT-EventQueue-0] INFO  org.helioviewer.jhv.JHelioviewer.lambda$main$0 - Load bundled plugins
2024-07-18 10:57:07.213 [AWT-EventQueue-0] SEVERE  org.helioviewer.jhv.plugins.swek.SWEKConfig.parseGroups: Cannot invoke "org.helioviewer.jhv.events.SWEK$Source.name()" because "this.source" is null

[bogdanni]

For the error with the builtin Java: were you connected to the network? Because it seems all network accesses at startup timed out. If you forgot to add the proxy certificate to Java truststore then the message would have been like before PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

For the exception with separate Java installation, it looks like it still needs the --add-exports java.desktop/sun.awt=ALL-UNNAMED --add-exports java.desktop/sun.swing=ALL-UNNAMED arguments on your computer.

[nabobalis]

For the error with the builtin Java: were you connected to the network? Because it seems all network accesses at startup timed out. If you forgot to add the proxy certificate to Java truststore then the message would have been like before PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

That is strange, I am connected to the network and can do say git pulls, wget or curls with my local proxy settings. So for the proxy settings, I export HTTP_PROXY and HTTPS_PROXY in my shell.

For the exception with separate Java installation, it looks like it still needs the --add-exports java.desktop/sun.awt=ALL-UNNAMED --add-exports java.desktop/sun.swing=ALL-UNNAMED arguments on your computer.

This fixed it for me thanks!