Recording proxy with browser javax.net.ssl.SSLHandshakeException

[sberequek]

Hi,

I try to recording a warc with jwarc in proxy mode and anything browser I use fail. For run jwarc in proxy mode I used this commands:

export PORT=8080
java -jar jwarc-0.13.1.jar recorder > test.warc

This is the log:

javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
    at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
    at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:356)
    at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293)
    at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:202)
    at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:171)
    at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1488)
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1394)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:441)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:412)
    at org.netpreserve.jwarc.net.HttpServer.upgradeToTls(HttpServer.java:137)
    at org.netpreserve.jwarc.net.HttpServer.interact(HttpServer.java:87)
    at org.netpreserve.jwarc.net.HttpServer.lambda$listen$1(HttpServer.java:58)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
    at java.base/java.lang.Thread.run(Thread.java:832)

How I can resolve this problem? There is a possibility to run jwarc in proxy mode with a new certificate?

Thanks

[ato]

The recorder tool is rather barebones and is more at the level of a proof of concept / example. If you're looking for a more production-ready WARC recording proxy try warcprox.

At the moment the best option is to run Chrome or Chromium with the --ignore-certificate-errors option. e.g.:

chromium --ignore-certificate-errors --proxy-server=http://localhost:8080

I've updated the README to note that.

As long as it didn't add any dependencies I'd be open to accepting a pull request that added an option to read a CA certificate and key from a file instead of generating one in-memory on startup though. There may be gotcha with having the browser trust a custom CA certificate due to modern browsers doing HSTS preloading though, I haven't looked at this stuff in a while.

[sberequek]

Hi,

thank you, I had already tried this:

Chrome --proxy-server="http://127.0.0.1:8080" --ignore-certificate-errors --ignore-urlfetcher-cert-requests

but I have not solved. I'll try warcprox although it seems expensive as a solution for my needs.

[ato]

Hmm, it works for me on Openjdk 8 on Linux. Maybe there's a platform difference. Which OS and Java version are you using?

[sberequek]

• MacOS Big Sur version 11.2;
• Openjdk 8 version "15.0.1";
• Google Chrome version "88.0.4324.150";