ripmeapp2 / ripme

Downloads albums in bulk
MIT License
311 stars 38 forks source link

[Enhancement Request] Add option: don't validate the server's certificate #171

Open colemar opened 10 months ago

colemar commented 10 months ago
Initializing Main thread pool with 5 threads
Found album ripper: com.rarchives.ripme.ripper.rippers.MotherlessRipper
...
Using album title 'motherless_E484CF1'
[+] Creating directory: ..\..\..\Downloads\RipMe\motherless_E484CF1
Set working directory to: C:\Users\colem\Downloads\RipMe\motherless_E484CF1
Initializing Main thread pool with 5 threads
Retrieving https://motherless.com/GE484CF1
Changed URL to https://motherless.com/GME484CF1
Trying to load cookies from config for motherless.com
Got doc location https://motherless.com/GME484CF1
Processing a doc...
retrieving urls from doc
Found image url #1: 'https://motherless.com/GE484CF1/50EED86'
Trying to load cookies from config for motherless.com
url: https://cdn5-images.motherlessmedia.com/images/50EED86.jpg, subdirectory, referrer: null, cookies: null, prefix: 001_, fileName: null
Downloading https://cdn5-images.motherlessmedia.com/images/50EED86.jpg to C:\Users\colem\Downloads\RipMe\motherless_E484CF1\001_50EED86.jpg
    Downloading file: https://cdn5-images.motherlessmedia.com/images/50EED86.jpg Retry #1
Key request.properties in  is: Request properties
Request properties: {Cookie=[], User-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36], accept=[*/*]}
IOException
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

This is not really a RipMe fault, because the certificate chain of cdn5-images.motherlessmedia.com is incomplete: https://www.ssllabs.com/ssltest/analyze.html?d=cdn5-images.motherlessmedia.com

I guess the remote web server should send the full certificate chain, but it does not.

This shows 3 certificates openssl s_client -servername motherless.com -connect motherless.com:443 -showcerts

This shows only 1 certificate, with error "Verify return code: 21 (unable to verify the first certificate)" openssl s_client -servername cdn5-images.motherlessmedia.com -connect cdn5-images.motherlessmedia.com:443 -showcerts

CURL also has an issue downloading from cdn5-images.motherlessmedia.com.

A regular browser just works, likely because it caches intermediate certificates. For example Chrome shows this chain: image

I solved the issue importing the intermediate certificate "GoGetSSL RSA DV CA" into the Java CA certificate store (%JAVA_HOME%\lib\security\cacerts) by means of the following command bin\keytool -import -trustcacerts -file "GoGetSSL RSA DV CA.crt" -keystore lib\security\cacerts where "GoGetSSL RSA DV CA.crt" was exported from Chrome web site informative box.

Could you add an option to ignore certificate validation? I mean like curl -k or wget --no-check-certificate.