Solved: HTTPS access not work

[mhtvsSFrpHdE]

I have generated "root-ca.crt.pem" and "root-ca.key.pem", then paste them into %Appdata%\James. After James launch, the bottom-right corner shows "Proxy: Online" that would be means James now is HTTPS enabled mode.

I have also imported the cert pem file into system, use "Manage computer certificates" from Windows Control panel. And import that file to "Trusted Root Certification Authorities". The operation request the Admin permission, of course.

Then set system proxy as "127.0.0.1:1338", use any browser the HTTPS can't work. The situation is, Microsoft Edge browser & Internet Explorer shows

"Can’t connect securely to this page

This might be because the site uses outdated or unsafe TLS security settings. If this keeps happening, try contacting the website’s owner."

It didn't give a option to ignore cert error just like if a site use a outdated cert and able to ignore. On another hand, Chrome & Chromium & Firefox give error is

This site can’t be reached
github.com unexpectedly closed the connection.
Try:

Checking the connection
Checking the proxy and the firewall
Running Windows Network Diagnostics
ERR_CONNECTION_CLOSED

There is no ignore option there too. Then I launched Chromium browser with command line argument

PS>.\chrome.exe --ignore-certificate-errors

The Chromium shows a banner that alert me it running in error ignore mode. Then it report the same error message like before. I don't counting on it is a cert installation issue right now... There must be something wrongly more.

Then I back to James interface, I notice that under Requests tab, all entry are HTTP, but no HTTPS shows. And if I try some pure HTTP site, no error. the HTTP mapping is also good. Because of HTTPS not work, so HTTPS mapping isn't working too.

The system is Windows 10 1803, James version 2.1.0 and 2.0.0.

[mitchhentges]

Interesting, I'll check this on my Windows VM later this week!

I know that our auto-configuration of browsers for HTTPs isn't really strong - I was playing with Firefox last week and realized that there seems to be a couple of undocumented steps required to make it work.

Based on how James says that "Proxy: Online", and how the browsers are noticing a certificate change, that's got me pretty confident that this is definitely something browser-configuration-related.

Thanks again for the bug report, I'm looking forward to investigating this further :)

[mhtvsSFrpHdE]

@mitchhentges Wait... did I required to tell browser that use 127.0.0.1:1338 as a "HTTPS" proxy instead of "HTTP" proxy? There is does a option there about it says I can type a HTTP, HTTPS, Socket, FTP.

I heard about that HTTPS proxy protocol is definitely different than a HTTP proxy protocol, but I never found a implementation of it(For example, a HTTPS proxy server like a HTTP proxy server Privoxy). People use HTTP proxy to handle HTTPS requests everywhere.

I will do the test later today. Now I away from my PC.

[mhtvsSFrpHdE]

HTTPS proxies were invented to ensure communication with end-to-end security. In this flow, the client sends a special request to the proxy with the CONNECT verb. The proxy builds an opaque tunnel by connecting to the requested server using TCP and nothing else. After the socket connection is established, the HTTPS proxy sends a 200 OK response to the client and starts forwarding data from the client to the server and back. Such a design means that the client and the server are not limited to HTTPS traffic. In fact, any protocol can be tunneled using an HTTPS proxy and the CONNECT verb.

So a HTTPS proxy is just HTTP proxy with CONNECT support? This not make sense...

[mitchhentges]

I'm not sure the specifics of how an HTTPS proxying server works, but it looks like you're right - just an HTTP proxy with CONNECT support. The implementation we're using is hoxy (see the HTTPS-specific initialization logic here).

I'm having issues reproducing your specific situation on Windows 1803 :thinking: Can you send me pictures of:

1. Your openssl certificate in certlm?
2. Your Windows proxy configuration?

From a fresh(-ish) VM, HTTPS seems to be working. I tried working backwards to reproduce your situation - my first step was removing my certificate from certlm. However,

• Edge/IE show "Your PC doesn’t trust this website’s security certificate.", rather than "outdated ..."
• Firefox shows "Your connection is not secure", "... uses an invalid security certificate", rather than "this site can't be reached"

I'm wondering how you reproduced the issue? Actually, here's a theory: the openssl situation on Windows isn't great (I think the top hit on Google is for an out-of-date version). How did you generate your certificates, and what version of openssl are you using? (Do openssl version in a command prompt. It should be 1.1.1a)

[mhtvsSFrpHdE]

Ok, I'll do more tests and provide information. Since this is a "In my machine that works fine" problem(I do programming too so I can understand this). Then it must be a environment problem instead of software problem.

I use Windows 10 Linux subsystem Debian from Windows Store to generate the certs. run openssl version result in:

OpenSSL 1.1.0j  20 Nov 2018

And the cert now is exactly the same as your screenshot image. The Internet Options confirmed have valid proxy address otherwise http proxy won't work.

Now when the James received a http requests, it will show a 302 status code that redirect url to https. This at least provide that no firewall rules prevent browser to use 1338 port. Then the browser shows connection closed. That means for a unknown reason James actively declined the connection. I don't have enough knowledge that can use a network monitor tool to analyze James's traffic, so I can't confirm that if James actually send a request to remote server then drop request by condition, or just drop that "CONNECT" request.

For the 2nd situation, I trust read James log can show more details about this behavior. Under

%Appdata%\James

There does a folder called "logs", but it is empty.

[mhtvsSFrpHdE]

So is there a method to save debug log for help me to find out the issue further? Life being harder while no a sharp tool available to use🤣

[mitchhentges]

Hey, sorry, I don't have a lot of time to investigate this right now. I appreciate your enthusiasm! I'm not sure how to set up debug logging with electron at the moment, but will follow-up once I find out (when I have time :smile:)

Thanks :)

[mitchhentges]

Hey, I think I've got a way that you can view the logs:

1. Open a command-line window (powershell is fine)
2. Drag James into your command line window and press Enter
3. Logs appear in the command line window

[mhtvsSFrpHdE]

So the log is

Loading URL mappings...
Starting proxy...
INFO: proxy listening on 1338
hoxy error:  undefined Error: Could not find openssl on your system on this path: openssl
    at C:\Users\<user>\AppData\Local\Programs\james\resources\app.asar\node_modules\pem\lib\openssl.js:237:23
    at F (C:\Users\<user>\AppData\Local\Programs\james\resources\app.asar\node_modules\which\which.js:68:16)
    at E (C:\Users\<user>\AppData\Local\Programs\james\resources\app.asar\node_modules\which\which.js:80:29)
    at C:\Users\<user>\AppData\Local\Programs\james\resources\app.asar\node_modules\which\which.js:89:16
    at C:\Users\<user>\AppData\Local\Programs\james\resources\app.asar\node_modules\isexe\index.js:42:5
    at C:\Users\<user>\AppData\Local\Programs\james\resources\app.asar\node_modules\isexe\windows.js:36:5
    at FSReqWrap.oncomplete (fs.js:182:21)

This is a pretty stupid error, but no a hint on the user interface right now. It's assume openssl is installed on this system to generate cert. Then the openssl should be available to use as expected.

One of the user generate the cert from Linux and copy them to a Windows without openssl installed.

[mitchhentges]

Wow! Thanks for the reply :) I'm impressed that one of the users moved a cert from Linux, that's really smart. Unfortunately, yeah, we depend on openssl.

I'll reproduce this locally, then I can probably make the UI show a little warning if it can't do HTTPs proxying due to missing openssl. I wonder if it's possible to have this work on Windows without users having to manually install openssl, that's a bummer :thinking:

(side-not: hahaha I love that clip! What's that from again? Gravity falls?)

[mhtvsSFrpHdE]

@mitchhentges I don't know exactly, the image pushed from Telegram channel "Programmer Jokes"

https://t.me/programmerjokes

By addition I'm looking a method to start James without add openssl folder to system path. (in some case change environment variable not a handy behavior) in a cmd script,

@set path=%path%;D:\Program1\openssl-0.9.8h-1-bin\bin
@cd /d %appdata%\..\Local\Programs\james
start James.exe

This can let James to recognize openssl. but it left a cmd window out there and print logs from time to time.

If me, I maybe move the openssl default path to a external config file, so once the program is started, it read openssl installation path from the config file.

But there should be a method that not have to modify James source code to implement that.

[ZusMexSide]

there is any solution?