HTTP Proxy Server doesn't do "https"

[asfimport]

Jean-Luc Peleran (Bug 7378): Hi,

I'm using JMeter 1.7 with JSSE 1.0.2 and JDK 1.3.1_02. The HTTP Proxy Server is unable to deal with HTTPS requests with any syntax : https://www.foo.com (raises a java.net.MalformedURLException: unknown protocol:) or http://www.foo.com:443 (raises a java.io.IOException: Error writing to server)

Thanks for your great work.

Votes in Bugzilla: 4 OS: All

[asfimport]

Mike Stover (migrated from Bugzilla): The proxy server has been improved. It won't crap out if using it on an https site. However, it won't record anything either.

[asfimport]

y2k (migrated from Bugzilla): Hi, I setup Proxy server in the workbench and a simple controller in the test plan and when I try to record this URL "https://my.screenname.aol.com/_cqr/login/login.psp", I am getting this with the latest nightly build:

---

D:\apache\jmeter\jakarta-jmeter\bin>CALL ..\lcp.bat ..\lib\Tidy.jar Initializing... Creating Config Object...OK Creating Cache Manager...OK Creating Daemon Socket... port 8080 OK Proxy up and running! Command = CONNECT my.screenname.aol.com:443 HTTP/1.0

java.net.ConnectException: Connection refused: connect at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(Unknown Source) at java.net.PlainSocketImpl.connectToAddress(Unknown Source) at java.net.PlainSocketImpl.connect(Unknown Source) at java.net.Socket.connect(Unknown Source) at java.net.Socket.connect(Unknown Source) at java.net.Socket.<init>(Unknown Source) at java.net.Socket.<init>(Unknown Source) at org.apache.jmeter.protocol.http.proxy.Proxy.writeToClient(Proxy.java: 318) at org.apache.jmeter.protocol.http.proxy.Proxy.run(Proxy.java:157) Delivering urlconfig to test tree

---

[asfimport]

aj slater (migrated from Bugzilla):

I made a patch that adresses this. Its not a general solution to JMeter SSL it just gets the Proxy working with HTTPS.

http://www.cybercom.net/~slater/jmeter/

Enjoy. Feel free to email me with any questions.

-aj

[asfimport]

Mike Stover (migrated from Bugzilla): Created attachment todo.html: proxy experiment

todo.html

````html

About Overview Download Changes Known Bugs License TODO Contributors Documentation User Manual Developer Manual Community Get Involved Mailing Lists CVS Repositories	ToDo List List of those things yet to do, wishes, dreams, etc.. (in no particular order) This page has been made obsolete by the Jakarta Bugzilla database, where enhancement requests are stored. You may submit your own enhancement request there as well.
Copyright © 1999-2001, Apache Software Foundation

````

[asfimport]

Jordi Salvat i Alabart (migrated from Bugzilla): Is this really a bug? Configuring your browser to use the proxy for HTTPS is a mistake with current releases of JMeter. Having the proxy be able to process https AND record is a difficult task -- and still it will require getting hold of the site's private key.

AJ's solution (a "man in the middle attack", in a way) is probably the best possible.

I'm attaching AJs code for later reference. It's not a clean patch, so it will take a while to review and merge. I'm marking this as a medium-priority enhancement.

[asfimport]

Mike Stover (migrated from Bugzilla): It turns out this really isn't possible - when you ssl through a proxy, the proxy creates a tunnel for you and the proxy is not privy to the details of your request (which is a good thing). Thus, there's no way JMeter could record your actions. The only solution that I can see working for this is embedding a browser in JMeter, and unfortunately, the only high-quality java browsers I know of are commercial.

Possibly a python browser exists that could be used inside JMeter - haven't looked at that possibility.

[asfimport]

Martin Ramshaw (migrated from Bugzilla): This is confusing ... HTTPS is not a different protocol than HTTP, it's merely HTTP over an SSL-encrypted channel (LDAP/LDAPS, amongst others, can also be handled this way). The channel is encrypted point-to-point before the actual HTTP takes place, so the proxy should function as what is called a 'blind relay'.

I would suggest that recording HTTPS in a useful way is impractical (if not impossible).

Here's a useful suggestion from Jordi:

<jordi> A usually practical way is to:

1/ Create a User Defined Variable in the Test Plan with value "http".
Name it, say, "protocol".

2/ Record your script. The proxy will cleverly (sometimes too cleverly)
replace every occurence of "http" with ${protocol}.

3/ Change the variable value to https.

4/ Run the script. </jordi>

This workaround gives you the possibility to perform HTTPS benchmarks, etc. after having recorded your original browser interactions (which must be HTTP rather than HTTPS) with the proxy.

I am closing this bug for these reasons as 'WONTFIX'.

[asfimport]

Ralf Hauser (migrated from Bugzilla): see also https://github.com/apache/jmeter/issues/744 and http://wiki.apache.org/jakarta-jmeter/JMeterAndHTTPS and http://mail-archives.apache.org/mod_mbox/jakarta-jmeter-user/200507.mbox/%3C21114420E9CACD49AD54B1C40D4571C757B136@pdcavantica.avanticatec.net%3E

badboy is not that good an idea since it is not really opensource :(

So, it would be good to be able to record via https!

How does http://selenium-ide.openqa.org/ work with jmeter?

BTW, I would object to <<I would suggest that recording HTTPS in a useful way is impractical (if not impossible).>> Why should "Secure Content Inspection Proxies" like http://summerweb.microdasys.com/products/scip/features-and-benefits/ only help large corporations and law enforcment, but not us testers?

[asfimport]

Marc Rennhard (migrated from Bugzilla): I agree with Ralf (and probably several others) that https recording would be a great addition to JMeter.

Of course it won't work when simply tunneling https traffic through the proxy (as proxies usually do), but it could relatively easily be enabled by splitting the end-to-end SSL/TLS session so we get two sessions, one between browser and proxy and the other between proxy and server. This gives JMeter access to all application data (http). The only difference for the browser is that it wouldn't receive the server's original certificate but one from JMeter (and therefore produce a warning), but that wouldn't be a problem.

Other recording proxies make use of this approach, among them the Java-based WebScarab project (http://www.owasp.org/index.php/Category:OWASP_WebScarab_Project).

To summarize: it can be done (probably even relatively easily using standard Java features (JSSE)) and would definitely make JMeter an even more complete package.

[asfimport]

Michael Tschannen (migrated from Bugzilla): Good news: It works!

Just check "Attempt HTTPS Spoofing" in the HTTP Proxy Server and JMeter will work with HTTPS, too. It doesn't really split the SSL-session as WebScarab does, but JMeter finally acts as the client-side endpoint of the HTTPS-session (which means that all traffic between the browser and JMeter isn't SSL-tunneled but HTTP, and it's JMeter that establishes the SSL-connection to the web server). Additionally, JMeter rewrites all "HTTPS"-strings in the page (e.g. in links) to "HTTP". Just use "HTTP" instead of "HTTPS" in the browser and JMeter will do the rest.

Hovewer, there is still a problem concerning hard-coded HTTPS-redirects, which aren't rewritten by JMeter. It is indeed possible to manually change the requested URL, but an automatically rewrite by JMeter would be a nice enhancement here...

[asfimport]

Ralf Hauser (migrated from Bugzilla): re comment 10, see also https://github.com/apache/jmeter/issues/2099