search

OWASP / wstg

The Web Security Testing Guide is a comprehensive Open Source guide to testing the security of web applications and web services.
https://owasp.org/www-project-web-security-testing-guide/
Creative Commons Attribution Share Alike 4.0 International
7.12k stars 1.31k forks source link

Content Review: Testing for Credentials Transported over an Encrypted Channel #419

Closed kingthorin closed 3 years ago

kingthorin commented 4 years ago

Split from: https://github.com/OWASP/wstg/issues/369

Quote: @patrickceg

I know the issue is looking specifically at the HTTP requests, but I see issues with the article as a whole if I read it as just someone wanting to test for the problem.

I assume this is the article? https://github.com/OWASP/wstg/blob/master/document/4-Web_Application_Security_Testing/04-Authentication_Testing/01-Testing_for_Credentials_Transported_over_an_Encrypted_Channel.md#how-to-test

For the header and black box testing

How to Test Black-Box Testing In the following examples we will use a web proxy in order to capture packet headers and to inspect them. You can use any web proxy that you prefer. [Examples start]

Now read that as if you are just someone who wants to test... You still have not told the person how to test it! Don't we need something like:

How to Test To test for credentials being sent insecurely, use a web browser with web developer tools debuggers enabled or an intercepting proxy including OWASP ZAP while you log in. Black Box Testing ....

The other issue is that for some reason, the "Good" and "Bad" cases aren't really outlined. I barely know if "Example 2" is good or not because of how it's buried ... as the second example.

Wouldn't we rather structure it something like this?

The summary section triggers one of my pet peeves:

The analysis focuses simply on trying to understand

Whenever I see "simple", "easy", "simply", etc., it triggers my bad memories of a university professor who filled an entire whiteboard with calculus and said it was "simple". It doesn't quite make me go into a nightmare spiral, but it's still distracting. If it's a process that can be "simply" performed, then stop wasting words on how easy it is: just say the procedure in as few words as possible and let the reader decide if it is easy.

Grey box testing is also weird... it looks more like it should be "Remediation"

Speak with the developers of the web application and try to understand if they are aware of the differences between HTTP and HTTPS protocols and why they should use HTTPS for transmitting sensitive information. Then, check with them if HTTPS is used in every sensitive request, like those in log in pages, to prevent unauthorized users to intercept the data.

If I was to write a Grey box testing, I'd probably phrase it like this:

: If testers have access to the server configuration, verify that server is configured to reject plain HTTP requests containing any credentials.

I'm not too familiar with web server configurations, but I know that you can set certain cookies to only be sent over HTTPS. I'm not sure if there's a security header in pages that forces the browser to fail out if you attempt to force HTTP. That said, honestly in 2020 when web browsers give you the red warning for any HTTP and a Raspberry Pi can server HTTPS, I'd be tempted to just say to verify the server is configured that the only thing it will do if you give it any HTTP is to redirect you to HTTPS...

Another "Grey box" method of looking for the issue would be to check actual web application firewall or intrusion detection logs of the running server. For example, Snort or Suricata with Emerging Threats, Corporate Policy enabled may give you a "ET POLICY Outgoing Basic Auth Base64 HTTP Password detected unencrypted" warning. ...that said the black box method is more reliable anyway, so it's probably a waste of words to mention this.

patrickceg commented 4 years ago

It looks like there were some changes since the last time I saw this, but I can take another look at the article and see if it still needs restructuring. You can assign the task to me to review the current state of the article.

(My limited knowledge on the topic is catching teams making what they think are protected services when they ask username and password and the entire server is HTTP and clear-text SQL database...)

If assigned, I can then review and try to come up with an outline of changes. (When I have the outline, I can also see if I can take it all the way to pull request or if I'll have to drop it or ask for advice on the subject matter.)

patrickceg commented 4 years ago

Okay... I did my initial pass through the article, and I have some questions:

About the article as a whole: Is there ever a reason to use HTTP without the S in production in 2020 other than big downloads that get verified after they make it to the client (i.e. Linux package updates)?

I feel like the article comes back from the time when it was common for web sites to have an HTTPS login page and HTTP for everything else. Do we want to update that general spin in the summary?

With Magecart and other sensitive information sniffing JavaScript, it would also be bad to make a tester think it is okay to have any HTTP components on a login or other page where the user expects to enter or see sensitive data (since the JavaScript will just absorb all that sensitive stuff and relay it to the attacker's command and control.) If you are using an app through compromised network infrastructure, the Javascript malware could get your credential or other sensitive info through the HTTP components ... even worse if it is the web site itself that has the HTTP components because then same origin policy can't protect you.

Other Technical questions I have:

kingthorin commented 4 years ago

Hey @patrickceg thanks for looking in to this.

I'd be fine with this being respun a bit.

I'm not sure we're worried about network compromise, I think it can generally be said if you're trusting your traffic to a network that's untrustworthy there are plenty of bad things that can happen.

Is there a reason we are suggesting a web proxy rather than just the browser in debug or developer mode?

It doesn't have to be limited. Nearly everyone that does web app testing uses a proxy, however, as you're pointing out chances are 100% use a browser 😉 Also keep in mind there isn't really a "we" here with regard to the content, much of it is from 4.0 six years ago so it's just whatever was contributed at that point. So we can and should re-write anything that could use it.

I also know man-in-the-middle type proxies is a bit crippled with TLS 1.3. Would that affect the article content?

I don't think I've done any TLS 1.3 testing yet, I know ZAP supports it but we haven't had any reports of issues. If there are then yes it probably impacts the article at least a bit. At the same time if TLS 1.3 is in play then the creds aren't being transported over a clear channel 😉

Should we consider Wireshark or other higher levels of analysis of packet capture including IDS/IPS?

Wireshark is cool with me. I would not expect the majority of testers to have IDS/IPS handy (even as a Snort VM or whatever and having to reconfigure networking ... though if you really want to write it I'm not going to stop you :grinning: ).

Edit: Are there services that use Web Sockets for auth? Maybe ws vs wss should be considered here?

patrickceg commented 4 years ago

The reason I mentioned Wireshark or other tools is I'm not sure how you would test a web app running on a mobile device or other platform where you can't conveniently have a whole debugging panel (again thinking about the web pages that present a completely different experience to a phone which may or may not mean communication with the back-end is also different).

There has also been talk of APIs being added more on this, which means you'll then have smart speakers or other home automation potentially trying to authenticate. Those for sure don't have screen space to have a debugging console 💡

I'm open to ideas on how you test a mobile device (open ZAP's proxy port and tell the mobile device to use it? 🤷 ).

In the meantime I'll see what I can do without adding new content and try drafting an outline on the weekend.

kingthorin commented 4 years ago

The options I've seen for mobile usually involve one or many of:

Anyway, I'm happy for wireshark etc to be included just keep in mind this is a single test scenario.

patrickceg commented 4 years ago

Does anyone know of a good site where I can test fake logins? For my changes to the article, I'm going to need logs of ZAP recording all the interactions of logging in to a site. (i.e. POSTing the credential and getting back a token or cookie) If I need to set up locally, then I plan to spin up a local MediaWiki server with HTTPS and capture interactions with that. (I figured MediaWiki would have the least additional junk in the logs being an older service, but other ideas for local VM to poke are welcome.)

I came up with my idea of the changes and put some rough notes in a GIST https://gist.github.com/patrickceg/2b555da3142d1f909bc28cf6cc86e9c2 (the GIST isn't really meant for other people to read - just keeping all my progress in Github)

kingthorin commented 4 years ago

You could use the one click Heroku deploy of Juice Shop: https://github.com/bkimminich/juice-shop#deploy-on-heroku-free-0month-dyno

Hmmm second thought, I dunno how http vs https works out for that.

If you just use LAMP or XAMP locally you’d probably have phpMyAdmin to play with.

patrickceg commented 4 years ago

It definitely took longer than I wanted, but I finally have my pentesting box pointed at a target to gather ZAP logs. (I'm noticing the effects on my concentration from the infamous global event this year... I just lost two hours due to a fumble with root privileges on one of my lab machines.)

Anyway for the task:

I'm going to first try getting the necessary data for a test case failure logging in to Juice Shop on HTTP without the S and see what that looks like in a ZAP log. I've never tried Juice Shop before, so I'm going to look around for a credential exchange that is relatively clean (hopefully with just an HTTP POST followed by an easy to read response that shows the session token).

It will be a learning experience for ZAP as well to be able to get the request logs.

patrickceg commented 4 years ago

Hmm I just started testing, and Juice Shop with intentionally misconfigured nginx providing HTTPS but not properly rejecting sensitive HTTP will work great as a "fail" case.

However I'll have to set up a "pass" case where even if I deliberately force HTTP login by making OWASP ZAP or similar tool replay the request for the login or "I forgot my password" page over HTTP without the S, the server responds by redirecting you to the HTTPS page. I think I can set up the nginx to do that (I did that ~2 years ago at work for a dev server), although it would be nice to have an app+server package where it's a configuration option to turn on HTTPS and the service does it properly. (Specifically the test case I'm trying to duplicate is if someone deliberately types in: http://site-under.test/login?forgotpassword or similar. The server should respond by giving them the home page or redirect to HTTPS login page rather than serving up an HTTP login or forgot password page.)

I suppose another intricacy is if someone logs on and then force browses to http://site-under.test, the request should not expose the user's session cookie. That said I'm pretty sure that's another test case (for the secure cookie).

ThunderSon commented 4 years ago

@patrickceg from my understanding, you're running nginx correct? It's pretty straight forward with a couple of lines:

server {
    listen 80;
    server_name <domain_here>;
    return 301 https://<domain_here>$request_uri;
}

For the cookie bit, they need to be set with the Secure directive in order to enforce TLS usage for them. For the nginx redirection, if you only want it for a certain URL, we'll need to create a location under the server block of port 80 and it's easily doable as well.

Hope I got you properly. If there's any other blockers, let me know.

patrickceg commented 4 years ago

I started out drafting the article to figure out which HTTP(S) captures I really need:

https://github.com/OWASP/wstg/compare/master...patrickceg:419-encrypted-channel-revise?expand=1

As I mentioned in the original comment that started this task, I essentially changed "Grey Box Testing" to "Remediation" because it was already worded that way.

While it may seem odd to anyone in application security, I came across a dev team who didn't know that Let's Encrypt existed (earlier this year!) and had this big discussion over paying a certificate authority (CA) or dropping all security and putting passwords over HTTP, so I just had to mention it in the remediation.

ThunderSon commented 4 years ago

563 @patrickceg what do you think about this? It'll fit perfectly with the work you're doing.

It's not an agreed upon decision to make the move, I am just opening up the discussion.

patrickceg commented 4 years ago

@ThunderSon I was thinking there should be a test to say that anything that can be considered sensitive in any way should be encrypted in transit using TLS. (I specifically mention TLS because password hashing and salting is a different form of cryptography with a different set of tests).

There's definitely a lot of things other than just credentials to test are transferred over TLS, including:

That said, thanks to the router takeovers in 2018 and 2019 that started injecting cryptocurrency mining into any page loaded with HTTP without the S, I already started changing the cryptography remediation section to state that you should TLS all the things. My additions to the section include the following

Remediation

The best remediation would be to make the entire web site or web application use HTTPS, and redirect plaintext requests to HTTPS. Using HTTPS on the whole site prevents attackers from modifying interactions with the web server (including placing JavaScript malware). Newer software also has a warning or restriction for HTTP traffic. Browsers mark HTTP based web sites as insecure and Android applications need overrides to connect to anything via HTTP. Prioritize HTTPS for sensitive operations first. For the medium term, plan to convert the whole application to HTTPS to avoid losing customers to compromise or the warnings of HTTP being insecure. If the organization does not already buy certificates for HTTPS, look in at least using Let's Encrypt or other free certificate authorities on the server.

ThunderSon commented 4 years ago

I'd prefer making the remediation to be more bulleted. This looks like a lot of text. We can make this fit properly into the guide once you bring in the commit.

@kingthorin one thing is clear from Patrick's comment. The #563 issue is going to change the order of the IDs pretty roughly. This is now becoming into a cryptography concern, more than an ATHN one. What's your take on this?

patrickceg commented 4 years ago

I'd prefer making the remediation to be more bulleted. This looks like a lot of text.

The existing article's style had a preference for wall of text rather than lists (it actually had no bullets at all). Are newer articles moving away from that?

Example walls of text:

In this last example, suppose that the application transfers data using the GET method. This method should never be used in a form that transmits sensitive data such as username and password, because the data is displayed in clear text in the URL and this causes a whole set of security issues. For example, the URL that is requested is easily available from the server logs or from your browser history, which makes your sensitive data retrievable for unauthorized persons. So this example is purely demonstrative, but, in reality, it is strongly suggested to use the POST method instead (https://www.example.com/success.html?user=test&pass=test).

Now, imagine having a web page reachable via HTTP and that only data sent from the authentication form are transmitted via HTTPS. This situation occurs, for example, when we are on a portal of a big company that offers various information and services that are publicly available, without identification, but the site also has a private section accessible from the home page when users log in. So when we try to log in, the header of our request will look like the following example (https://www.example.com:443/login.do):

kingthorin commented 4 years ago

Don't feel stuck with anything that "is" or "was", I personally find bulleted lists easier to consume. But if a particular thing seems better as a paragraph then whatever. There isn't really a right or wrong way IMHO.

I know some questions were asked of me earlier, I'm still trying to digest and decide. If I haven't replied within a few days then please feel free to ping me.

patrickceg commented 4 years ago

Hmm I'm seeing a case where we may want to block this task and then replace it with tasks to start building a cryptography section.

I notice we have the weak cryptography section: https://github.com/OWASP/wstg/tree/master/document/4-Web_Application_Security_Testing/09-Testing_for_Weak_Cryptography

...and there's already an article that's very close to this one in there: https://github.com/OWASP/wstg/blob/master/document/4-Web_Application_Security_Testing/09-Testing_for_Weak_Cryptography/03-Testing_for_Sensitive_Information_Sent_via_Unencrypted_Channels.md

Therefore I'm seeing a case where we rename the whole section "Testing Quality of Cryptography" and we have all the items related to checking whether cryptography exists in particular subsystem of an application and that it is properly implemented if it does exist.

Next time I get a chance (probably 3 to 6 days from now), I'll check over the whole guide to see which tests would fit in a dedicated crypto section. Off the top of my head, I can think of:

At rest:

In transit:

patrickceg commented 4 years ago

I'd also like to add that there was one case that actually does belong more in an auth test, specifically the secure attribute/tag in cookies. (If you don't have said tag, the browser is allowed to spit out session tokens over HTTP.)

kingthorin commented 4 years ago

@kingthorin one thing is clear from Patrick's comment. The #563 issue is going to change the order of the IDs pretty roughly. This is now becoming into a cryptography concern, more than an ATHN one. What's your take on this?

This scenario deals specifically with auth data. Other types of data and concerns around cookie details are covered elsewhere.

ThunderSon commented 4 years ago

@patrickceg I feel like the best thing to be done first is to see what's the progress from your side. No need to worry about #563 for now, it was just a discussion. Can you please open a PR with whatever there is ready, and we'll help with things?