Better NTLM and Negotiate Auth Support

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Using ZAP as a proxy, connect to a web app supporting NTLM (v1/v2) or 
Kerberos authentication
2. ZAP fails respond with the required Authentication/Authorization headers

What is the expected output? What do you see instead?
ZAP should authenticate the user (or prompt in case where the provided 
credentials are not valid) and proceed with the next request in the queue

What version of the product are you using? On what operating system?
2.0.0, All OSes

Please provide any additional information below.
NTLM support has been lacking in ZAP. This request is to bolster NTLM/Negotiate 
support for ZAP so intranet applications that rely on AD for authZ/authN will 
be able to use ZAP as an attack proxy for security assessments.

Original issue reported on code.google.com by prasad.s...@gmail.com on 15 Feb 2013 at 3:53

[GoogleCodeExporter]

Original comment by psii...@gmail.com on 15 Feb 2013 at 4:15

• Changed state: Accepted

[GoogleCodeExporter]

Do you mind to provide the ZAP log with an authentication attempt?
Note that sensitive information may be logged, so you should take care to 
remove it.

I've attached a log4j.properties that should be used to generate the log 
(changed to debug only the required components and to use other file name 
(zap_report.log)).

Original comment by THC...@gmail.com on 15 Feb 2013 at 5:16

Attachments:

• log4j.properties

[GoogleCodeExporter]

Attached is the log file as requested. I noted several things that can possibly 
explain the failure but I will let the experts chime in first. 

Original comment by prasad.s...@gmail.com on 15 Feb 2013 at 8:41

Attachments:

• ntlm_zap_report.txt

[GoogleCodeExporter]

A few years ago while using Paros and another well know proxy for web app pen 
tests,  I had run into this issue. Back then, I figured that when native NTLM 
is required i.e. the currently logged on user has the required rights and the 
intent to log into the application using the windows logged on credentials, 
adding a certain HTTP Response header to a proxied response does the trick. The 
header is "Proxy Support: Session-Based-Authentication". I had written a paper 
(private to the org) on this topic and for some months my team was doing this 
via an automated rule for that other well know proxy (till I believe it was 
added as a feature by the author). So, while trying to do the same thing for 
ZAP today, I noticed that it works for most part. Attached is a second set of 
logs. I also noticed that the logs don't contain the custom response header I 
had added after breaking the authentication request/response. In any case, you 
can clearly see HTTP 200 OK from the server and I could see the page loads. A 
tcpdump also confirmed that all 3 types of NTLM messages were being transmitted 
between the client and the server. This might be a good news. 

Original comment by prasad.s...@gmail.com on 15 Feb 2013 at 9:26

Attachments:

• ntlm_zap_report-proxysupport.txt

[GoogleCodeExporter]

Hi :-)

It would be great looking into this issue, which I'm planning to do as well, as 
I'm having trouble crawling/scanning (officially allowed pentests) from a 
corporate network with an internet access protected by NTLM/Kerberos proxy. 
Actually, in this case I have to use Tor (very slow) or forget about scanning 
at all.

Cheers,
Sergey

Original comment by serge....@gmail.com on 19 Feb 2013 at 3:26

[GoogleCodeExporter]

Sergey,

Are you already working on a related issue? Someone else had reiterated your 
thoughts/concerns. However, the problem I am trying to address is not of an 
upstream proxy for ZAP but for an end user application/server that is 
configured to use NTLM/Negotiate type authentication which fails when ZAP is 
the only intermediary/attack proxy.

Although in your case, I suppose you have an upstream proxy for ZAP that 
requires NTLM/Kerb token for authentication. That's slight a different problem 
in my opinion unless I am getting it wrong. 

If I may suggest, you can try chaining ZAP with Burp as an upstream proxy - 
Burp for proxy-authentication support while ZAP as an attack proxy. Further, 
configure Burp's "Upstream Proxy Servers" section with your IP and NTLM 
credentials for your corporate proxy. This is the quick and easy way to get 
around. Tor can be useful but also painful :). 

Hope this helps.

Original comment by prasad.s...@gmail.com on 19 Feb 2013 at 3:44

[GoogleCodeExporter]

Thanks for providing the logs.

From the first log it seems that the message flow is correct. If possible, 
could you provide the NTLM messages (with sensitive data replaced with the 
field names) when authenticating using ZAP and when authenticating successfully 
using other client?

Would you mind check if the user name (in the message type 3) is correct when 
the authentication is performed by ZAP?
It may not be properly encoded (the same for the password).

Regarding the header "Proxy-Support", it shouldn't be needed when the 
authentication is performed by ZAP. We could add it when ZAP is not configured 
to authenticate (or when ZAP is unable to authenticate?) and the response is 
forward to the client.

Original comment by THC...@gmail.com on 19 Feb 2013 at 8:10

[GoogleCodeExporter]

Are you asking for a tcpdump for both scenarios? I can provide those for sure.  
The username password is accurate for  both these logs. In fact, when adding 
proxy support header manually for ZAP (log submission #2), it seemed to work. 
May be because ZAP fails and the pass through gets invoked? Not sure, have to 
investigate more.

In reading a bit about it, appears as if the HttpClient does not handle the 
Domain part of authentication for NTLM as it should (domain\username). It 
either sends a NULL entry for the domain or uses the hostname as the entry for 
domain treating it like an Basic Auth realm (username@hostnamerealm)

Original comment by prasad.s...@gmail.com on 19 Feb 2013 at 8:41

[GoogleCodeExporter]

If it's easier/faster to capture the whole packets that will be OK too.
OK, one less problem to worry.

When ZAP doesn't have any authentication credentials it simply forwards the 
responses to the client, which seems to be the second case. What client 
application are you using to authenticate?

In what cases does that happen?

Original comment by THC...@gmail.com on 20 Feb 2013 at 12:08

[GoogleCodeExporter]

I am sorry I didn't understand your question. What do you mean by "What client 
application�" and "In what case does that happen�." ???

Original comment by prasad.s...@gmail.com on 20 Feb 2013 at 4:30

[GoogleCodeExporter]

With the "client application" I want to know what was the application that was 
proxying through ZAP (in the second case, log submission #2).

The cases that you said that the Domain part is not handled correctly.

Original comment by THC...@gmail.com on 25 Feb 2013 at 5:00

[GoogleCodeExporter]

It was a browser (IE7) loading an enterprise app supporting NTLM. In case of 
Log#2, I manually added the Proxy Support header for it to work. Again, same 
app and same settings. 

The issue with Domain part not being handled might be more clear when you see 
the logs. I supposes its not handled correctly when we simply rely on ZAP to 
handle NTLM. Working on getting the logs.....:)

Original comment by prasad.s...@gmail.com on 25 Feb 2013 at 2:25

[GoogleCodeExporter]

OK.

Original comment by THC...@gmail.com on 27 Feb 2013 at 3:43

[GoogleCodeExporter]

Has this been tested in an up-to-date version of ZAP?

This issue is getting pretty stale, if it isn't updated by the requester or 
owner by 20140719 it will be closed.

Original comment by kingtho...@gmail.com on 13 Jul 2014 at 8:15

[GoogleCodeExporter]

Closing InsufficientEvidence.

Original comment by kingtho...@gmail.com on 10 Aug 2014 at 7:01

• Changed state: InsufficientEvidence