No caching of username/password for NTLM authentication

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Logon to web site with NTLM authentication and enter username/password
2. new https session will request username/password again
  --> credentials are not cached

What is the expected output?
Credentials (username/password) should be cached, no additional prompt for 
username/password for new https sessions.

What version of the product are you using? On what operating system?
Android 2.3.3
stock rom (2.3.x)
cyanogenmod 7.1

Original issue reported on code.google.com by leitnerr...@gmail.com on 21 Dec 2011 at 2:12

[GoogleCodeExporter]

Original comment by supp.san...@gmail.com on 21 Dec 2011 at 5:50

• Changed state: Started

[GoogleCodeExporter]

tested: 
android <-> NTLM site
android <-> SSL with client certificate required <-> NTLM site 

Credentials are stored. Performed Force Close Of SandroB.
Credentials are still stored.
Deleting Passwords: Settings -> Clear Passwords
Asked for credentials again.

Reasons for additional prompts for username/password: 
- Settings -> Remember passwords = OFF
- username/password is wrong (username:DOMAIN\sandrob password:test)
- sometimes two prompts appears on begining because android makes 
  additional request  if the first one is taking too long 
  (slow import of username/password)
- maybe some special characters that are not proper handled in SandroB...

To be more helpful I need more information. 
Is there also proxy used? Maybe NTLM proxy?
Works login from firefox/ie from PC?

Original comment by supp.san...@gmail.com on 22 Dec 2011 at 11:32

[GoogleCodeExporter]

Login works with Windows Mobile (IE), iPhone and PC (IE).

I have analyzed the log and I have extracted the important parts (see 
attachment):
 https session (125424) is properly authenticated
 response to POST is a redirect (HTTP/1.1 302 Found)
 new session (125428) requires new authorization and new user input 
Remark: The original host name and the original server IP address have been 
replaced in the log.

Original comment by leitnerr...@gmail.com on 23 Dec 2011 at 8:56

Attachments:

• NTLM.log

[GoogleCodeExporter]

I looked at log and something is wrong in NTLM handshake.
Some steps are missing. SandroB sends NTLM type 3 message too soon.
After first 401 there should be also new connection that is not created.
Attached txt file.

I tried to reproduce problem in test environment with no luck.
So I just made some changes to make it more robust.
Change is that resets NTLM state for connection if new header 401 appears 
without type2 challenge.

Changed SandroB is attached so you can try if it works.

There should be someting like that in android logcat:

D/NTLM(837): MSG_TYPE1_GENERATED192.168.1.111;80;/test/test1.asp
D/IdleCache(837):  IdleCache size 0 host http://192.168.1.111:80
D/NTLM(837): MSG_TYPE3_GENERATED192.168.1.111;80;/test/test1.asp
D/IdleCache(837): Reused conn  0 192.168.1.111;80;/test/test1.asp

Original comment by supp.san...@gmail.com on 24 Dec 2011 at 1:08

Attachments:

• ntlm_diag.txt

[GoogleCodeExporter]

Thank you for your quick response.
I have installed the sandrob version above and enhanced the log file an the 
server with time stamps and session start and termination entries. The sessions 
are https and the server requests a certificate from the client.
The situation is as follows:
During the first NTLM logon there is no username/password prompt. For further 
web request there were 3 username/password promts (see detailed log in the 
attached file). If you are looking at the time stamps, you can estimate where 
user input was required.

Original comment by leitnerr...@gmail.com on 27 Dec 2011 at 9:05

Attachments:

• 2011-12-27-Log.zip

[GoogleCodeExporter]

I see some problems in mixing responses. 
For example 75/76 connections.
75 should be closed after receiving 401 but it still remains and it is used for 
type3 response instead of 76.

I will try to reproduce problem and resolve it.
Maybe just make it sure that 75 like connections are not floating around.

http://davenport.sourceforge.net/ntlm.html#ntlmHttpAuthentication

Original comment by supp.san...@gmail.com on 27 Dec 2011 at 11:30

[GoogleCodeExporter]

connection is closed if 401/407 is received for NTLM.
Please try if it works.

Original comment by supp.san...@gmail.com on 28 Dec 2011 at 11:35

[GoogleCodeExporter]

I tried again using the program version above. It is much better now but there 
are still some username/password prompts. At the end of the two attached log 
files there were these prompts (the second log file continues the first one, no 
log entries are missing).
Remark:
If I switch the web page to http I do not face any username/password prompts.

Original comment by leitnerr...@gmail.com on 29 Dec 2011 at 9:52

Attachments:

• 2011-12-29-Log.zip

[GoogleCodeExporter]

Some other problems is seen.
2022/2023, 1992/1993 connections.
2023 should not be created and used for type3 response instead 2022.

I will try to find and fix it.  
Thank you for logs. They are very helpful.

Original comment by supp.san...@gmail.com on 30 Dec 2011 at 9:59

[GoogleCodeExporter]

Please try with this one.
I changed some settings about connection pooling and reusing.

Original comment by supp.san...@gmail.com on 1 Jan 2012 at 2:05

[GoogleCodeExporter]

I have tried the latest version. It works perfectly! No more prompts for 
username/password!
I have added the log in case you are interested in.
Just another 2 questions:
- Why are the connections closed by the browser rather soon?
The performance would be increased significantly if SSL setup, client 
certificate checking and NTLM authentication would not be necessary for almost 
every web request. Reusing connections with a timeout of about 30 to 60 seconds 
would be very helpful.
- When will this enhanced version be part of the standard distribution?
Thank you for your support and a Happy New Year!

Original comment by leitnerr...@gmail.com on 2 Jan 2012 at 3:25

Attachments:

• 2012-01-02-Log.zip

[GoogleCodeExporter]

Timeout is increased to 12 seconds instead of 6 as is in original android 
source.
Maybe there are some mobile provider reasons that it is not longer.

https://github.com/CyanogenMod/android_frameworks_base/blob/gingerbread/core/jav
a/android/net/http/IdleCache.java#L46
...
/* six second timeout for connections */
private final static int TIMEOUT = 6 * 1000;
...

This means if nobody is using connection it will be dropped from cache.
But if you are using connection it can stays longer. Every time when it is put 
to the cache idle timeout is set.

Connection is dropped:
- if server decide to create new NTLM hanshake. -> Sends initial NTLM header.
- some SSL timeouts for symmetric key renegotiation
- there is no http header for keep-alive
  https://github.com/CyanogenMod/android_frameworks_base/blob/gingerbread/core/java/android/net/http/Connection.java#L513
- ...

I'm glad that you find it working properly.
I must test it now with NTLM/Basic/Digest proxy/web_servers that all 
combinations are working.
Then I will create new version of SandroB for 2.2/2.3 and put it to the Market.

Happy New Year to you to!
And thank you for providing logs. They were very helpful.

Original comment by supp.san...@gmail.com on 2 Jan 2012 at 5:24

[GoogleCodeExporter]

During testing I found out that one combination is not working okey.
android <-> digest proxy (squid) <-> ntlm site on plain http 
But this is not working even in FF or IE.
It is not working either with current version on the Market so I will open new 
issue for that.

Original comment by supp.san...@gmail.com on 4 Jan 2012 at 9:42

[GoogleCodeExporter]

Version for 2.3.x is on the Market.
Still need to implement fix for version 2.2.x

Original comment by supp.san...@gmail.com on 6 Jan 2012 at 5:40

[GoogleCodeExporter]

Version for 2.2.x is on the Market

Original comment by supp.san...@gmail.com on 7 Jan 2012 at 9:56

• Changed state: Fixed