Closed maciejwie closed 1 year ago
Hmm I don't think this is the first time I've heard about it but I'm unsure if this would be the right way to go about it. If Windows/SSPI always sets the workstation maybe we should too. It also doesn't help that this feature would be for NTLM only.
I guess I'm more interested as to whether it's the workstation
field in the negotiate message or the authentication message. I can certainly see some example NTLM tokens I've generated from SSPI without the workstation field set.
For what it's worth, I patched my local pyspnego
to always set the workstation
field as a workaround so I'm not sure if this PR is the way to do it either. My thinking was was pyspnego
has a mechanism for forcing the field to be present, so it shouldn't be the one changing, and only noticed later that you have authored both which makes choosing the "more correct" place a lot easier. I defer to your thoughts here.
Capturing the logs at DEBUG level, it looks like the 2nd SMB2_SESSION_SETUP message to me. I've redacted the step input an output to avoid leaking things when I don't understand the format, but can pull things out if it will help. In this capture, the auth_protocol
is the default negotiate
and running in a Debian Docker container.
INFO:smbprotocol.connection:Negotiated dialect: (514) SMB_2_0_2
INFO:smbprotocol.connection:Connection require signing: True
INFO:smbprotocol.session:Initialising session with username: MY_DOMAIN\MY_USERNAME
DEBUG:smbprotocol.session:Decoding SPNEGO token containing supported auth mechanisms
DEBUG:spnego._negotiate:SPNEGO step input: [step input 1 redacted]
DEBUG:spnego._negotiate:Attempting to create ntlm context when building SPNEGO mech list
DEBUG:spnego._negotiate:SPNEGO step output: [step output 1 redacted]
INFO:smbprotocol.session:Sending SMB2_SESSION_SETUP request message
INFO:smbprotocol.session:Receiving SMB2_SESSION_SETUP response message
DEBUG:smbprotocol.transport:Socket recv() returned 4 bytes (total 4)
DEBUG:smbprotocol.transport:Socket recv(357) (total 357)
DEBUG:smbprotocol.transport:Socket recv() returned 357 bytes (total 357)
DEBUG:smbprotocol.transport:Socket recv(4) (total 4)
INFO:smbprotocol.session:More processing is required for SMB2_SESSION_SETUP
DEBUG:spnego._negotiate:SPNEGO step input: [step input 2 redacted]
DEBUG:spnego._negotiate:SPNEGO step output: [step output 2 redacted]
INFO:smbprotocol.session:Sending SMB2_SESSION_SETUP request message
INFO:smbprotocol.session:Receiving SMB2_SESSION_SETUP response message
DEBUG:smbprotocol.transport:Socket recv() returned 4 bytes (total 4)
DEBUG:smbprotocol.transport:Socket recv(73) (total 73)
DEBUG:smbprotocol.transport:Socket recv() returned 73 bytes (total 73)
DEBUG:smbprotocol.transport:Socket recv(4) (total 4)
The response message returns a failure.
In Windows, the logs look very similar, go further and result in success:
INFO:smbprotocol.connection:Negotiated dialect: (514) SMB_2_0_2
INFO:smbprotocol.connection:Connection require signing: True
INFO:smbprotocol.session:Initialising session with username: MY_DOMAIN\MY_USERNAME
DEBUG:smbprotocol.session:Decoding SPNEGO token containing supported auth mechanisms
DEBUG:spnego._sspi:SSPI step input: [same as step input 1 redacted above]
DEBUG:spnego._sspi:SSPI step output: [slightly different than step output 1 redacted above]
INFO:smbprotocol.session:Sending SMB2_SESSION_SETUP request message
INFO:smbprotocol.session:Receiving SMB2_SESSION_SETUP response message
DEBUG:smbprotocol.transport:Socket recv() returned 4 bytes (total 4)
DEBUG:smbprotocol.transport:Socket recv(357) (total 357)
DEBUG:smbprotocol.transport:Socket recv() returned 357 bytes (total 357)
DEBUG:smbprotocol.transport:Socket recv(4) (total 4)
INFO:smbprotocol.session:More processing is required for SMB2_SESSION_SETUP
DEBUG:spnego._sspi:SSPI step input: [very similar to step input 2 redacted above, but different in two areas]
DEBUG:spnego._sspi:SSPI step output: [different in contents and length than step output 2 above]
INFO:smbprotocol.session:Sending SMB2_SESSION_SETUP request message
INFO:smbprotocol.session:Receiving SMB2_SESSION_SETUP response message
DEBUG:smbprotocol.transport:Socket recv() returned 4 bytes (total 4)
DEBUG:smbprotocol.transport:Socket recv(101) (total 101)
DEBUG:smbprotocol.transport:Socket recv() returned 101 bytes (total 101)
DEBUG:smbprotocol.transport:Socket recv(4) (total 4)
DEBUG:spnego._sspi:SSPI step input: [step input 3 redacted]
DEBUG:spnego._sspi:SSPI step output:
INFO:smbprotocol.session:Setting session id to 3752342914529755392
INFO:smbprotocol.session:Verifying the SMB Setup Session signature as auth is successful
If there is anything from my side you'd like to see or for me to test, please let me know.
At least when using the pure Python NTLM provider in pyspnego the workstation
field in the Authenticate
message is only applied if the server has responded with the Version
flag in it's challenge message
There's no way to control this flag as it's controlled by input from the server. When reading the NTLM spec I think I was confused by the statement in 3.1.5.1.2 Client Receives a CHALLENGE_MESSAGE from the Server
If the NTLMSSP_NEGOTIATE_VERSION flag is set by the client application, the Version field MUST be set to th11302e current version (section 2.2.2.10), and the Workstation field MUST be set to NbMachineName. Otherwise, if the NTLMSSP_NEGOTIATE_VERSION flag is not set by the client application, the Version field MUST be set to all-zero.
I interpreted it as the version flag from the server but this would indicate it's maybe a client setting. The question is whether always sending the workstation will cause other problems. I'm pretty sure this might affect the Log on To option.
The pyspnego library has a builtin tool to help parse the authentication values you've redacted. You can do
python -m spnego --token base64value
You can use this to compare the NTLM tokens exchanged and see if there is any other noticeable changes.
Fantastic, the parser is very helpful and shows the Version
flag differences show up in NEGOTIATE_MESSAGE (1)
.
In both cases, step input 1 is the same:
"MessageType": "SPNEGO InitialContextToken",
"Data": {
"thisMech": "SPNEGO (1.3.6.1.5.5.2)",
"innerContextToken": {
"MessageType": "SPNEGO NegTokenInit2",
"Data": {
"mechTypes": [
"MS Kerberos (1.2.840.48018.1.2.2)",
"NTLM (1.3.6.1.4.1.311.2.2.10)"
],
"reqFlags": null,
"mechToken": null,
"mechListMIC": null,
"negHints": {
"hintName": "HOST/HOSTNAME.DOMAIN.COM@DOMAIN.COM",
"hintAddress": null
}
},
"RawData": "[...]"
}
},
"RawData": [...]
}
With SSPI, NTLMSSP_NEGOTIATE_VERSION
and NTLMSSP_NEGOTIATE_LM_KEY
are set, and Version
is populated, plus some of the offsets are different (larger in a smaller packet with fewer flags?):
{
"MessageType": "SPNEGO InitialContextToken",
"Data": {
"thisMech": "SPNEGO (1.3.6.1.5.5.2)",
"innerContextToken": {
"MessageType": "SPNEGO NegTokenInit",
"Data": {
"mechTypes": [
"NTLM (1.3.6.1.4.1.311.2.2.10)"
],
"reqFlags": null,
"mechToken": {
"MessageType": "NEGOTIATE_MESSAGE (1)",
"Data": {
"NegotiateFlags": {
"raw": 3792208567,
"flags": [
"NTLMSSP_NEGOTIATE_56 (2147483648)",
"NTLMSSP_NEGOTIATE_KEY_EXCH (1073741824)",
"NTLMSSP_NEGOTIATE_128 (536870912)",
"NTLMSSP_NEGOTIATE_VERSION (33554432)",
"NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY (524288)",
"NTLMSSP_NEGOTIATE_ALWAYS_SIGN (32768)",
"NTLMSSP_NEGOTIATE_NTLM (512)",
"NTLMSSP_NEGOTIATE_LM_KEY (128)",
"NTLMSSP_NEGOTIATE_SEAL (32)",
"NTLMSSP_NEGOTIATE_SIGN (16)",
"NTLMSSP_REQUEST_TARGET (4)",
"NTLMSSP_NEGOTIATE_OEM (2)",
"NTLMSSP_NEGOTIATE_UNICODE (1)"
]
},
"DomainNameFields": {
"Len": 0,
"MaxLen": 0,
"BufferOffset": 0
},
"WorkstationFields": {
"Len": 0,
"MaxLen": 0,
"BufferOffset": 0
},
"Version": {
"Major": 10,
"Minor": 0,
"Build": 19041,
"Reserved": "000000",
"NTLMRevision": 15
},
"Payload": {
"DomainName": null,
"Workstation": null
}
},
"RawData": "[...]"
},
"mechListMIC": null
},
"RawData": "[...]"
}
},
"RawData": "[...]"
}
vs spnego:
{
"MessageType": "SPNEGO InitialContextToken",
"Data": {
"thisMech": "SPNEGO (1.3.6.1.5.5.2)",
"innerContextToken": {
"MessageType": "SPNEGO NegTokenInit",
"Data": {
"mechTypes": [
"NTLM (1.3.6.1.4.1.311.2.2.10)"
],
"reqFlags": null,
"mechToken": {
"MessageType": "NEGOTIATE_MESSAGE (1)",
"Data": {
"NegotiateFlags": {
"raw": 3758654007,
"flags": [
"NTLMSSP_NEGOTIATE_56 (2147483648)",
"NTLMSSP_NEGOTIATE_KEY_EXCH (1073741824)",
"NTLMSSP_NEGOTIATE_128 (536870912)",
"NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY (524288)",
"NTLMSSP_NEGOTIATE_ALWAYS_SIGN (32768)",
"NTLMSSP_NEGOTIATE_NTLM (512)",
"NTLMSSP_NEGOTIATE_SEAL (32)",
"NTLMSSP_NEGOTIATE_SIGN (16)",
"NTLMSSP_REQUEST_TARGET (4)",
"NTLMSSP_NEGOTIATE_OEM (2)",
"NTLMSSP_NEGOTIATE_UNICODE (1)"
]
},
"DomainNameFields": {
"Len": 0,
"MaxLen": 0,
"BufferOffset": 32
},
"WorkstationFields": {
"Len": 0,
"MaxLen": 0,
"BufferOffset": 32
},
"Version": null,
"Payload": {
"DomainName": null,
"Workstation": null
}
},
"RawData": "[...]"
},
"mechListMIC": null
},
"RawData": "[...]"
}
},
"RawData": "[...]"
}
The NTLMSSP_NEGOTIATE_VERSION
flag being set in SSPI is the only difference for CHALLENGE_MESSAGE (2)
, but things diverge for AUTHENTICATE_MESSAGE (3)
.
SSPI:
{
"MessageType": "SPNEGO NegTokenResp",
"Data": {
"negState": "accept-incomplete (1)",
"supportedMech": null,
"responseToken": {
"MessageType": "AUTHENTICATE_MESSAGE (3)",
"Data": {
"LmChallengeResponseFields": {
"Len": 24,
"MaxLen": 24,
"BufferOffset": 154
},
"NtChallengeResponseFields": {
"Len": 328,
"MaxLen": 328,
"BufferOffset": 178
},
"DomainNameFields": {
"Len": 30,
"MaxLen": 30,
"BufferOffset": 88
},
"UserNameFields": {
"Len": 10,
"MaxLen": 10,
"BufferOffset": 118
},
"WorkstationFields": {
"Len": 26,
"MaxLen": 26,
"BufferOffset": 128
},
"EncryptedRandomSessionKeyFields": {
"Len": 16,
"MaxLen": 16,
"BufferOffset": 506
},
"NegotiateFlags": {
"raw": 3800597045,
"flags": [
"NTLMSSP_NEGOTIATE_56 (2147483648)",
"NTLMSSP_NEGOTIATE_KEY_EXCH (1073741824)",
"NTLMSSP_NEGOTIATE_128 (536870912)",
"NTLMSSP_NEGOTIATE_VERSION (33554432)",
"NTLMSSP_NEGOTIATE_TARGET_INFO (8388608)",
"NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY (524288)",
"NTLMSSP_NEGOTIATE_ALWAYS_SIGN (32768)",
"NTLMSSP_NEGOTIATE_NTLM (512)",
"NTLMSSP_NEGOTIATE_SEAL (32)",
"NTLMSSP_NEGOTIATE_SIGN (16)",
"NTLMSSP_REQUEST_TARGET (4)",
"NTLMSSP_NEGOTIATE_UNICODE (1)"
]
},
"Version": {
"Major": 10,
"Minor": 0,
"Build": 19041,
"Reserved": "000000",
"NTLMRevision": 15
},
"MIC": "43265BEDA5A239190F1D63E6CFFA9D38",
"Payload": {
"LmChallengeResponse": {
"ResponseType": "LMv2",
"LMProofStr": "00000000000000000000000000000000",
"ChallengeFromClient": "0000000000000000"
},
"NtChallengeResponse": {
"ResponseType": "NTLMv2",
"NTProofStr": "1799B600CFE10D89664EE77E5CD935EB",
"ClientChallenge": {
"RespType": 1,
"HiRespType": 1,
"Reserved1": 0,
"Reserved2": 0,
"TimeStamp": "2023-06-07T21:44:57Z",
"ChallengeFromClient": "CEA27762517C6F22",
"Reserved3": 0,
"AvPairs": [
{
"AvId": "MSV_AV_NB_COMPUTER_NAME (1)",
"Value": "HOSTNAME"
},
{
"AvId": "MSV_AV_NB_DOMAIN_NAME (2)",
"Value": "MY_DOMAIN"
},
{
"AvId": "MSV_AV_DNS_COMPUTER_NAME (3)",
"Value": "HOST.DOMAIN.COM"
},
{
"AvId": "MSV_AV_DNS_DOMAIN_NAME (4)",
"Value": "DOMAIN.COM"
},
{
"AvId": "MSV_AV_DNS_TREE_NAME (5)",
"Value": "DOMAIN.COM"
},
{
"AvId": "MSV_AV_TIMESTAMP (7)",
"Value": "2023-06-07T21:44:57Z"
},
{
"AvId": "MSV_AV_FLAGS (6)",
"Value": {
"raw": 2,
"flags": [
"MIC_PROVIDED (2)"
]
}
},
{
"AvId": "MSV_AV_SINGLE_HOST (8)",
"Value": {
"Size": 48,
"Z4": 0,
"CustomData": "0100000000200000",
"MachineId": "[...]"
}
},
{
"AvId": "MSV_AV_CHANNEL_BINDINGS (10)",
"Value": "00000000000000000000000000000000"
},
{
"AvId": "MSV_AV_TARGET_NAME (9)",
"Value": "cifs/10.10.10.10"
},
{
"AvId": "MSV_AV_EOL (0)",
"Value": null
}
],
"Reserved4": 0
}
},
"DomainName": "MY_DOMAIN",
"UserName": "MY_USERNAME",
"Workstation": "MY_MACHINE",
"EncryptedRandomSessionKey": "AF237C4A670A3DA5AE2C02FF364B9F54"
},
"SessionKey": "Failed to derive"
},
"RawData": "[...]"
},
"mechListMIC": "0100000082F3961F44F299A800000000"
},
"RawData": "[...]"
}
spnego:
{
"MessageType": "SPNEGO NegTokenResp",
"Data": {
"negState": "accept-incomplete (1)",
"supportedMech": null,
"responseToken": {
"MessageType": "AUTHENTICATE_MESSAGE (3)",
"Data": {
"LmChallengeResponseFields": {
"Len": 24,
"MaxLen": 24,
"BufferOffset": 80
},
"NtChallengeResponseFields": {
"Len": 256,
"MaxLen": 256,
"BufferOffset": 104
},
"DomainNameFields": {
"Len": 30,
"MaxLen": 30,
"BufferOffset": 360
},
"UserNameFields": {
"Len": 10,
"MaxLen": 10,
"BufferOffset": 390
},
"WorkstationFields": {
"Len": 0,
"MaxLen": 0,
"BufferOffset": 400
},
"EncryptedRandomSessionKeyFields": {
"Len": 16,
"MaxLen": 16,
"BufferOffset": 400
},
"NegotiateFlags": {
"raw": 3767173653,
"flags": [
"NTLMSSP_NEGOTIATE_56 (2147483648)",
"NTLMSSP_NEGOTIATE_KEY_EXCH (1073741824)",
"NTLMSSP_NEGOTIATE_128 (536870912)",
"NTLMSSP_NEGOTIATE_TARGET_INFO (8388608)",
"NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY (524288)",
"NTLMSSP_TARGET_TYPE_SERVER (131072)",
"NTLMSSP_NEGOTIATE_ALWAYS_SIGN (32768)",
"NTLMSSP_NEGOTIATE_NTLM (512)",
"NTLMSSP_NEGOTIATE_SIGN (16)",
"NTLMSSP_REQUEST_TARGET (4)",
"NTLMSSP_NEGOTIATE_UNICODE (1)"
]
},
"Version": null,
"MIC": "0A8CAA25EF81FB079229C67E6E87D694",
"Payload": {
"LmChallengeResponse": {
"ResponseType": "LMv2",
"LMProofStr": "00000000000000000000000000000000",
"ChallengeFromClient": "0000000000000000"
},
"NtChallengeResponse": {
"ResponseType": "NTLMv2",
"NTProofStr": "A6A26E4D08E8C5D7F33AF06878860D32",
"ClientChallenge": {
"RespType": 1,
"HiRespType": 1,
"Reserved1": 0,
"Reserved2": 0,
"TimeStamp": "2023-06-07T21:40:41Z",
"ChallengeFromClient": "B046DA7FFB9635A4",
"Reserved3": 0,
"AvPairs": [
{
"AvId": "MSV_AV_NB_COMPUTER_NAME (1)",
"Value": "HOSTNAME"
},
{
"AvId": "MSV_AV_NB_DOMAIN_NAME (2)",
"Value": "MY_DOMAIN"
},
{
"AvId": "MSV_AV_DNS_COMPUTER_NAME (3)",
"Value": "HOST.DOMAIN.COM"
},
{
"AvId": "MSV_AV_DNS_DOMAIN_NAME (4)",
"Value": "DOMAIN.COM"
},
{
"AvId": "MSV_AV_DNS_TREE_NAME (5)",
"Value": "DOMAIN.COM"
},
{
"AvId": "MSV_AV_TIMESTAMP (7)",
"Value": "2023-06-07T21:40:41Z"
},
{
"AvId": "MSV_AV_TARGET_NAME (9)",
"Value": "cifs/10.10.10.10"
},
{
"AvId": "MSV_AV_FLAGS (6)",
"Value": {
"raw": 2,
"flags": [
"MIC_PROVIDED (2)"
]
}
},
{
"AvId": "MSV_AV_EOL (0)",
"Value": null
}
],
"Reserved4": 0
}
},
"DomainName": "MY_DOMAIN",
"UserName": "MY_USERNAME",
"Workstation": null,
"EncryptedRandomSessionKey": "0A50DBE97C5833F06879FA5F8EB9F937"
},
"SessionKey": "Failed to derive"
},
"RawData": "[...]"
},
"mechListMIC": "01000000A2B7F127E072E2BE00000000"
},
"RawData": "[...]"
}
At least when using the pure Python NTLM provider in pyspnego the
workstation
field in theAuthenticate
message is only applied if the server has responded with theVersion
flag in it's challenge message
Indeed, my local workaround was to replace this condition with if True:
. Looking at the code now, I could have also tried to have set the default flags in https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/src/spnego/_ntlm.py#L263 so it's interesting that what I did was not set the flag but include the field anyway.
There's no way to control this flag as it's controlled by input from the server. When reading the NTLM spec I think I was confused by the statement in 3.1.5.1.2 Client Receives a CHALLENGE_MESSAGE from the Server
If the NTLMSSP_NEGOTIATE_VERSION flag is set by the client application, the Version field MUST be set to th11302e current version (section 2.2.2.10), and the Workstation field MUST be set to NbMachineName. Otherwise, if the NTLMSSP_NEGOTIATE_VERSION flag is not set by the client application, the Version field MUST be set to all-zero.
I interpreted it as the version flag from the server but this would indicate it's maybe a client setting. The question is whether always sending the workstation will cause other problems. I'm pretty sure this might affect the Log on To option.
In this case, it does look like it's the client setting (SSPI) or not setting (spnego) the flag in the step output.
The NTLMSSP_NEGOTIATE_VERSION flag being set in SSPI is the only difference for CHALLENGE_MESSAGE (2),
I wonder if you were to change this line in your local pyspnego and see if it's enough to get things working
Change it to self._temp_negotiate = Negotiate(self._context_req, version=Version.get_current())
. It seems like your server is sending the challenge response with the version flag only if the negotiate message also contained the version flag. Once the server replies with this flag then the authentication message should now include the version as well as the workstation message.
I wonder if you were to change this line in your local pyspnego and see if it's enough to get things working
It was, and that's the cleanest solution so far as long as it doesn't break anything else.
Thanks for confirming, I've opened https://github.com/jborean93/pyspnego/pull/65 that sets this flag and will push a new release once it has been merged.
Thanks, and I've closed my other PR.
Hello, I'm talking to an older IBM i server which can be picky about what it connects to. In particular, it rejects connections where the workstation name is null. This works on Windows because the underlying SSPI library always sets the
Version
flag and and sends the workstation name. On Linux, thespnego
library checks thein_token
for theVersion
flag and omits those sections if it's not found. In my testing, if I force enable sending that section then everything works.What do you think about adding an ability to pass some parameters through to
spnego
? The "nicest" way would be to take in just the flags, and the more generic way would be to take the encoded bytes of the token itself.