Fails to authenticate in subdir that I have access

[pauloneves]

I'm trying to access a smb folder that my user has access. I can successfully list a parent dir, but it fails to list a subdirectory.

Here is a sample:

import smbclient
smbclient.ClientConfig(username=r'DOMAIN\PPPPP', password='XXX')
smbclient.listdir(r'\\company.net\company\Aplicacoes') # Works! The dir is listed

# but below fails:
smbclient.listdir(r'\\company.net\company\Aplicacoes\Site')

It displays the following error:

SMBAuthenticationError: Failed to authenticate with server: SpnegoError (6): A token had an invalid Message Integrity Check (MIC), Context: Invalid Message integrity Check (MIC) detected

The authenticated user is mine and I can access the directory \\company.net\company\Aplicacoes\Siteusing Windows Explorer.

[jborean93]

This is most certainly a DFS problem with trying to re-authenticate with the DFS target. Things should work but finding out what the problem is might be problematic. Are you able to add the following to your script

import logging

file_log = logging.FileHandler('smb.log')
file_log.setLevel(logging.DEBUG)
file_log.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(message)s'))

for l in ['smbprotocol', 'spnego']:
    logger = logging.getLogger(l)
    logger.setLevel(logging.DEBUG)
    logger.addHandler(file_log)

... your code here

This will produce quite a large log file of all the SMB messages that are exchanged with the server. What will be good to see is the message tokens exchanged in the spnego library. You can use python -m spnego --token <base64 value> to decode those tokens into a more human friendly output. Also seeing the DFS referrals that are exchanged will be quite helpful.

The data in there may contain environment specific information so feel free to either mask the values or send me the logs privately (or analyse them yourself).

smbclient.ClientConfig(username=r'DOMAIN\PPPPP', password='XXX')

I would recommend using the UPN format PPPPP@DOMAIN.COM for the username when dealing with domain accounts. While usually not a problem for NTLM, the UPN format is a lot more explicit and also works for Kerberos if you can set that up in your environment.

[pauloneves]

Thanks for your answer. I don't know what is a DFS Target :-)

I tried to use Kerberos but the behavior was the same. Accessed the parent dir, but not the subdir.

The log is attached now (the upload failed when I originally posted). smb.log

[jborean93]

I don't know what is a DFS Target :-)

Put simply (can DFS ever be simple) it's mapping a bunch of file servers under 1 name, i.e. \\mydomain.com\info\.... It appears as 1 host/endpoint but in reality it's backed by many and it's up to the client to negotiate it.

Putting that aside the error is very weird and not something I expect at all. Part of the authentication phase is to exchange a bunch of NTLM tokens. This is usually done using 3-4 (4 if wrapping with SPNEGO) tokens:

• NTLM Negotiate - sent by the client to negotiate NTLM capabilities
• NTLM Challenge - sent by the server to respond to the negotiation
• NTLM Authenticate - sent by the client to provide the NTLM hashes for verification
• SPNEGO mechListMIC - sent by the server (optional) when wrapping the NTLM exchange through SPNEGO

What's happening here is that the NTLM Challenge message received from the server contains a value for the mechListMIC which is definitely not expected at that point in time. The server shouldn't even have enough information to derive the MIC and when looking at the raw data you can see the MIC is just the copy of the responseToken

MessageType: SPNEGO NegTokenResp
Data:
  negState: accept-incomplete (1)
  supportedMech: NTLM (1.3.6.1.4.1.311.2.2.10)
  responseToken:
    MessageType: CHALLENGE_MESSAGE (2)
    Data:
      TargetNameFields:
        Len: 10
        MaxLen: 10
        BufferOffset: 48
      NegotiateFlags:
        raw: 3767108113
        flags:
        - NTLMSSP_NEGOTIATE_56 (2147483648)
        - NTLMSSP_NEGOTIATE_KEY_EXCH (1073741824)
        - NTLMSSP_NEGOTIATE_128 (536870912)
        - NTLMSSP_NEGOTIATE_TARGET_INFO (8388608)
        - NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY (524288)
        - NTLMSSP_TARGET_TYPE_DOMAIN (65536)
        - NTLMSSP_NEGOTIATE_ALWAYS_SIGN (32768)
        - NTLMSSP_NEGOTIATE_NTLM (512)
        - NTLMSSP_NEGOTIATE_SIGN (16)
        - NTLMSSP_NEGOTIATE_UNICODE (1)
      ServerChallenge: 2EF68D323C82D9BC
      Reserved: '0000000000000000'
      TargetInfoFields:
        Len: 96
        MaxLen: 96
        BufferOffset: 58
      Version:
      Payload:
        TargetName: DOMAIN
        TargetInfo:
        - AvId: MSV_AV_NB_DOMAIN_NAME (2)
          Value: DOMAIN
        - AvId: MSV_AV_NB_COMPUTER_NAME (1)
          Value: NASFS01
        - AvId: MSV_AV_DNS_DOMAIN_NAME (4)
          Value: domain.net
        - AvId: MSV_AV_DNS_COMPUTER_NAME (3)
          Value: nasfs01.domain.net
        - AvId: MSV_AV_EOL (0)
          Value:
    RawData: edited out for brevity
  mechListMIC: same value as responseToken.RawData
RawData: edited out for brevity

Do you have any idea what OS, or just anything in general on, NASFS01? It seems to be returning invalid tokens which is mostly a bug on it's side. I see this behaviour whenever I see the CHALLENGE_MESSAGE from that host, if I was to compare it with another host that is working then the CHALLNEGE_MESSAGE is as follows

MessageType: SPNEGO NegTokenResp
Data:
  negState: accept-incomplete (1)
  supportedMech: NTLM (1.3.6.1.4.1.311.2.2.10)
  responseToken:
    MessageType: CHALLENGE_MESSAGE (2)
    Data:
      TargetNameFields:
        Len: 10
        MaxLen: 10
        BufferOffset: 56
      NegotiateFlags:
        raw: 3800662581
        flags:
        - NTLMSSP_NEGOTIATE_56 (2147483648)
        - NTLMSSP_NEGOTIATE_KEY_EXCH (1073741824)
        - NTLMSSP_NEGOTIATE_128 (536870912)
        - NTLMSSP_NEGOTIATE_VERSION (33554432)
        - NTLMSSP_NEGOTIATE_TARGET_INFO (8388608)
        - NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY (524288)
        - NTLMSSP_TARGET_TYPE_DOMAIN (65536)
        - NTLMSSP_NEGOTIATE_ALWAYS_SIGN (32768)
        - NTLMSSP_NEGOTIATE_NTLM (512)
        - NTLMSSP_NEGOTIATE_SEAL (32)
        - NTLMSSP_NEGOTIATE_SIGN (16)
        - NTLMSSP_REQUEST_TARGET (4)
        - NTLMSSP_NEGOTIATE_UNICODE (1)
      ServerChallenge: E745FFC8EEC9D503
      Reserved: '0000000000000000'
      TargetInfoFields:
        Len: 130
        MaxLen: 130
        BufferOffset: 66
      Version:
        Major: 6
        Minor: 3
        Build: 9600
        Reserved: '000000'
        NTLMRevision: 15
      Payload:
        TargetName: DOMAIN
        TargetInfo:
        - AvId: MSV_AV_NB_DOMAIN_NAME (2)
          Value: DOMAIN
        - AvId: MSV_AV_NB_COMPUTER_NAME (1)
          Value: VRT0600
        - AvId: MSV_AV_DNS_DOMAIN_NAME (4)
          Value: domain.net
        - AvId: MSV_AV_DNS_COMPUTER_NAME (3)
          Value: vrt0600.domain.net
        - AvId: MSV_AV_DNS_TREE_NAME (5)
          Value: domain.net
        - AvId: MSV_AV_TIMESTAMP (7)
          Value: '2021-09-15T14:43:35.8562408Z'
        - AvId: MSV_AV_EOL (0)
          Value:
    RawData: edited out for brevity
  mechListMIC:
RawData: edited out for brevity

Notice how vrt0600 has an empty value for mechListMIC, because of this smbclient is able to continue the exchange. I'm trying to find out more as to whether the auth logic should just ignore the mechListMIC here but in reality the problem lies with the auth token crafted by NAS host. The reason why this is probably not occurring on Windows is because Windows will most likely just be sending the raw NTLM tokens and not have it wrapped in SPNEGO and thus the mechListMIC is not present. This is most likely why the bug has gone unnoticed on the server side software because usually NTLM doesn't come wrapped in SPNEGO.

As a workaround you can follow the same Windows behaviour by specifying smbclient.ClientConfig(..., auth_protocol='ntlm') to force the use of NTLM only without any SPNEGO wrappings.

I'll keep the issue open until I receive further clarifications of what to do in this scenario from some more knowledgable people but in the meantime are you able to try the auth_protocol='ntlm' in your config and hopefully try and track down some more info on NASFS01 if at all possible. Being able to replicate this myself will be very helpful.

[jborean93]

Looks like this logic may be based on Windows Server 2000 behavior and MIT krb5's SPNEGO will explicitly handle that https://github.com/krb5/krb5/blob/3f5a348287646d65700854650fe668b9c4249013/src/lib/gssapi/spnego/spnego_mech.c#L3734-L3744. I still think it's wrong but the precedence has been set so I'll update the code to handle it as well :)

[jborean93]

I've just opened https://github.com/jborean93/pyspnego/pull/17 which fixes this particular scenario and should get things working on the default path.