hierynomus / smbj

Server Message Block (SMB2, SMB3) implementation in Java
Other
713 stars 180 forks source link

MacOS Monterey authentication failure. #698

Open isabsent opened 2 years ago

isabsent commented 2 years ago

MacOS Monterey 12.1 for com.hierynomus:smbj versions 0.11.1 and 0.11.3

Error in the method:

public Session establish(AuthenticationContext authContext) {
    try {
        Authenticator authenticator = getAuthenticator(authContext);
        BuilderContext ctx = newContext(authContext, authenticator);

        authenticator.init(config);
        processAuthenticationToken(ctx, connectionContext.getGssNegotiateToken());

        Session session = setupSession(ctx);
        logger.info("Successfully authenticated {} on {}, session is {}", authContext.getUsername(), connection.getRemoteHostname(), session.getSessionId());
        sessionTable.registerSession(session.getSessionId(), session);
        return session;
    } catch (SpnegoException | IOException e) {
        throw new SMBRuntimeException(e);
    }
}

on the string Session session = setupSession(ctx)

    com.hierynomus.mssmb2.SMBApiException: STATUS_LOGON_FAILURE (0xc000006d): Authentication failed for 'lev' using com.hierynomus.smbj.auth.NtlmAuthenticator@5702f0b
    at com.hierynomus.smbj.connection.SMBSessionBuilder.setupSession(SMBSessionBuilder.java:144)
    at com.hierynomus.smbj.connection.SMBSessionBuilder.setupSession(SMBSessionBuilder.java:142)
    at com.hierynomus.smbj.connection.SMBSessionBuilder.establish(SMBSessionBuilder.java:109)
    at com.hierynomus.smbj.connection.Connection.authenticate(Connection.java:202)

ConnectionContext{
  serverGuid=c77ee07e-ce60-0751-b571-0be29c955039,
  serverName='192.168.10.3',
  negotiatedProtocol=NegotiatedProtocol{dialect=SMB_3_0_2, maxTransactSize=4194304, maxReadSize=4194304, maxWriteSize=4194304},
  clientGuid=d2f28808-7cd9-4ecc-94d9-2070b6f2062c,
  clientCapabilities=[SMB2_GLOBAL_CAP_LARGE_MTU],
  serverCapabilities=[SMB2_GLOBAL_CAP_LEASING, SMB2_GLOBAL_CAP_LARGE_MTU, SMB2_GLOBAL_CAP_ENCRYPTION],
  clientSecurityMode=1,
  serverSecurityMode=3,
  server='com.hierynomus.smbj.server.Server@8e988bf'
}

authContext = {
     domain = ""
     password = {char[7]@16689} [l, e, v, 1, 9, 6, 7]
     username = "lev"
}

authenticator = {
   completed = false
   initialized = true
   random = {SecureRandom@16659} 
   securityProvider = {BCSecurityProvider@16660} 
   workStationName = null
}

ctx = {
   authContext = {AuthenticationContext@16573} "AuthenticationContext[lev@]"
   authenticator = {NtlmAuthenticator@16641} 
   digest = null
   request = null
   response = null
   securityContext = {byte[66]@16662} [96, 64, 6, 6, 43, 6, 1, 5, 5, 2, -96, 54, 48, 52, -96, 14, 48, 12, 6, 10, 43, 6, 1, 4, 1, -126, 55, 2, 2, 10, -94, 34, 4, 32, 78, 84, 76, 77, 83, 83, 80, 0, 1, 0, 0, 0, 21, -126, -120, -32, 0, 0, 0, 0, 32, 0, 0, 0, 0, 0, 0, 0, 32, 0, 0, 0]
   sessionId = 0
   sessionKey = null
}

config = {SmbConfig@16572} 
   authenticators = {ArrayList@16665}  size = 1
    clientGSSContextConfig = {GSSContextConfig@16666} 
    clientGuid = {UUID@16614} "d2f28808-7cd9-4ecc-94d9-2070b6f2062c"
    dfsEnabled = false
    dialects = {RegularEnumSet@16667}  size = 5
    encryptData = false
    random = {SecureRandom@16659} 
    readBufferSize = 1048576
    readTimeout = 60000
    securityProvider = {BCSecurityProvider@16660} 
    signingRequired = false
    soTimeout = 0
    socketFactory = {ProxySocketFactory@16668} 
    transactBufferSize = 1048576
    transactTimeout = 60000
    transportLayerFactory = {DirectTcpTransportFactory@16669} 
    useMultiProtocolNegotiate = false
    workStationName = null
    writeBufferSize = 1048576
    writeTimeout = 60000
 }

As I have found the same error was faced on Big Sur year ago https://github.com/hierynomus/smbj/issues/454#issuecomment-817713480, but the provided solution doesn't work now.

Screenshot 2022-02-23 at 18 44 19 Screenshot 2022-02-23 at 18 43 38

The same code works fine for shared folders on Ubuntu 20.04.

P.S.: com.hierynomus:smbj:0.10.0 doesn't have this problem on MacOS Monterey 12.1! I have found the same situation in this message https://github.com/hierynomus/smbj/issues/478#issuecomment-962856583.

chrisgch commented 2 years ago

A user of my free Android app has reported the same problem when accessing a share on MacOS. For him it fails when connecting to MacOS 10.15.7.

Until recently, I was using a much older version of smbj, a snapshot from 28.8.2017. With this version, connecting to MacOS does work! It used config = SmbConfig.builder().withMultiProtocolNegotiate(true).build(); client = new SMBClient(config);

Then I changed to a current snapshot recently for the SMB3 support, and with this version the connection fails with STATUS_LOGON_FAILURE (0xc000006d). The user tried SMB2 only (builder.withDialects(SMB2Dialect.SMB_2_1, SMB2Dialect.SMB_2_0_2)) and also SMB3 (builder.withDialects(SMB2Dialect.SMB_3_1_1, SMB2Dialect.SMB_3_0_2, SMB2Dialect.SMB_3_0)) and a combination of both, it always failed with the same error.

Maybe a code comparison could give you a clue why it works with the older smbj version but not the latest.

hierynomus commented 2 years ago

Between 2017 and now, quite a number of things have changed. Is there any way of obtaining a pcap file of a working connect and a non-working to analyze the difference?

isabsent commented 2 years ago

Between 2017 and now, quite a number of things have changed. Is there any way of obtaining a pcap file of a working connect and a non-working to analyze the difference?

I will try, but I have no idea what is pcap and how to get it.

hierynomus commented 2 years ago

pcap is a packet capture. There are multiple ways to grab one, depending on your OS.

isabsent commented 2 years ago

pcap is a packet capture. There are multiple ways to grab one, depending on your OS.

Ok, I will try to get it.

chrisgch commented 2 years ago

isabsent: Why did you close this issue? Did you find a solution? The user of my app who has this problem would also appreciate a solution.

isabsent commented 2 years ago

isabsent: Why did you close this issue? Did you find a solution? The user of my app who has this problem would also appreciate a solution.

I have closed it accidentally. I have no the solution yet.

chrisgch commented 2 years ago

I have now made a code comparison between the smbj version from 2017 which works, and the current version which doesn't, and found the following differences in the authentication functions which may have an influence. Unfortunately I don't have access to a Mac so it would be nice if you (isabsent) or anyone else with this problem could test whether one of these changes causes the problem.

  1. mssmb2\messages\SMB2SessionSetup.java new version: buffer.putUInt32(clientCapabilities & 0xFF); // Capabilities (4 bytes) old version: buffer.putUInt32(clientCapabilities & 0x01); // Capabilities (4 bytes) (only last byte can be set)

  2. ntlm\messages\NtlmNegotiate.java new: buffer.putUInt32(0x20); // DomainNameBufferOffset (4 bytes) // WorkstationFields (8 bytes) buffer.putUInt16(0x0); // WorkstationLen (2 bytes) buffer.putUInt16(0x0); // WorkstationMaxLen (2 bytes) buffer.putUInt32(0x20); // WorkstationBufferOffset (4 bytes) old: buffer.putUInt32(0x0); // DomainNameBufferOffset (4 bytes) // WorkstationFields (8 bytes) buffer.putUInt16(0x0); // WorkstationLen (2 bytes) buffer.putUInt16(0x0); // WorkstationMaxLen (2 bytes) buffer.putUInt32(0x0); // WorkstationBufferOffset (4 bytes)

  3. smbj\auth\NtlmAuthenticator.java new: Object msAvTimestamp = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvTimestamp); if (msAvTimestamp != null) { old: Object msvAvFlags = challenge.getAvPairObject(AvId.MsvAvFlags); if (msvAvFlags instanceof Long && ((long) msvAvFlags & 0x00000002) > 0) { you can't use this old code directly because the object names changed. You will need to use something like this: Object msvAvFlags = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvFlags); if (msvAvFlags instanceof Long && ((long) msvAvFlags & 0x00000002) > 0) {

  4. a lot was rearranged in smbj\connection\Connection.java, so if the problem is there, it will be difficult to analyze.

Also there are a lot of protocol negotiations if SMB2Dialect.SMB_3_1_1 is set, so try connecting with SMB2 only: builder.withDialects(SMB2Dialect.SMB_2_1, SMB2Dialect.SMB_2_0_2)

isabsent commented 2 years ago

I will try to check it in this weekend.

chrisgch commented 2 years ago

isabsent: Have you tried my 3 suggested changes in the meantime? They are so small, you can just edit the 3 files by hand and recompile. Unfortunately I don't have access to a Mac, so I can't test it myself. In the meantime, more MacOS users have reported the login failure.

marosseleng commented 2 years ago

@chrisgch I applied first 3 changes you proposed (points 1, 2 and 3) and it started working. The changes, however caused test NegTokenInitSpec."should correctly encode ntlm choice negInitToken" to fail. Afterwards, I tried applying changes one by one with results (in all cases the negotiated protocol dialect is SMB_3_0_2):

  1. "1." only: a. NOT working;
  2. "1." and "2.": a. NOT working;
  3. "1." and "2." and "3.": a. WORKING;
  4. "1." and "3.": a. WORKING;
  5. "3." only: a. WORKING;

TL;DR version would be, that it seems that changes in NtlmAuthenticator seem to make it working again on MacOS 12.3.1.

After applying only change "3" (in NtlmAuthenticator), the failing test is now working allowing for build.

chrisgch commented 2 years ago

Thanks a lot for your tests! So apparently there is no msAvTimestamp field in the authentication with the Mac, but the flags field is there. @hierynomus Any idea of what this could be? Was there a specific reason for switching to the timestamp check? The comment above the changed check still reads: // MIC (16 bytes) provided if in AvPairType is key MsvAvFlags with value & 0x00000002 is true

chrisgch commented 2 years ago

@isabsent: Are you still using MacOS 12.3.1? A user with MacOS 12.4 has reported that the bugfix doesn't work for him.

marosseleng commented 2 years ago

@chrisgch I tried running it on my machine (MacOS 12.4) and the fix seems to be working fine (I haven't tried the original library, though).

L-JINBIN commented 1 year ago

3. smbj\auth\NtlmAuthenticator.java new: Object msAvTimestamp = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvTimestamp); if (msAvTimestamp != null) { old: Object msvAvFlags = challenge.getAvPairObject(AvId.MsvAvFlags); if (msvAvFlags instanceof Long && ((long) msvAvFlags & 0x00000002) > 0) { you can't use this old code directly because the object names changed. You will need to use something like this: Object msvAvFlags = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvFlags); if (msvAvFlags instanceof Long && ((long) msvAvFlags & 0x00000002) > 0) {

It work for me. macOS Ventura 13.1

hierynomus commented 1 year ago

Does the original code work, or the updated for 13.1?

L-JINBIN commented 1 year ago

Does the original code work, or the updated for 13.1?

The original code doesn't work.

Object msAvTimestamp = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvTimestamp); (msAvTimestamp != null) is true

Object msvAvFlags = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvFlags); (msvAvFlags instanceof Long && ((long) msvAvFlags & 0x00000002) > 0) is false

When the code takes the false branch, it works fine.

dkocher commented 1 year ago

Looks like this is a duplicate of #730 with a proposed fix in #731 and changeset c81367c4b35aedd468a75483f1e130ed2d8323c9.