STATUS_LOGON_FAILURE - Githubissues

DragosFlorea commented 8 months ago

image (1)

Hi I'm using this library in a multithread application with multiple connections. Currently we manage a pool of like 10 sessions and we will reuse them for all the requests. The sessions are used all time and should not get into idle. Do you know if these sessions have a expiration date... because I didn't see and information about that We need to manage ourselves these sessions expirations and recreate them periodically?

Thanks

adiroiban commented 8 months ago

Thanks for the report.

Can you please share the source code that can be used to reproduce this error?

A Short, Self Contained, Correct, Example would be awesome.

Please consider adding a traceback to the erors.

From the screenshot I can't see any traceback into smbprotocol code. It's very hard to troubleshoot this.

Do you use the smbclient high level API or the low-level smbprotocol API ?

DragosFlorea commented 7 months ago

Hi, coming back with stacktrace Looking in the logs I found 2 errors and both looks related to auth to smb server

Error 1: Received unexpected status from the server: The attempted logon is invalid. This is either due to a bad username or authentication information. (3221225581) STATUS_LOGON_FAILURE: 0xc000006d`

traceback (most recent call last): File "/src/services/operations_service.py", line 57, in download multipart_total = math.ceil(self.objects_service.get_size_of_file(file_path, connection_cache=session_info.connection_cache) / CHUNK_SIZE) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/services/objects_service.py", line 115, in get_size_of_file return stat(path=file_path, connection_cache=connection_cache).st_size ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/smbclient/_os.py", line 576, in stat raw = SMBRawIO( ^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/smbclient/_io.py", line 362, in __init__ tree, fd_path = get_smb_tree(path, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/smbclient/_pool.py", line 304, in get_smb_tree session = register_session( ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/smbclient/_pool.py", line 422, in register_session session.connect() File "/usr/local/lib/python3.12/site-packages/smbprotocol/session.py", line 300, in connect response = self.connection.receive(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/smbprotocol/connection.py", line 1095, in receive raise SMBResponseException(response) smbprotocol.exceptions.LogonFailure: Received unexpected status from the server: The attempted logon is invalid. This is either due to a bad username or authentication information. (3221225581) STATUS_LOGON_FAILURE: 0xc000006d

Same context To be able to mitigate https://github.com/jborean93/smbprotocol/issues/219, I've created a pool of 10 Session that will be reused for all the requests that are done for the same user/pass share combination. These sessions will passed to every action on the share read/write/rename/delete as connection_cache These sessions have a expiration date? because I didn't find one...

Error 2: Failed to authenticate with server: SpnegoError (1): SpnegoError (16): Operation not supported or available, Context: Retrieving NTLM store without NTLM_USER_FILE set to a filepath, Context: Unable to negotiate common mechanism

Unfortunately i do not have a stacktrace for this,,, The context for the second error is the same with Error 1, Sometimes I'm getting this error I see there is also this https://github.com/jborean93/smbprotocol/issues/272 but not sure if apply to my situation as well...

Thanks

adiroiban commented 7 months ago

I am not familiar with the design of the high level smbclient API.

By reading the code of smbclient, I can see there is not implicit error recovering.

I guess that it works as expected.

Looking at the general functionality of smbclient API, my guess is that you are expected to catch the smbprotocol.exceptions.LogonFailure and retry / recover from errors.

Errors can happen at login, or in the middle of a transfer, and smbclient will not automatically try to recover from these errors.

From what I can see in the current API, smbclient API is not a transfer manager.

I know that APIs like boto3 can receive a TransferConfig and can automatically retry on errors, but I don't think that the current API for smbclient is expected to handle retries and error recovery.

Have you found somewhere in the smbcliet API in which auto-recovery is documented as something that will be handled by smbclient ?

DragosFlorea commented 7 months ago

I do not expect auto recovery, I want to understand where the issue is because based on the following snipped from smbClient, passing a connection cache should not create a new session And for sure I pass a connection cache

Question is why is not found here next((s for s in connection.session_table.values()

The only explanation is that somehow disconnects without a reason and will enter in this if if not connection or not connection.transport.connected: connection = Connection(ClientConfig().client_guid, server, port, require_signing=require_signing) connection.connect(timeout=connection_timeout) connection_cache[connection_key] = connection

which in the end will create a new session

adiroiban commented 7 months ago

I guess that when the connection is lost, the previous session is removed ... see last line

https://github.com/jborean93/smbprotocol/blob/459100718eb6f54e6b7f90df6bb94f1a139e9b33/src/smbprotocol/session.py#L397-L426

Do you have smbprotocol logging enabled , are the any clues there ?

A self contained example would be best, so that others can reproduce and investiage :)

jborean93 commented 7 months ago

Do you know if these sessions have a expiration date

AFAIK they do have an expiration but it is controlled by the server and when expired they return STATUS_NETWORK_SESSION_EXPIRED as per https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/717222e4-72c6-40e7-9121-a9646d874058. I have not attempted to implement any of the re-authentication scenarios in this library right now so it would require the session to be closed and re-created like a new one.

As for how long the expiration is it seems like https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/9c882bcd-99f0-46c9-8a35-3f09d0e75699 and the 210 footnote indicates it's 45 seconds.

I've tried to replicate this scenario by running the below but it's not expiring for me so there's something more to the expiration from what I can tell.

import time

import smbclient

print(smbclient.listdir(r"\\server2022.domain.test\c$\temp"))

time.sleep(60)

print(smbclient.listdir(r"\\server2022.domain.test\c$\temp"))

Error 2: Failed to authenticate with server: SpnegoError (1): SpnegoError (16): Operation not supported or available, Context: Retrieving NTLM store without NTLM_USER_FILE set to a filepath, Context: Unable to negotiate common mechanism

I think it might be related to #272 where there is something that is causing a new session to be created but any explicit credentials being provided are dropped so it tries to use a cached credential which in your case doesn't exist. Without having a reproducer or traceback that will be hard to solve unfortunately.

Question is why is not found here next((s for s in connection.session_table.values()

I'm sorry I'm not 100% sure what your question is here. The logic for re-using a connection/session is as follows

Build a connection key of server:port to identify the connection
See if the connection cache contains this key
- If not create a new connection and add it to the cache with that key
See if the connection contains a session we can use
- If no username is specified then it will just use the first session found on that connection
- If a username is specified then it will only get a session that was created from that user
- If no matching session is found then it will create a new one and store it in the session_table of that connection

The only explanation is that somehow disconnects without a reason and will enter in this if

Even if that's the case I don't believe we have any pro-active cleanup on the connections, the only time a connection is removed from the cache is if someone has explicitly deleted it.

I guess that when the connection is lost, the previous session is removed ... see last line

This is true, each session is scoped specifically to a connection, you cannot share a session with multiple connections as the authentication is tied directly to that socket. Closing a connection, or creating a new one will always result in a new session needing to be authenticated.

STATUS_LOGON_FAILURE

This is a result of an authentication failure as denoted by the server. This could be for any reason such as

User isn't valid
Password isn't valid
Authentication protocol used isn't allowed (newer Windows server can restrict NTLM specifically for SMB)
SPNs aren't correct (we use cifs/servername in the authentication
User is not allowed to logon as a network logon
Some bug in the authentication payload

Unfortunately we are limited to reporting what the server responds back to us, it simply provides the error code and the message is the Win32 message associated with that code. To find out why the server might be rejecting the logon you really need to look into the logs on the server side and see what it reports.

jborean93 / smbprotocol

STATUS_LOGON_FAILURE #273