Open andrewsiemer opened 10 months ago
What is the service you are using here, is it a Windows DFS implementation/something else? I've seen a few reports for this but was never able to figure out what was happening. Typically you see STATUS_FS_DRIVER_REQUIRED
when a DFS referral request was sent to a service that cannot handle that IOCTL code but in this case it seems like the open method. If it's possible it would be great to see the Wireshark trace for this but I can understand if the data is sensitive and cannot be shared.
Yes, sadly this is not a share that I set up. This is the result of smbutil statshares -a
if that's even helpful. Sorry I cannot share much more. Is there any other non-sensitive info I can share?
SERVER_NAME test
USER_ID 501
SMB_NEGOTIATE SMBV_NEG_SMB1_ENABLED
SMB_NEGOTIATE SMBV_NEG_SMB2_ENABLED
SMB_NEGOTIATE SMBV_NEG_SMB3_ENABLED
SMB_VERSION SMB_3.1.1
SMB_ENCRYPT_ALGORITHMS AES_128_CCM_ENABLED
SMB_ENCRYPT_ALGORITHMS AES_128_GCM_ENABLED
SMB_ENCRYPT_ALGORITHMS AES_256_CCM_ENABLED
SMB_ENCRYPT_ALGORITHMS AES_256_GCM_ENABLED
SMB_CURR_ENCRYPT_ALGORITHM OFF
SMB_SHARE_TYPE DISK
SIGNING_SUPPORTED TRUE
EXTENDED_SECURITY_SUPPORTED TRUE
LARGE_FILE_SUPPORTED TRUE
OS_X_SERVER TRUE
FILE_IDS_SUPPORTED TRUE
DFS_SUPPORTED TRUE
FILE_LEASING_SUPPORTED TRUE
MULTI_CREDIT_SUPPORTED TRUE
PERSISTENT_HANDLES_SUPPORTED TRUE
DFS_SHARE TRUE
Thanks for the info, can you share the SMB server implementation, i.e. is it Samba, some NAS box? Can you try out this low level code to see what happens?
import uuid
from smbprotocol.connection import Connection
from smbprotocol.tree import TreeConnect
from smbprotocol.session import Session
from smbprotocol.open import (
Open,
ImpersonationLevel,
FileAttributes,
FilePipePrinterAccessMask,
ShareAccess,
CreateDisposition,
)
server = "server-name"
username = "adminuser"
password = "mypassword"
share = "share"
path = r"builds\path\to\remote\file.zip"
c = Connection(uuid.uuid4(), server)
c.connect()
try:
print(f"Connection - Capabilities: {c.server_capabilities}")
s = Session(c, username, password)
s.connect()
t = TreeConnect(s, rf"\\{server}\{share}")
t.connect()
print(f"Tree - IsDfsShare {t.is_dfs_share}")
o = Open(t, path)
o.create(
impersonation_level=ImpersonationLevel.Impersonation,
desired_access=FilePipePrinterAccessMask.FILE_READ_DATA,
file_attributes=FileAttributes.FILE_ATTRIBUTE_NORMAL,
share_access=ShareAccess.FILE_SHARE_READ,
create_disposition=CreateDisposition.FILE_OPEN,
create_options=0,
create_contexts=None,
)
print("File opened")
finally:
c.disconnect()
Comparing it with one that works would also be great. If this does fail what happens if you do t.is_dfs_share = False
straight after t.connect()
?
@jborean93 Thanks for the help, here are the results using the known good and known bad share.
Known bad (same result with t.is_dfs_share = False
):
Connection - Capabilities: (23) SMB2_GLOBAL_CAP_DFS, SMB2_GLOBAL_CAP_LARGE_MTU, SMB2_GLOBAL_CAP_LEASING, SMB2_GLOBAL_CAP_PERSISTENT_HANDLES
Tree - IsDfsShare True # or False when t.is_dfs_share = False
Traceback (most recent call last):
File "/Users/andrewsiemer/Desktop/test.py", line 35, in <module>
o.create(
File "/Users/andrewsiemer/Library/Python/3.9/lib/python/site-packages/smbprotocol/open.py", line 1160, in create
return self._create_response(request)
File "/Users/andrewsiemer/Library/Python/3.9/lib/python/site-packages/smbprotocol/open.py", line 1168, in _create_response
response = self.connection.receive(request)
File "/Users/andrewsiemer/Library/Python/3.9/lib/python/site-packages/smbprotocol/connection.py", line 1095, in receive
raise SMBResponseException(response)
smbprotocol.exceptions.PathNotCovered: Received unexpected status from the server: The contacted server does not support the indicated part of the DFS namespace. (3221226071) STATUS_PATH_NOT_COVERED: 0xc0000257
Known good (same result with t.is_dfs_share = False
):
Connection - Capabilities: (23) SMB2_GLOBAL_CAP_DFS, SMB2_GLOBAL_CAP_LARGE_MTU, SMB2_GLOBAL_CAP_LEASING, SMB2_GLOBAL_CAP_PERSISTENT_HANDLES
Tree - IsDfsShare True # or False when t.is_dfs_share = False
File opened
The extra flag added on a request when the tree connect has marked it as a DFS share was introduced with https://github.com/jborean93/smbprotocol/pull/190 which was designed to replicate the Windows behaviour for the issue https://github.com/jborean93/smbprotocol/issues/170.
I'm not really sure what the best thing to do here, it seems like some server like this while others do not. You could use Wireshark to see how Windows opens the same file as well other macOS' SMB client does as well. Comparing them all to see what flags are set for the request and how they handle things would definitely help to see what the next steps should be.
Do you have any quick resources I can reference for checking the request in Wireshark? It has been a few years 😅
Essentially install Wireshark and capture port 445. You want to compare the headers for the TreeConnect and Open requests. The code I gave you lets you do it with smbprotocol, for macOS you can have a look at mounting the path and doing a cat
on the file, for Windows just open the file with Get-Item
in PowerShell.
I have captured the following packets for the good/bad case and they seem to have similar headers (up to the encrypted ones). I do not see any of the error codes that I see from the python script. Am I missing something?
I did realize that mounting manually and opening the file from the command line produces "Create File Request" packets instead of Encrypted SMB3.
Ah my apologies, I forgot smbprotocol defaults to enabling encryption. You can disable it to see the same Tree and Open/Create packets by doing Session(..., require_encryption=False)
when creating the session.
Thanks, that did it. So now the only difference is in the Create Response packet from the server. Although other than the error code, the flag fields match.
Create Response, Error: STATUS_PATH_NOT_COVERED
Another thing I noticed is it seems to only be an issue in this directory. Other directories on the same share don't have the error.
//server/share/a/b/c/d/e/file1.txt <--- anything beyond the //server/share/a/b/c directory throws the error
//server/share/a/b/f/g/h/file2.txt <--- no issues
Have you compared the flags on the header part of the message? It seems like having the SMB2_FLAGS_DFS_OPERATIONS
flag (that smbprotocol is adding) is causing the STATUS_FS_DRIVER_REQUIRED
. Does your macOS tool also add them, have you been able to test with Windows to see if it also sets that flag?
The STATUS_PATH_NOT_COVERED
is somewhat expected for a DFS path as the server is telling the client the path isn't covered by this server and to send a DFS request to figure out the correct server and it's path that can process that request. Knowing the error flow for this type of path and the subsequent messages that are sent after t3he error help to identify what needs to be done for DFS operations here.
All requests have the flag (when is_dfs_share == True) but same result when the flag is not set.
...1 .... .... .... .... .... .... .... = DFS operation: This is a DFS OPERATION
All responses always:
...0 .... .... .... .... .... .... .... = DFS operation: This is a normal operation
Mac + windows CLI utility do not use the SMB2_FLAGS_DFS_OPERATIONS
flag.
I think I'm going to have to play around with some DFS scenarios a bit more and get a better understanding of SMB2_FLAGS_DFS_OPERATIONS
. IIRC I saw WIndows set that when the tree reported itself as a DFS share but maybe that's not the case.
Anything I can do to help here?
After printing out some debug messages I have found that the following line causes the unknown exception.
Line 307 of smbclient/_io.py
:
for smb_open in _resolve_dfs(self.raw):
Are there any changes to dfs_request
that might help?
Sorry I haven't gotten back to this, I just haven't found the time needed to dig into the SMB protocol. Ultimately there's a problem in how I interpret the SMB2_FLAGS_DFS_OPERATIONS
in the SMB headers and what to do when the tree reports IsDfsShare
. At the time I thought my current behaviour was based on how Windows works but based on your responses I need to revisit it and figure out what exactly is happening.
No worries, in the meantime I am happy to help provide debug info if needed.
Could it be due to path normalization required by dfs?
If the request received has SMB2_FLAGS_DFS_OPERATIONS set in the Flags field of the SMB2 header, and TreeConnect.Share.IsDfs is TRUE, the server MUST verify the value of IsDfsCapable:
As specified in [MS-SMB2] section 3.3.5.9 and [MS-SMB] section 3.3.5.5, the SMB server invokes the DFS server to normalize the path name.
- If the DFS namespace initialization (as specified in section 3.2.3) corresponding to the share in the path is not yet complete, the DFS server MUST fail the path normalization request with STATUS_DFS_UNAVAILABLE.
- Otherwise, the DFS server matches the path name against DFS metadata. If the path matches or contains a DFS link, the DFS server MUST respond to the path normalization request with STATUS_PATH_NOT_COVERED, indicating to the client to resolve the path by using a DFS link referral request. Otherwise, the DFS server MUST change the path name to a path relative to the root of the namespace and return STATUS_SUCCESS. For example, if the path name is "\MyDomain\MyDfs\MyDir\file1", then the DFS server MUST change the path name to "MyDir\file1"
Yea the problem is that the request from smbprotocol is including SMB2_FLAGS_DFS_OPERATIONS
when talking to a host that doesn't know how to resolve the path. Currently smbprotocol sets this header flag when the TreeConnect IsDfs
flag is set, introduced with https://github.com/jborean93/smbprotocol/pull/190. Why I did that I unfortunately cannot remember and the PR/comments/issue don't seem to indicate why unfortunately. It could be I misread that particular entry you shared or it could be behaviour I saw with a real DFS share at the time but obviously based on this issue it's not correct. The trick is figuring out when it should be set and when it should not be.
Hi I'm trying to jump back into this issue and figure out the problem once and for all. If you still have the environment around and are willing to test things it would be great if you could share the following:
//server/share/a/b/c
a DFS targetSMB2_SHAREFLAG_DFS
in the tree flagsDFS
capabilityI'm trying to setup a similar environment to test out so I can play around with it all but Windows seems to handle whatever I throw at it just fine.
I don't know if it will solve the issue in your case but https://github.com/jborean93/smbprotocol/pull/253 makes setting this flag a bit more selective and maybe will help you. If you get time to test it out that would be great.
I have tried #253 with no luck (smbprotocol.exceptions.SMBOSError: [Error 0] [NtStatus 0xc000019c] Unknown NtStatus error returned 'STATUS_FS_DRIVER_REQUIRED':
). It still works if I use the DFS referral path just not with the actual DFS path.
That's a pity.
It still works if I use the DFS referral path just not with the actual DFS path.
So something is going on with the referral process that's causing it to send the the open to potentially the DFS server(?) causing it to fail. Unfortunately at this point in time I still cannot replicate it in my environment so short of seeing the network traces there's not much else I can do here. I essentially need to compare the network operations likes
Unless you can share more about the network environment such as the DFS servers used, the namespace and target setup, target paths, etc so I can potentially replicate it I'm nearing the end of my abilities here sorry.
No worries, thank you so much for all the help!
I am running into the following issue when trying to copy a file from a DFS share.
The code is:
I use this code to copy many files and it seems to only throw this error with one share. When I mount via the CLI, I can copy the file manually.
Thanks in advance!