jborean93 / smbprotocol

Python SMBv2 and v3 Client
MIT License
322 stars 73 forks source link

Cannot use `share_access="r"` with `smbclient.shutil.copytree()` #258

Closed andrewsiemer closed 9 months ago

andrewsiemer commented 10 months ago

Hi, it seems it is not possible to use kwarg share_access="r" when using smbclient.shutil.copytree() (but can when using smbclient.shutil.copyfile()). It seems this is because the arg is already defined in an internal function (with SMBDirectoryIO(path, share_access="rwd", **kwargs) as fd:). The complete traceback is below.

Usage

smbclient.shutil.copyfile(src, dst, username=_u, password=_p, share_access="r")   # No issues
smbclient.shutil.copytree(src, dst, username=_u, password=_p, dirs_exist_ok=True, copy_function=smbclient.shutil.copyfile, share_access="r")   # Issues

Error:

Traceback (most recent call last):
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/module/netutils.py", line 191, in copy_directory
    smbclient.shutil.copytree(
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/smbclient/shutil.py", line 296, in copytree
    dir_entries = list(scandir(src, **kwargs))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/smbclient/_os.py", line 543, in scandir
    with SMBDirectoryIO(path, share_access="rwd", **kwargs) as fd:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: smbclient._io.SMBDirectoryIO() got multiple values for keyword argument 'share_access'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/module/netutils.py", line 191, in copy_directory
    smbclient.shutil.copytree(
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/smbclient/shutil.py", line 296, in copytree
    dir_entries = list(scandir(src, **kwargs))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/smbclient/_os.py", line 543, in scandir
    with SMBDirectoryIO(path, share_access="rwd", **kwargs) as fd:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: smbclient._io.SMBDirectoryIO() got multiple values for keyword argument 'share_access'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/module/netutils.py", line 191, in copy_directory
    smbclient.shutil.copytree(
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/smbclient/shutil.py", line 296, in copytree
    dir_entries = list(scandir(src, **kwargs))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/smbclient/_os.py", line 543, in scandir
    with SMBDirectoryIO(path, share_access="rwd", **kwargs) as fd:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: smbclient._io.SMBDirectoryIO() got multiple values for keyword argument 'share_access'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/module/netutils.py", line 191, in copy_directory
    smbclient.shutil.copytree(
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/smbclient/shutil.py", line 296, in copytree
    dir_entries = list(scandir(src, **kwargs))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/myuser/git/web-app/.venv/lib/python3.12/site-packages/smbclient/_os.py", line 543, in scandir
    with SMBDirectoryIO(path, share_access="rwd", **kwargs) as fd:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: smbclient._io.SMBDirectoryIO() got multiple values for keyword argument 'share_access'

I need share access set to read-only or else I get [NtStatus 0xc0000043] The process cannot access the file because it is being used by another process: .... Why is a basic copy not read-only by default? Also, is there another way of achieving this with the current module version? Thanks!

jborean93 commented 10 months ago

I'm not even allowing such a thing will help you, the current code is setting the share_access="rwd" which is essentially trying to be as relaxed as possible when attempting to open the directory in case something else has or will open a new handle on that directory. Using rwd includes r inside the list of FILE_SHARE values used.

It sounds like the there is an existing handle on some directory in the tree, or the root dir itself. has been opened without the read share access itself so it cannot be opened again no matter what share_access value you specify.

Granted the file sharing stuff is complicated and I could have missed something but here is an example in PowerShell where you open two file handles, one with a more restricted file share of Read while the second handle is doing the rwd equivalent:

$path = 'C:\temp\test.txt'
New-Item $path -Value foo -ItemType File -Force

$fs1 = [System.IO.FileStream]::new($path, 'Open', 'Read', 'Read')
$fs2 = [System.IO.FileStream]::new($path, 'Open', 'Read', 'ReadWrite, Delete')

$fs1.Dispose()
$fs2.Dispose()

Compare that to this example where the first file was opened without any share access rendering the next two attempts unable to open the handle regardless of the sharing option specified

$path = 'C:\temp\test.txt'
New-Item $path -Value foo -ItemType File -Force

# Opens handle with read access
$fs1 = [System.IO.FileStream]::new($path, 'Open', 'Read', 'None')

# Both fail with
# Exception calling ".ctor" with "4" argument(s): "The process cannot access the file 'C:\temp\test.txt' because it is
being used by another process."
[System.IO.FileStream]::new($path, 'Open', 'Read', 'Read')
[System.IO.FileStream]::new($path, 'Open', 'Read', 'ReadWrite, Delete')
andrewsiemer commented 10 months ago

Are you suggesting this is the intended response when including share_access as kwarg with smbclient.shutil.copytree()?

jborean93 commented 10 months ago

I would argue that the kwargs on all these methods are designed more for connection kwargs like username/password/etc. Trying to override internal logic this way is a very complex thing to do especially as you need to content whether the share_access is only for SMB backed files, whether it applies to the src or dst, only for files, dirs, or both, etc.

But even if it was a supported scenario I don't think it will fix your problem, the problem is that something has already opening the directory in question without the read file share access blocking anything else from opening it with read themselves.

andrewsiemer commented 10 months ago

It seems that even the smbclient.shutil.copyfile will raise [NtStatus 0xc0000043] when two processes execute it at the same time (but not if I include share_access="r"). Is that still an outside issue?

jborean93 commented 10 months ago

Not sure what you mean sorry, can you share an actual reproducer there.

andrewsiemer commented 10 months ago

Sure,

example.py

import smbclient.shutil

smbclient.shutil.copyfile( r"\\path\to\file", "file", username="username", password="password")

Running example.py concurrently:

(venv) % python3 example.py &
[1] 1487
(venv) % python3 example.py &
[2] 1494
(venv) % Traceback (most recent call last):
  File "example.py", line 3, in <module>
    smbclient.shutil.copyfile(
  File "/venv/lib/python3.12/site-packages/smbclient/shutil.py", line 181, in copyfile
    with src_open(src, mode="rb", **src_kwargs) as src_fd, dst_open(dst, mode="wb", **dst_kwargs) as dst_fd:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/smbclient/_os.py", line 389, in open_file
    raw_fd.open()
  File "/venv/lib/python3.12/site-packages/smbclient/_io.py", line 463, in open
    transaction.commit()
  File "/venv/lib/python3.12/site-packages/smbclient/_io.py", line 349, in commit
    raise failures[0]
smbprotocol.exceptions.SMBOSError: [Error 1] [NtStatus 0xc0000043] The process cannot access the file because it is being used by another process: ‘\\path\to\file’
[2]  + exit 1     python3 

However, if you add share_access="r" the problem goes away.

jborean93 commented 9 months ago

Thanks for the example, in this case it means that shutil.copyfile should be adding the share_access="r" argument itself so you don't have to. Essentially the functions should be opening the files as best as they can and it should not be something you as the caller specify.

jborean93 commented 9 months ago

I've opened the PR https://github.com/jborean93/smbprotocol/pull/265 to fix the copyfile issue here.

jborean93 commented 9 months ago

https://github.com/jborean93/smbprotocol/pull/265 fixes that particular scenario, if you have any more where you've needed to explicitly specify share_access in the shutil methods then please let me know so we can fix them.

andrewsiemer commented 9 months ago

Thanks @jborean93!