os.makedirs fails on long-path UNC-paths if it is the first sub-folder

[fdc08b81-f10f-4d56-a4c0-de5650414d13]

BPO	41705
Nosy	@pfmoore, @tjguk, @zware, @eryksun, @zooba, @Safihre

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', '3.8', 'OS-windows'] title = 'os.makedirs fails on long-path UNC-paths if it is the first sub-folder' updated_at = user = 'https://github.com/Safihre' ``` bugs.python.org fields: ```python activity = actor = 'eryksun' assignee = 'none' closed = False closed_date = None closer = None components = ['Windows'] creation = creator = 'Safihre' dependencies = [] files = [] hgrepos = [] issue_num = 41705 keywords = [] message_count = 2.0 messages = ['376312', '376326'] nosy_count = 6.0 nosy_names = ['paul.moore', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'Safihre'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue41705' versions = ['Python 3.8'] ```

[fdc08b81-f10f-4d56-a4c0-de5650414d13]

It consistently fails on the first directory in a long-path UNC notation server-folder.

>>> os.makedirs(r"\\?\UNC\DiskStation\already_exists", exist_ok=True)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 223, in makedirs
    mkdir(name, mode)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '\\\\?\\UNC\\'

>>> os.makedirs(r"\\?\UNC\DiskStation\already_exists")
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 223, in makedirs
    mkdir(name, mode)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '\\\\?\\UNC\\'

The second level directory is working correctly as expected:

>> os.makedirs(r"\\?\UNC\DiskStation\already_exists\new")

>>> os.makedirs(r"\\?\UNC\DiskStation\already_exists\new")
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 223, in makedirs
    mkdir(name, mode)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: '\\\\?\\UNC\\DiskStation\\test_get2\\test2'

Inspecting the code, I think the problem is in the os.path.exists function that is called from within os.makedirs, the line in os.makedirs says: if head and tail and not path.exists(head):

But:
>>> head, tail = path.split(r"\\?\UNC\DiskStation\already_exists")
('\\\\?\\UNC\\DiskStation', 'already_exists')

>>> os.path.exists(r'\\?\UNC\DiskStation')
False
>>> os.path.exists(r'\\DiskStation')
False

So it wrongly goes ahead and tries to create \\?\UNC\DiskStation

[eryksun]

This behavior is due to bpo-37609, i.e. ntpath.splitdrive fails for "UNC" device paths.

    >>> ntpath.splitdrive('//?/UNC/server/share')
    ('//?/UNC', '/server/share')

The result should be "//?/UNC/server/share". A path on the "UNC" device requires a share component, on which is mounted a local or remote filesystem directory. It's functionally part of the 'drive' path. Using just the root path or a server path on the "UNC" device is malformed in the context of a normal file API open. The former fails as ERROR_INVALID_NAME (123), and the latter fails as ERROR_BAD_PATHNAME (161).

The incorrect splitdrive result in turn makes ntpath.split misbehave:

    >>> ntpath.split('//?/UNC/server/share')
    ('//?/UNC/server', 'share')
    >>> ntpath.split('//?/UNC/server')
    ('//?/UNC/', 'server')

The correct result should be ('//?/UNC/server/share', '').

[Safihre]

@barneygale can you confirm if this is also resolved with your patch? The source of the problem is indicated by @eryksun to be in splitdrive. https://github.com/python/cpython/issues/85871#issuecomment-1093882544

[picnixz]

@barneygale friendly ping for confirmation (otherwise, we can close this issue)