g-fusion commented 4 years ago

Environment

Windows version: 10.0.14393
Processor architecture: x64
Dokany version: 1.1.0
Library type (Dokany/FUSE): n/a

Check List

[x] I checked my issue doesn't exist yet
[x] My issue is valid with mirror default sample and not specific to my user-mode driver implementation
[x] I can always reproduce the issue with the provided description below.
[x] I have updated Dokany to the latest version and have reboot my computer after.
[ ] I tested one of the last snapshot from appveyor CI

Description

Running mirror with options Mount Manager & removable drive fails to mount.

Steps to reproduce: Run mirror.exe with following parameters (including Mount Manager & removable drive) mirror.exe /o /m /d /s /r C:\tmp /l h

CMD logs (tested with Dokany/mirror 1.1.0 & 1.3.0):

With option "use removable drive"

mirror.exe /o /m /d /s /r C:\Users /l h RootDirectory: C:\Users Attempting to add SE_SECURITY_NAME privilege to process token ## failed: Unable to adjust token privileges: 1300 Failed to add security privilege to process => GetFileSecurity/SetFileSecurity may not work properly => Please restart mirror sample with administrator rights to fix it Dokan: debug mode on Dokan: use stderr AllocationUnitSize: 512 SectorSize: 512 device opened Dokan Error: CreateFile failed \.\Volume{d6cc17c5-1724-4085-bce7-964f1e9f5de9}: 5 Dokan Error: CreateFile failed \.\Volume{d6cc17c5-1724-4085-bce7-964f1e9f5de9}: 5 Dokan Error: DokanKeepAlive CreateFile failed \.\Volume{d6cc17c5-1724-4085-bce7-964f1e9f5de9}: 5 Dokan Error: CreateFile failed \.\Volume{d6cc17c5-1724-4085-bce7-964f1e9f5de9}: 5 Dokan Error: CreateFile failed \.\Volume{d6cc17c5-1724-4085-bce7-964f1e9f5de9}: 5 Dokan Error: CreateFile failed \.\Volume{d6cc17c5-1724-4085-bce7-964f1e9f5de9}: 5 mounted: h -> \Volume{d6cc17c5-1724-4085-bce7-964f1e9f5de9} Mounted Unmounted unload Success

================= Without option "use removable drive"

mirror.exe /o /d /s /r C:\Users /l h RootDirectory: C:\Users Attempting to add SE_SECURITY_NAME privilege to process token ## failed: Unable to adjust token privileges: 1300 Failed to add security privilege to process => GetFileSecurity/SetFileSecurity may not work properly => Please restart mirror sample with administrator rights to fix it Dokan: debug mode on Dokan: use stderr AllocationUnitSize: 512 SectorSize: 512 device opened

QueryVolumeInfo -001

GetVolumeInfmounted: h -> \Volume{d6cc17c5-173f-4085-bce7-964f1e9f5de9} ormation: max component length 255 Mounted GetVolumeInformation: file system name NTFS GetVolumeInformation: got file system flags 0x03e700ff, returning 0x0004000f

Create 0000

.....

Strange is that despite the message "Failed to add security privilege to process ...", visible on both options for mount, the one without "use removable drive" manages to mount, whereas the other is failing

Logs

logs_mountmanager&removabledrive.txt

Liryna commented 4 years ago

Hi @g-fusion ,

The error message is GetFileSecurity/SetFileSecurity. Using a shell with admin rights will make it disappear.

Otherwise about mount manager option and removable device, I do not think it was made to work together at all. I have to remember what was the reason of this (possible?) incompatibility.

g-fusion commented 4 years ago

Hi @Liryna,

Running as "admin" is not an option for end users
Regarding Mount Manager & Removable drive - we have been using this option since 1.0.0-rc1 and had no issues. The reason to use MountManager is due to bug opened (by me again) - https://github.com/dokan-dev/dokany/issues/340
What also strange is: a. using mirror.exe and "LocalDrive" and message about "GetFileSecurity/SetFileSecurity" does not prevent the mount itself (not sure about functionality later on though) b. using mirror.exe and "RemovalbleDrive" only (no Mount manager this time) and message about "GetFileSecurity/SetFileSecurity" DOES affect mount itself, it just do not work
"LocalDriver" vs "RemovableDrive" - Is there a difference how windows behave comparing both driver options and in conjunction with https://github.com/dokan-dev/dokan-dotnet/blob/master/DokanNet/IDokanOperations.cs interface and point 3. from above

Liryna commented 4 years ago

Sorry, I wanted to say the error is only from mirror that need to have admin right for full file security rights. https://github.com/dokan-dev/dokany/blob/master/samples/dokan_mirror/mirror.c#L1684-L1688 It does not affect at all the mirror mount.

So for you the issue happened between 1.0.0-rc1and 1.1.0 ?

g-fusion commented 4 years ago

Hi @Liryna,

Issue is noticed now on 1.1.0, have not tested on rc1 now, but back then it was working. Back to mount issue:

mirror mount is affected when"RemovalbleDrive" only is selected (see 3b from my previous replay, no Mount manager this time) whereas "LocalDrive" mount is not affected! What is the difference "LocalDriver" vs "RemovableDrive" if security rights are not the reason?

Liryna commented 4 years ago

I just tried the 1.3.0 without admin rights with mount manager and removable device. The mount and browse work.

C:\Program Files\Dokan\Dokan Library-1.3.0\sample\mirror>mirror.exe /r C:\Users /l x /o /m /d /s
## Attempting to add SE_SECURITY_NAME privilege to process token ##
  failed: Unable to adjust token privileges: 1300
Failed to add security privilege to process
        => GetFileSecurity/SetFileSecurity may not work properly
        => Please restart mirror sample with administrator rights to fix it
Dokan: debug mode on
Dokan: use stderr
AllocationUnitSize: 512 SectorSize: 512
device opened
###QueryVolumeInfo -001
GetVolumeInformation: max component length 255
mounted: x -> \Volume{d6cc17c5-1737-4085-bce7-964f1e9f5de9}
Mounted
GetVolumeInformation: file system name NTFS
GetVolumeInformation: got file system flags 0x03e706ff, returning 0x0004000f
###Create 0000
CreateFile : C:\Users\
  AccountName: liryna, DomainName: DESKTOP-LLP73O9
        ShareMode = 0x3
        FILE_SHARE_READ
        FILE_SHARE_WRITE
        DesiredAccess = 0x100000
        SYNCHRONIZE
        FlagsAndAttributes = 0x0
        OPEN_EXISTING

CreateFile status = 0

I am also adding this mount test to the CI https://github.com/dokan-dev/dokany/commit/b12ee1a67329e8a537eb5a6e0bfd86f7b7b07781 CI build: https://ci.appveyor.com/project/Maxhy/dokany/builds/28544198

Could it not be something in you env that could after this behavior (antivirus, ...) ? Maybe one of these software is acting differently on dokan between both devices type?

g-fusion commented 4 years ago

Hi @Liryna,

Kernel logs attached and I can see message like " Here we only go in if some antivirus software tries to create files before startup is finished.", but could not see anything from proc mon. Strange things is that with "Run as Administrator" - the mount works! Any advice to trouble shoot it further?

Liryna commented 4 years ago

@g-fusion The anti-virus is pretty generic message. It is printed for every software who would like to acces files system before dokan library himself.

Would it be possible to have two clear separate logs for mount with and without admin rights?

g-fusion commented 4 years ago

Hello @Liryna ,

Attached both logs for admin (successful mount) and without admin rights (failed mount). Admin_mount.txt EndUser_Mount(failed).txt

Liryna commented 4 years ago

Hi @g-fusion

Looking at the logs, seems like there is something blocking the communication between the kernel and library. You seem to have software in your env that tries to access the device before the library can responses but since there is no communication... dokan will unmount some second after. You even see earlier that library cannot creatfile on the kernel probably blocked by the software.

Do you have any idea which one could produce this behavior ?

g-fusion commented 4 years ago

Hi @Liryna ,

Thank your for the input, yes, I saw "createfile" failing however, I was wondering why some "software", would not have effect when "run as admin" is used. Is there some way (a tool maybe) which can give me more info, what could cause this issue, because I do not have any idea which "software" may cause such behaviour?

g-fusion commented 4 years ago

Hi @Liryna, I was wondering will it be helpful to understand this behaviour, if I collect some Procmon traces?

Liryna commented 4 years ago

Procmoc would probably not show more than an error for createfile. However, you can use fltmc to find the filter drivers that are on top of dokan and can block the createfile. https://blogs.msdn.microsoft.com/ntdebugging/2013/03/25/understanding-file-system-minifilter-and-legacy-filter-load-order/ It is a little of reading ;'( you might see what software are running and removes one by one to find out who is the faulty driver

DimitarKapashikov commented 4 years ago

Hi @Liryna ,

I have run the command: C:\Windows\system32>fltmc

Filter Name Num Instances Altitude Frame

FileCrypt 0 141100 0 npsvctrig 1 46000 0 FileInfo 2 45000 0 Wof 1 40700 0

In the list I cannot see dokan, am I executing the right command?

When I try to mount I see only a new instance of FileInfo created for the device:

C:\Windows\system32>fltmc instances -f FileInfo

Instances for FileInfo filter:

Volume Name Altitude Instance Name Frame VlStatus

\Device\Mup 45000 FileInfo 0 C: 45000 FileInfo 0 \Device\Volume{d6cc17c5-1736-4085-bce7-964f1e9f5de9} 45000 FileInfo 0

Thanks and Regards

DimitarKapashikov commented 4 years ago

Hi @Liryna ,

I have tried to bypass UAC , as in https://blog.netspi.com/tokenvator-a-tool-to-elevate-privilege-using-windows-tokens/ and was able mount. I was wondering why using an admin token with limited privileges didn't prevent the mount . Did you experience something similar related to UAC and mount issue?

Thanks and Regards

Liryna commented 4 years ago

Hi @DimitarKapashikov ,

Sorry for the delay. To know exactly wish filter driver are on top of dokan during runtime, it requires to attach a windbg to the VM. Here are the commands to run when dokan is mount (might be needed to break the machine right between the mount and unmount procude by keepalive failure):

#load filter driver commands
>.load fltkd
#list volumes:
>!volumes
....will find dokan mount id (fff8b....) here with filter instance.....
#use the id the get exact filter driver list in InstanceList field
>!volume fff8b....

About UAC, it might be some software blocking this operations and only letting the CreateFile pass when it is admin...

DimitarKapashikov commented 4 years ago

Hi @Liryna ,

Thanks. I have tried to increase the timeout of the mirror (/i) and tried to see what it is the output in windbg. Here it is:

lkd> !volumes

Volume List: ffffb8098bde1140 "Frame 0" FLT_VOLUME: ffffb8098bef37f0 "\Device\Mup" FLT_INSTANCE: ffffb8098bef2b40 "FileInfo" "45000" FLT_VOLUME: ffffb8098bf8c010 "\Device\HarddiskVolume1" FLT_INSTANCE: ffffb8098ececb60 "wcnfs Instance" "409900" FLT_INSTANCE: ffffb8098d792010 "wcifs Instance" "189900" FLT_INSTANCE: ffffb8098d79c010 "luafv" "135000" FLT_INSTANCE: ffffb8098c1e4560 "FileInfo" "45000" FLT_INSTANCE: ffffb8098c1ac010 "Wof Instance" "40700" FLT_VOLUME: ffffb8098c4ab010 "\Device\NamedPipe" FLT_INSTANCE: ffffb8098c4c33b0 "npsvctrig" "46000" FLT_VOLUME: ffffb8098c4aa010 "\Device\Mailslot" FLT_VOLUME: ffffb809932b7010 "\Device\Volume{d6cc17c5-1738-4085-bce7-964f1e9f5de9}" lkd> !volume ffffb809932b7010

FLT_VOLUME: ffffb809932b7010 "\Device\Volume{d6cc17c5-1738-4085-bce7-964f1e9f5de9}" FLT_OBJECT: ffffb809932b7010 [04000000] Volume RundownRef : 0x0000000000000002 (1) PointerCount : 0x00000001 PrimaryLink : [ffffb8098bde1140-ffffb8098c4aa020] Frame : ffffb8098bde1010 "Frame 0" Flags : [00000022] PendingSetupNotify EnableNameCaching FileSystemType : [00000000] FLT_FSTYPE_UNKNOWN VolumeLink : [ffffb8098bde1140-ffffb8098c4aa020] DeviceObject : ffffb8099349c060 DiskDeviceObject : ffffb80992bae080 FrameZeroVolume : ffffb809932b7010 VolumeInNextFrame : 0000000000000000 Guid : "" CDODeviceName : "\Device\DokanFs1" CDODriverName : "\FileSystem\dokan1" TargetedOpenCount : 0 Callbacks : (ffffb809932b7130) ContextLock : (ffffb809932b7518) VolumeContexts : (ffffb809932b7520) Count=0 StreamListCtrls : (ffffb809932b7528) rCount=0 FileListCtrls : (ffffb809932b75a8) rCount=0 NameCacheCtrl : (ffffb809932b7628) InstanceList : (ffffb809932b70b0)

Under FLT_VOLUME: ffffb809932b7010 "\Device\Volume{d6cc17c5-1738-4085-bce7-964f1e9f5de9}" I cannot see a an instance attached , maybe because it exited too quickly.

Whan I run it as administrator I can see an instanche of FileInfo attached :

FLT_VOLUME: ffffb80990c5e010 "\Device\Volume{d6cc17c5-1739-4085-bce7-964f1e9f5de9}" FLT_OBJECT: ffffb80990c5e010 [04000000] Volume RundownRef : 0x0000000000000004 (2) PointerCount : 0x00000001 PrimaryLink : [ffffb8098bde1140-ffffb8098c4aa020] Frame : ffffb8098bde1010 "Frame 0" Flags : [00000064] SetupNotifyCalled EnableNameCaching FilterAttached FileSystemType : [00000000] FLT_FSTYPE_UNKNOWN VolumeLink : [ffffb8098bde1140-ffffb8098c4aa020] DeviceObject : ffffb80993386960 DiskDeviceObject : ffffb809936402b0 FrameZeroVolume : ffffb80990c5e010 VolumeInNextFrame : 0000000000000000 Guid : "" CDODeviceName : "\Device\DokanFs1" CDODriverName : "\FileSystem\dokan1" TargetedOpenCount : 0 Callbacks : (ffffb80990c5e130) ContextLock : (ffffb80990c5e518) VolumeContexts : (ffffb80990c5e520) Count=0 StreamListCtrls : (ffffb80990c5e528) rCount=1 FileListCtrls : (ffffb80990c5e5a8) rCount=0 NameCacheCtrl : (ffffb80990c5e628) InstanceList : (ffffb80990c5e0b0) FLT_INSTANCE: ffffb8098eb942e0 "FileInfo" "45000"

As far I can see this a standard windows FileInfo Filter. Any idea if it can cause the issue ?

Thanks and Regards, Dimitar

Liryna commented 4 years ago

@DimitarKapashikov have you been able to find a workaround or more information ?

DimitarKapashikov commented 4 years ago

Hi , @Liryna unfortunately we are still facing the issue. Do you have any idea , how we can further investigate it? Thanks

Liryna commented 4 years ago

No idea unfortunately, I tried on my side and can confirm mount manager + removable drive does work together. It don't believe there is anything special but, could you take a look at the event viewer in system for Dokan1 logs if any of them has an error level or a message of failure ? Have you also looked with procmon the createfile call stack ? it might display who answer with an access denied.

DimitarKapashikov commented 4 years ago

Hi @Liryna , Thanks for the suggestions. I have checked the call stack for createFile operation on Path \Device\Volume{d6cc17c5-1730-4085-bce7-964f1e9f5de9}\ but the operation Result is Success with the following call stack. I can only see the Filter Manager.

Frame	Module	Location	Address	Path
0	fltmgr.sys	fltmgr.sys + 0x20ba	0xfffff80016eb60ba	C:\Windows\system32\drivers\fltmgr.sys
1	fltmgr.sys	fltmgr.sys + 0x2d0c	0xfffff80016eb6d0c	C:\Windows\system32\drivers\fltmgr.sys
2	fltmgr.sys	fltmgr.sys + 0x2b335	0xfffff80016edf335	C:\Windows\system32\drivers\fltmgr.sys
3	ntoskrnl.exe	ntoskrnl.exe + 0x3c5529	0xfffff802531e1529	C:\Windows\system32\ntoskrnl.exe
4	ntoskrnl.exe	ntoskrnl.exe + 0x48ca2e	0xfffff802532a8a2e	C:\Windows\system32\ntoskrnl.exe
5	ntoskrnl.exe	ntoskrnl.exe + 0x3c1f33	0xfffff802531ddf33	C:\Windows\system32\ntoskrnl.exe
6	ntoskrnl.exe	ntoskrnl.exe + 0x4863f5	0xfffff802532a23f5	C:\Windows\system32\ntoskrnl.exe
7	ntoskrnl.exe	ntoskrnl.exe + 0x43a72c	0xfffff8025325672c	C:\Windows\system32\ntoskrnl.exe
8	ntoskrnl.exe	ntoskrnl.exe + 0x1502e3	0xfffff80252f6c2e3	C:\Windows\system32\ntoskrnl.exe
9	ntdll.dll	ntdll.dll + 0x90a8a	0x7ff906840a8a	C:\Windows\SYSTEM32\ntdll.dll
10	KERNELBASE.dll	KERNELBASE.dll + 0x94b44	0x7ff903a84b44	C:\Windows\system32\KERNELBASE.dll
11	KERNELBASE.dll	KERNELBASE.dll + 0x646c	0x7ff9039f646c	C:\Windows\system32\KERNELBASE.dll
12	mscorlib.ni.dll	mscorlib.ni.dll + 0x58d3b0	0x7ff8f54ad3b0	C:\Windows\assembly\NativeImages_v4.0.30319_64\mscorlib\34d3daa41387618390516025073e6ef2\mscorlib.ni.dll
13	mscorlib.ni.dll	mscorlib.ni.dll + 0x4f974e	0x7ff8f541974e	C:\Windows\assembly\NativeImages_v4.0.30319_64\mscorlib\34d3daa41387618390516025073e6ef2\mscorlib.ni.dll
14	mscorlib.ni.dll	mscorlib.ni.dll + 0x4f94a8	0x7ff8f54194a8	C:\Windows\assembly\NativeImages_v4.0.30319_64\mscorlib\34d3daa41387618390516025073e6ef2\mscorlib.ni.dll
15	mscorlib.ni.dll	mscorlib.ni.dll + 0xdc1c06	0x7ff8f5ce1c06	C:\Windows\assembly\NativeImages_v4.0.30319_64\mscorlib\34d3daa41387618390516025073e6ef2\mscorlib.ni.dll
16		0x7ff8972029d9	0x7ff8972029d9

DimitarKapashikov commented 4 years ago

Hi @Liryna , a quick update on the issue. We've found recently, that mounting a drive in a windows service solves the issue. We are investigating what is the difference in the security context by running the command prompt with a local user and using one and the same user to run the windows service. If you have any suggestion, it will be highly appreciated. Regards, Dimitar

Liryna commented 4 years ago

Hi @DimitarKapashikov , Thank you for the feedback! I also got another feedback https://github.com/dokan-dev/dokany/issues/920 where they had issue with the option. I will add some doc around the option to inform future users. If you can find further details about the reason, it would be highly appreciated !

Liryna commented 3 years ago

Closing this as the issue is now documented. We can reopen if something point that Dokan is doing something wrong.

Also the new version is using new types of ioctl that have some chance to be a workaround here.

dokan-dev / dokany

Cannot Mount Removable Drive with mirror.exe #843

Environment

Check List

Description

CMD logs (tested with Dokany/mirror 1.1.0 & 1.3.0):

With option "use removable drive"

================= Without option "use removable drive"

QueryVolumeInfo -001

Create 0000

Logs