search

PyFilesystem / pyfilesystem

Python filesystem abstraction layer
http://pyfilesystem.org/
BSD 3-Clause "New" or "Revised" License
285 stars 63 forks source link

Fixing dokan support discussion... #256

Open apocalypse2012 opened 8 years ago

apocalypse2012 commented 8 years ago

Python 2.7.9 64 bit and 32 bit Dokan 0.7.4 FS 0.5.4 Windows 8.1

from fs.memoryfs import MemoryFS from fs.expose import dokan fs = MemoryFS() mp = dokan.mount(fs,"Q") Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\site-packages\fs\expose\dokaninit.py", line 936, in mount check_ready(mp) File "C:\Python27\lib\site-packages\fs\expose\dokaninit.py", line 913, in check_ready raise OSError("dokan mount process seems to be hung") OSError: dokan mount process seems to be hung

dokan.mount(fs,"Q",foreground=True) Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\site-packages\fs\expose\dokaninit.py", line 929, in mount raise OSError("Dokan failed with error: %d" % (res,)) OSError: Dokan failed with error: -5

I tried the newer versions before I realized that 8.0 was API breaking. I've read through all the message boards I can find with no apparent 'fix'. Dokan 0.6.0 no longer appears to be available on the Dokan project page at Github. I

Liryna commented 8 years ago

I only tested with python 2.7 for now and Windows 10. Otherwise than python limitations that I would not be aware, it should work on every systems where dokan is compatible.

apocalypse2012 commented 8 years ago

What is my next step on this? Do I need to pull this from Github or is there a new package going up on PyPi? Raymond

arekbulski commented 8 years ago

Please edit your post. Those auto attached quotes are spamming my little, 5.5 inch screen. :)

Liryna commented 8 years ago

You can clone my pull request if you wanna test it before it is accepted.

Liryna commented 8 years ago

@apocalypse2012 have you been to give a try ?

apocalypse2012 commented 8 years ago

Sadly Liryna, no. I have to admit, I have limited github experience and don't know how to do a pull from pyfilesystem or dokany to get at the version I need. Unfortunately, my plate has been too full to get it sorted. If you have any quick pointers to shortcut me through the process, they would be much appreciated... Raymond 

On Thursday, August 11, 2016 12:40 PM, Liryna <notifications@github.com> wrote:

@apocalypse2012 have you been to give a try ?— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

Liryna commented 8 years ago

git clone https://github.com/Liryna/pyfilesystem.git cd pyfilesystem python setup.py install

Pyfs doc http://docs.pyfilesystem.org/en/latest/getting_started.html#installing

You will probably need to remove package previously installed with pip.

apocalypse2012 commented 8 years ago

Should the same work for Dokany to get the 1.0RC?

On Thursday, August 11, 2016 2:00 PM, Liryna <notifications@github.com> wrote:

git clone https://github.com/Liryna/pyfilesystem.git cd pyfilesystem python setup.py installPyfs doc http://docs.pyfilesystem.org/en/latest/getting_started.html#installingYou will probably need to remove package previously installed with pip.— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

Liryna commented 8 years ago

direct link to download it: https://github.com/dokan-dev/dokany/releases/download/v1.0.0-RC4/DokanSetup_redist.exe

Liryna commented 8 years ago

@lurch have you been able to test the pull request ?

lurch commented 8 years ago

Not yet... as I mentioned in https://groups.google.com/forum/#!topic/pyfilesystem-discussion/0n7EGGIT0fo I'm actually away from home all this week.

JokerQyou commented 8 years ago

I gave it a try just now, with a Python 2.7.11 (32 bit) on Windows 10 64 bit, and the mounting subprocess went wrong, Windows shows a dialog saying python.exe has stopped working. And the traceback is as follows:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "fs\expose\dokan\__init__.py", line 973, in mount
    check_ready(mp)
  File "fs\expose\dokan\__init__.py", line 943, in check_ready
    check_alive(mp)
  File "fs\expose\dokan\__init__.py", line 931, in check_alive
    raise OSError("dokan mount process exited prematurely")
OSError: dokan mount process exited prematurely

The code I tried is:

import fs.memoryfs
import fs.expose.dokan
fs.memoryfs.__file__
s = fs.memoryfs.MemoryFS()
mp = fs.expose.dokan.mount(s, 'Q:\\')

My dokan installation is Dokoan Library 1.0.0.4000 Bundle as shown in Windows control panel.

I'm not sure whether it's a problem with my dokan installation or with your code. If I get a chance to reboot my computer I'll test again and let you know the result.

Liryna commented 8 years ago

Hi @JokerQyou ,

Did you copy the dokan1.dll from x86 folder (C:\Program Files\Dokan\x86) ? It has to be in the same folder as your code.

JokerQyou commented 8 years ago

I tested again with dokan1.dll copied to the pyfilesystem project root folder, and it did not work as well.

>>> import fs.memoryfs
>>> import fs.expose
>>> s = fs.memoryfs.MemoryFS()
>>> mp = fs.expose.
>>> from fs.expose import dokan
>>> mp = dokan.mount(s, 'Q:')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "fs\expose\dokan\__init__.py", line 923, in mount
    _check_path_string(path)
  File "fs\expose\dokan\__init__.py", line 896, in _check_path_string
    raise ValueError("invalid path: %r" % (path,))
ValueError: invalid path: 'Q:'
invalid path: 'Q:'
>>> mp = dokan.mount(s, 'Q:\\')
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "fs\expose\dokan\__init__.py", line 1061, in _do_mount
    mount(fs, path, **opts)
  File "fs\expose\dokan\__init__.py", line 964, in mount
    res = libdokan.DokanMain(ctypes.byref(opts), ctypes.byref(opstruct))
WindowsError: exception: access violation reading 0x00DF0000
KeyboardInterrupt

And I tested again with no mistake typing letters.

>>> import fs.memoryfs
>>> from fs.expose import dokan
>>> s = fs.memoryfs.MemoryFS()
>>> mp = dokan.mount(s, 'Z:\\')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "fs\expose\dokan\__init__.py", line 973, in mount
    check_ready(mp)
  File "fs\expose\dokan\__init__.py", line 940, in check_ready
    check_alive(mp)
  File "fs\expose\dokan\__init__.py", line 931, in check_alive
    raise OSError("dokan mount process exited prematurely")
OSError: dokan mount process exited prematurely
dokan mount process exited prematurely

And this time the dokan.mount() takes very long to finish (with an error though), and the drive Z: did appear in explorer, but it says drive disconnected when I try to open it.

I was testing within an interactive shell, does that affect anything?

Liryna commented 8 years ago 
>>> mp = dokan.mount(s, 'Q:\\')
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "fs\expose\dokan\__init__.py", line 1061, in _do_mount
    mount(fs, path, **opts)
  File "fs\expose\dokan\__init__.py", line 964, in mount
    res = libdokan.DokanMain(ctypes.byref(opts), ctypes.byref(opstruct))
WindowsError: exception: access violation reading 0x00DF0000

That's very strange

Can you try to run the mirror.exe in C:\Program Files\Dokan\sample ? (you need to enable developement files option during dokan installation) mirror.exe /r C:\Users /l m

I was testing within an interactive shell, does that affect anything?

This should not affect dokan.

If I understood correctly it mount the device but something is making it umount. I have not been able to reproduce this so we should first look on your side if it come from the python or the library (by runing the mirror)

Liryna commented 8 years ago

@lurch have you been able to test it ?

lurch commented 8 years ago

@lurch have you been able to test it ?

Unfortunately not yet, been far too busy with other stuff :-( It's still on my TODO list though...

Liryna commented 7 years ago

Dokan 1.0.0 has been released https://github.com/dokan-dev/dokany/releases/tag/v1.0.0 and can be used with my PR :wink:

Liryna commented 7 years ago

@JokerQyou I made a change on the pull request, do you think can you clone again and try to test it ?

apocalypse2012 commented 7 years ago

Sorry for not having followed up on this. I've been impacted by a layoff that has taken me offline from my original motivation. Still interested in this library and will Follow up as soon as I can...

Raymond

JokerQyou commented 7 years ago

I'd like to give it a try but I've been too busy... And I cannot afford a system restart (I reinstalled dokan with development files but have not restarted my computer) now. I'll test it if I got time tonight (GMT+8).

arekbulski commented 7 years ago

I moved on to Construct. Unsubscribing from this thread. Ping me if needed. :wave:

lurch commented 7 years ago

Moving this comment to here to try to separate 'general discussion' from 'code comments'...

@lurch After 2 month, It would be nice if you could clone the PR and make your own opinion.

Sorry, I'm not sure what you're asking me for?

I do appreciate all the hard work you and others have been putting into getting this working :+1: I've never written Win32 API code myself, but I understand how 'murky' it can be. Unfortunately, if even Windows Explorer or Notepad are currently unable to save files into a dokan-exposed filesystem (which AFAICT is the whole reason for the 'expose' interface in the first place - right @willmcgugan ?) then IMHO #258 has limited usefulness for typical users. People would only complain when it doesn't work the way they expect it to.

OTOH if you wanted to modify the PR so that it can only be used to expose ReadOnly filesystems (see e.g. http://docs.pyfilesystem.org/en/latest/base.html#fs.base.FS.getmeta and http://docs.pyfilesystem.org/en/latest/wrapfs/readonlyfs.html ), and documented it as such, then that could probably be merged. (Which will then hopefully be 'upgraded' later on to allow full dokan read/write access when this security issue gets fixed) It would be limited functionality (no modification to the exposed FS), but at least it would be consistent behaviour for the end-user.

I hope that doesn't sound too harsh, I hope you can see where I'm coming from. I'm afraid I don't get as much time to look at pyfilesystem as I'd like, and from their lack of input I assume @willmcgugan and @rfk are even busier than I am :-/

willmcgugan commented 7 years ago

Thanks @lurch. Sorry I haven't been keeping up with the work here.

Not being a massive Windows user I'll need a little guidance here. If we're talking about fixes then I'm all for it. But I'm guessing its not as clear cut?

JokerQyou commented 7 years ago

I've managed to test directly with the whole repo cloned from @Liryna 's remote, and here's the result with memory fs.

λ ptpython                                                                       
>>> import fs.memoryfs                                                           
>>> import fs.expose                                                             
>>> fs                                                                           
<module 'fs' from 'fs\__init__.py'>                                              
>>> s = fs.memoryfs.MemoryFS()                                                   
>>> from fs.expose import dokan                                                  
>>> mp = dokan.mount(s, 'Q:')                                                    
Traceback (most recent call last):                                               
  File "<stdin>", line 1, in <module>                                            
  File "fs\expose\dokan\__init__.py", line 957, in mount                         
    _check_path_string(path)                                                     
  File "fs\expose\dokan\__init__.py", line 929, in _check_path_string            
    raise ValueError("invalid path: %r" % (path,))                               
ValueError: invalid path: 'Q:'                                                   
invalid path: 'Q:'                                                               
>>> mp = dokan.mount(s, 'Q:\\')                                                  
>>> mp                                                                           
<fs.expose.dokan.MountProcess object at 0x050FEBB0>                              
>>> s                                                                            
MemoryFS()                                                                       
>>> s.listdir()                                                                  
[]                                                                               
>>> s.listdirinfo()                                                              
[]

At this time Q: is mounted successfully and can is visible through explorer, and I've created a folder in it (using explorer's context menu), but the created folder does not show up with s.listdir().

And I've also tried to created a new folder with s.makedir('a'), this folder showed up with s.listdir(), but not in explorer. The mounted drive in explorer does not sync up with the memory fs in python.

And I used the following code to unmount the drive:

>>> dokan.unmount('Q:\\')
>>> Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "fs\expose\dokan\__init__.py", line 1096, in _do_mount
    mount(fs, path, **opts)
  File "fs\expose\dokan\__init__.py", line 998, in mount
    res = libdokan.DokanMain(ctypes.byref(opts), ctypes.byref(opstruct))
WindowsError: exception: access violation reading 0x00000010

This error was not raised right after my command returned. Actually the unmount method returned very fast, but it was after several more seconds did the error come up. I suspect it was the background process which crashed.

For pyfilesystem it was a fresh clone from https://github.com/Liryna/pyfilesystem.git , as for Dokan it was 1.0.0.4000 (version of dokan1.dll). I'll try the newer version of Dokan some time later.

Rondom commented 7 years ago

Current master in this repo should have all necessary changes.

BiatuAutMiahn commented 5 years ago

I can't get this to work, Dokany v1.3.0.1000

Output:

>>> from fs.memoryfs import MemoryFS
>>> from fs.expose import dokan
>>> fs = MemoryFS()
>>> # Mount device mount point
... mp = dokan.mount(fs, "Q:\\")
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan\__init__.py", line 1107, in _do_mount
    mount(fs, path, **opts)
  File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan\__init__.py", line 1011, in mount
    raise OSError("Dokan failed with error: %d" % (res,))
OSError: Dokan failed with error: -7
Traceback (most recent call last):
  File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan\__init__.py", line 983, in check_ready
    os.stat(path)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'Q:\\'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan\__init__.py", line 1018, in mount
    check_ready(mp)
  File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan\__init__.py", line 985, in check_ready
    check_alive(mp)
  File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan\__init__.py", line 976, in check_alive
    raise OSError("dokan mount process exited prematurely")
OSError: dokan mount process exited prematurely
Liryna commented 5 years ago

Will take a look when I can. Otherwise are you sure to use same 32/64 platform for dokan dll and python?

Le sam. 31 août 2019 à 23:09, BiatuAutMiahn notifications@github.com a écrit :

I can't get this to work, Dokany v1.3.0.1000

Output:

from fs.memoryfs import MemoryFS from fs.expose import dokan fs = MemoryFS()

Mount device mount point

... mp = dokan.mount(fs, "Q:\") Traceback (most recent call last): File "", line 3, in File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan__init.py", line 1107, in _do_mount mount(fs, path, **opts) File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan\init.py", line 1011, in mount raise OSError("Dokan failed with error: %d" % (res,)) OSError: Dokan failed with error: -7 Traceback (most recent call last): File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan\init__.py", line 983, in check_ready os.stat(path) FileNotFoundError: [WinError 3] The system cannot find the path specified: 'Q:\'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "", line 2, in File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan__init__.py", line 1018, in mount check_ready(mp) File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan__init.py", line 985, in check_ready check_alive(mp) File "C:\ProgramData\Miniconda3\lib\site-packages\fs\expose\dokan\init__.py", line 976, in check_alive raise OSError("dokan mount process exited prematurely") OSError: dokan mount process exited prematurely

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/PyFilesystem/pyfilesystem/issues/256?email_source=notifications&email_token=ABA4S65LOCPF4HSSOJ3YJQTQHLMY7A5CNFSM4CJUAUMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5TVIEY#issuecomment-526865427, or mute the thread https://github.com/notifications/unsubscribe-auth/ABA4S642QUVFWBZSHNCQ3SLQHLMY7ANCNFSM4CJUAUMA .

BiatuAutMiahn commented 5 years ago

Tried both, CWD was in the dokany dll folder

BiatuAutMiahn commented 5 years ago

I ran dokanctl to see options, enabled debug then found DebugView, tried to run mp = dokan.mount(fs, "Q:\\") again and I found this... image

Liryna commented 5 years ago

Oh this is helpful!

There is a version number in dokan py file normal. You would need to update the value but this also mean probably something in the python wrapper need to be updated also. Will try to take a look in the next weeks if you want.

Best regards Adrien JUND

Le ven. 6 sept. 2019 à 21:42, BiatuAutMiahn notifications@github.com a écrit :

I ran dokanctl to see options, enabled debug then found DebugView, tried to run mp = dokan.mount(fs, "Q:\") again and I found this... [image: image] https://user-images.githubusercontent.com/6149596/64455944-f6874500-d0bc-11e9-8c6a-3526448b86be.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/PyFilesystem/pyfilesystem/issues/256?email_source=notifications&email_token=ABA4S62BCHHIX75RJ5Z76FTQIKXDNA5CNFSM4CJUAUMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6D3W2Y#issuecomment-528989035, or mute the thread https://github.com/notifications/unsubscribe-auth/ABA4S66HDVLUWVTKTTAXZ5DQIKXDNANCNFSM4CJUAUMA .

BiatuAutMiahn commented 5 years ago

Changed: DOKAN_MINIMUM_COMPATIBLE_VERSION = 100 # this is release 1.0.0 to DOKAN_MINIMUM_COMPATIBLE_VERSION = 110 # this is release 1.0.0

and Q:\ has mounted!

Rondom commented 5 years ago

There is a version number in dokan py file normal. You would need to update the value but this also mean probably something in the python wrapper need to be updated also.

You need to a diff between dokan.h of 1.0.0 and latest Dokan and adapt the Python code accordingly. Otherwise you will get random crashes (which is why there are those version checks).

vpuhoff commented 4 years ago

https://github.com/nznaza/pyfilesystem_dokan

Liryna commented 4 years ago

If anyone would be interested, dokan-dev is still looking for an official dokan python wrapper! There is already some code base available here and there, it might not be much work to put everything together and make it available to the community 👍

Ping @nznaza

nznaza commented 4 years ago

I have finally decided to look back into my repo. (When I first uploaded it I got burned out, and studies and whatnot got in the way)

Creating, Deleting and Renaming Folders work. Copying Files works. Creating, Deleting and Modifying files works too, Moving files to another drive works too. Moving/renaming files it's the only major bug that I noticed. I'm looking into it. I have tested it with MemoryFS, but since it relies in PyFilesystem any should work.

If people test and report bugs I will happily look into them.

hugo-hur commented 2 years ago

Hi!

Any updates on this? Looking to use this to mount ramfs on Windows to display decrypted files using Python3.

hugo-hur commented 2 years ago

Could be willing to help with this, but would it be better suited here or the Dokany repo? Imho would be simpler to write an intermediate wrapper using C/C++ and then create python bindings to that intermediate wrapper optimized to use with Python? @Liryna

hugo-hur commented 2 years ago

And then most of the work would be associated with updating the C/C++ intermediate binding interface?

Liryna commented 2 years ago

@hugo-hur @kdschlosser Started a new python wrapper for Dokan https://github.com/kdschlosser/py_dokany From his last comment https://github.com/dokan-dev/dokany/issues/964#issuecomment-774325735 it looks like it is working and probably need to be finished / polished. Unfortunately, there was no update after that so we would need someone else to take over 💪

kdschlosser commented 2 years ago

No matter what I did it would not enumerate a directory properly. .. the sheer number duplicate requests was also an issue and made it not visible to use for what I had wanted to use it for.