search

winfsp / cgofuse

Cross-platform FUSE library for Go - Works on Windows, macOS, Linux, FreeBSD, NetBSD, OpenBSD
https://winfsp.dev
MIT License
511 stars 82 forks source link

Possibility to render folder asynchronously with Readdir #83

Closed nduong-ol closed 3 months ago

nduong-ol commented 4 months ago

Hello, I'm experimenting with cgofuse and winfsp to mount a cloud file system into windows. I'm noticing that when a folder is open through the file explorer, Readdir is called and a function fill() is passed as parameter for Readdir to fill the children attribute. If a folder is very large (hundred thousands children) the Readdir gonna take quite some times waiting for all the fill to complete. I don't know if there is any possibility to render the folder asynchronously (return the Readdir and then fill the children gradually and make the file explorer render gradually received children)?

billziss-gh commented 4 months ago

I do not believe that this is possible with the regular FUSE API.

WinFsp on Windows and FUSE low-level on UNIX do have the capability to satisfy a readdir asynchronously. But this capability cannot be exposed via the regular FUSE API.

nduong-ol commented 4 months ago

Is it possible to modify the current fuse API of cgofuse to make it possible? If yes, can you give me some pointers on how it could be done?

billziss-gh commented 4 months ago

As said this is not possible within the FUSE high-level API.

For Windows you would have to use the native WinFsp API. For UNIX you would have to use the FUSE low-level API. In either case cgofuse would not be of much help to you.

nduong-ol commented 4 months ago

Which Winfsp API should I use to implement this async mechanism?

billziss-gh commented 4 months ago

You would use the native WinFsp API:

https://github.com/winfsp/winfsp/wiki/WinFsp-API-winfsp.h

See the ReadDirectory operation.

nduong-ol commented 4 months ago

Thanks for the pointer. After digging a little bit, I found previous issue on rclone that mentioned that with the FileSystemHost.Notify() we can trigger the file explorer to reload the directory when we have new element available right?

billziss-gh commented 4 months ago

Yes, that's possible.

nduong-ol commented 4 months ago

Is the FileSystemHost.Notify() function only works on Windows? I saw the NotifyFs only made available for Windows. I tried to used this function on Mac but I wouldn't see the same effect as on windows, the Notify() return false. Could you tell me if I'm missing something?

billziss-gh commented 4 months ago

The notify functionality is only available on Windows. It is a WinFsp extension not available on FUSE.

nduong-ol commented 4 months ago

Do you have any recommendation on how we can port it to Fuse? I would make a PR to have this function available for Fuse

nduong-ol commented 3 months ago

Hello @billziss-gh with winfsp on windows, I only manage to notify the file explorer that there is a new file when emit an event on the specific file. The file explorer seems to add only the file indicated in the event instead of reloading the whole folder. This cause a few issue when I have a lot of files popping up, the file explorer tent to ignore these events. Is there anyway I can trigger a full reload of the file explorer instead of it just adding a single file per event at a time?