VFS on Linux

[tobiasKaminsky]

VFS on Linux needs to have a proper support on KDE/Gnome first, so first step would be to create and commit such a RFC. Then monitor it, so that we can afterwards implement it.

As many Linux users also use CLI, maybe we can also enhance our nextcloud to sync/upload/download, e.g.: nextcloudcmd myVirtualImage.png.nextcloud -> downloads and opens it

[vasyugan]

For consistency, VFS support on Linux should not be bound to any specific desktop environment. A situation where nautilus or dolphin show some file as present yet command line utilities or even graphical programmes that don't use the GNOME/KDE libraries don't would be rather odd and irritating.

[FlexW]

@vasyugan @tobiasKaminsky At the moment it is planned to register a custom xattr and read that when opening files. That would still need support from the program that reads the file, but it would not be bound to GNOME or KDE. A real vfs like on Windows and macOS would need support from the kernel side I think.

[johnnyasantoss]

pCloud does an excellent job at using virtual fs to browse/sync files. I think that it would be a great addition to the Nextcloud Linux Client if it could do the same. They created their own fs it might be a good reference https://github.com/pcloudcom/pclsync

[vasyugan]

A real vfs like on Windows and macOS would need support from the kernel side I think.

Couldn't such a thing be implemented using FUSE? This way, you would not have to get code into the kernel, I guess. In other words: Isn't that exactly what FUSE can do for you?

[mgallien]

A real vfs like on Windows and macOS would need support from the kernel side I think.

Couldn't such a thing be implemented using FUSE? This way, you would not have to get code into the kernel, I guess. In other words: Isn't that exactly what FUSE can do for you?

FUSE is not a good solution when network is involved because the normal file API you end up using to access FUSE filesystems is not able to cope with network errors. Currently we have an experimental VFS feature on all platforms that is using some suffix appended to files when they are virtual empty placeholders. You can activate it by manually editing your configuration file ($HOME/.config/Nextcloud/nextcloud.cfg) adding showExperimentalOptions = true in the first section. You can then use the settings dialog to enable it. Keep in mind that this experimental and not considered stable enough for daily usage. Reports are more than welcome (bugs, performance issues, suggestions, ...).

[AditiThirdEye]

Hello @FlexW , @mgallien , If the PR has not been done yet, can I work on this thread?

Regards, Aditi

[mgallien]

Hello @FlexW , @mgallien , If the PR has not been done yet, can I work on this thread?

Regards, Aditi

Hello Aditi feel free to contribute by opening a PR you can find some documentation on our workflow here wiki especially the page about Pull requests

[tennox]

For reference, here's a related fuse prototype for Owncloud: https://github.com/jnweiger/OCFFS

(also, for anyone searching for virtual file support, check this)

[vasyugan]

Currently we have an experimental VFS feature on all platforms that is using some suffix appended to files when they are virtual empty placeholders.

I suppose that this only works with GTK/QT applications, right? I definitely would need something that also works in the shell. I very often find that I can get things done quicker and with more ease in the shell.

[KaeTuuN]

@mgallien @tobiasKaminsky

1. Is there any (Dev) Documentation on how this feature Works on Linux?
2. Is there any Roadmap for this feature?
3. How can I support here with tests?

My employer is searching for a solution like nextcloud, but a must have is a VFS for Mac and Linux.

[tobiasKaminsky]

Currently there is no such thing for Linux. First we implemented it on Windows and then we want to focus on MacOS, as there is a proper cloud API already. Then we will start with Linux

[sinni800]

When I enable the experimental support that is there, it crashes when i hit the "enable virtual file support" button. The application segfaults and the last line in the log is this:

2022-01-08 10:10:33:653 [ warning default unknown:0 ]: QObject::connect(QDialog, OCC::AccountSettings): invalid nullptr parameter

[aviallon]

Using FUSE for such a feature is clearly the right thing to do here, in my opinion.

[Cloud-Kid]

Any update on this ? I use Arch + Gnome and the client really lack of this feature :(

[vasyugan]

Using FUSE for such a feature is clearly the right thing to do here, in my opinion.

As @mgallien writes, fuse is unable to cope with network errors, so it would need to be fixed first.

[aviallon]

Using FUSE for such a feature is clearly the right thing to do here, in my opinion.

As @mgallien writes, fuse is unable to cope with network errors, so it would need to be fixed first.

Correct me if I'm wrong, but I think kio-fuse is able to cope with network errors, and actually implements webdav support. Couldn't we look at what they currently do?

[SuperSandro2000]

I did a little research and at least for dolphin/kio the xattr support does not seem to be great. Also there is no universal xattr support so many tools and programs seem to strip them on save.

ref https://userbase.kde.org/Baloo

[hardwareadictos]

Is showExperimentalOptions still available on v3.4.4? I'm on Fedora 36 but i am not able to see it under settings rear adding the line on the config file and restarting.

[mgallien]

Is showExperimentalOptions still available on v3.4.4? I'm on Fedora 36 but i am not able to see it under settings rear adding the line on the config file and restarting.

yes should still work as we did not change anything there

[hardwareadictos]

Is showExperimentalOptions still available on v3.4.4? I'm on Fedora 36 but i am not able to see it under settings rear adding the line on the config file and restarting.

yes should still work as we did not change anything there

¿Maybe a Fedora issue? On F35 it was working fine.

[tennox]

¿Maybe a Fedora issue? On F35 it was working fine.

Same for me - my solution was to switch from flatpak version to AppImage

Works now :)

[yajo]

If using flatpak, the file can be found in ~/.var/app/com.nextcloud.desktopclient.nextcloud/config/Nextcloud/nextcloud.cfg

[Morganlej]

To those who say it is working for you on Linux: what is working and what not? What are the problems?

For me it is good enough if it only works in file browser, and when I click it it gets downloaded without the extra name suffix, as described here.

(Tried to enable it in the readily packaged client on Mageia, but it segfaults like described Jan 8 and just here above. I might try Flatpak or Appimage another day)

[tobiasKaminsky]

You need to:

1. Set showExperimentalOptions=true in the the nextcloud.cfg
2. Associate *.nextcloud files with Nextcloud client executable, so they get opened with desktop client when double-clicked
3. switch to VFS, best is to set up a new folder synchronisation

[XueSheng-GIT]

What are the problems?

While testing on NC25, I noticed that the new feature "local editing of files" doesn't work with VFS enabled. At least if the file was not syned/downloaded yet.

[Zocker1999NET]

fuse is unable to cope with network errors, so it would need to be fixed first.

What about the handling sshfs uses? That worked good enough for me in the past.

In general I would also prefer a FUSE solution as it would work natively with all applications (GUI & shell) and if implemented even allows to save bandwidth when seeking in a big file. E.g. when an application like file only needs a few blocks of a file, with FUSE you could translate those seekings to their corresponding HTTP range requests (this library try to implement that). The current solution wouldn't allow such improvements later without changing the whole concept.

[ACiDGRiM]

I'm using the AppImage and I created a custom mime xml for .nextcloud and associated it with the AppImage executable. but when I pass it in cmd or open with the .nextcloud file, I get "nextcloud.gui.application: Already running, exiting..."

[mgallien]

I'm using the AppImage and I created a custom mime xml for .nextcloud and associated it with the AppImage executable. but when I pass it in cmd or open with the .nextcloud file, I get "nextcloud.gui.application: Already running, exiting..."

that is expected and it should forward the request to the already running instance

[ACiDGRiM]

I'm using the AppImage and I created a custom mime xml for .nextcloud and associated it with the AppImage executable. but when I pass it in cmd or open with the .nextcloud file, I get "nextcloud.gui.application: Already running, exiting..."

that is expected and it should forward the request to the already running instance

Understood, I didn't know that is how it's supposed to behave. However no file gets downloaded or indication that it's waiting for sync. Is there a debug facility?

[mgallien]

I'm using the AppImage and I created a custom mime xml for .nextcloud and associated it with the AppImage executable. but when I pass it in cmd or open with the .nextcloud file, I get "nextcloud.gui.application: Already running, exiting..."

that is expected and it should forward the request to the already running instance

Understood, I didn't know that is how it's supposed to behave. However no file gets downloaded or indication that it's waiting for sync. Is there a debug facility?

yes in the settings dialog, one can generate a debug archive to get an archive with the logs or you can try reading them on your own should be located in ~/.config/Nextcloud/logs

[ACiDGRiM]

I'm using the AppImage and I created a custom mime xml for .nextcloud and associated it with the AppImage executable. but when I pass it in cmd or open with the .nextcloud file, I get "nextcloud.gui.application: Already running, exiting..."

that is expected and it should forward the request to the already running instance

Understood, I didn't know that is how it's supposed to behave. However no file gets downloaded or indication that it's waiting for sync. Is there a debug facility?

yes in the settings dialog, one can generate a debug archive to get an archive with the logs or you can try reading them on your own should be located in ~/.config/Nextcloud/logs

I see what the issue is...I have symbolic links to Desktop,Documents,Downloads,etc... in my home directory which allow me to right click and choose the Nextcloud options "make available online, share, etc..." but because the file's symbolic path is not in "Nextcloud" folder, it's ignored by the VFS handler.

It would be convienent if the .nextcloud files stored an obfuscated file identifier within them, so that the desktop client relied on that instead of the path of the file.

I can reproduce this issue by changing the directory directly to where the file is stored, and running nextcloud.AppImage myfile.pdf.nextcloud This fails to launch the file, whereas nextcloud.AppImage /home/user/Nextcloud/Documents/myfile.pdf.nextcloud Does download and launch the file successfully

[jeremyfritzen]

Hi!

Don't know if it's the right place to say it: I really love Nextcloud and job made by developers/contributors :)! That's why I would love even more to have the VFS feature on Linux.

Any idea on when it will be possible to enjoy this feature in a stable version?

Thanks a lot!

[antrmn]

I'm using Nextcloud 3.9.0 (Fedora 39, non flatpak). I've noticed that there's a row in .config/Nextcloud/nextcloud.cfg like this:

[General]
...
isVfsEnabled=false

Toggling this option seems to have no effect. Also, whenever I restart Nextcloud, it just gets reverted to false. What's the purpose of it?

When I enable the experimental support that is there, it crashes when i hit the "enable virtual file support" button. The application segfaults and the last line in the log is this:

2022-01-08 10:10:33:653 [ warning default unknown:0 ]: QObject::connect(QDialog, OCC::AccountSettings): invalid nullptr parameter

@sinni800 did you manage to find a way to get it to work? I'm having the same issue.

[tnjoga]

Toggling this option seems to have no effect. Also, whenever I restart Nextcloud, it just gets reverted to false. What's the purpose of it?

i have the same problem using Debian 12 and Nextcloud 3.9.0, i have also tried different package sources but the problem occurred with all of them.

[palto42]

For the Solus Linux packaging I had to move the plugin files /usr/lib64/nextcloudsync_vfs_suffix.so and /usr/lib64/nextcloudsync_vfs_xattr.so to /usr/lib64/qt5/plugins. This was reported as an issue for the AppImage build #3781, maybe worth to check the same on other distros?

[antrmn]

For the Solus Linux packaging I had to move the plugin files /usr/lib64/nextcloudsync_vfs_suffix.so and /usr/lib64/nextcloudsync_vfs_xattr.so to /usr/lib64/qt5/plugins. This was reported as an issue for the AppImage build #3781, maybe worth to check the same on other distros?

In the Fedora version there's no nextcloudsync_vfs_*.so file.

In the end I've dropped it and picked up the AppImage. Now VFS with suffixes works. (No idea for vfs_xattr, though)

[palto42]

The _vfs_ packages for Fedora are containe in the devel sub-package nextcloud-client-devel-3.9.0-1.fc38.x86_64.rpm, maybe because this is still an experimental feature? They are installed in /usr/lib64, so maybe they need to be moved as I commented above.

[antrmn]

The _vfs_ packages for Fedora are containe in the devel sub-package nextcloud-client-devel-3.9.0-1.fc38.x86_64.rpm, maybe because this is still an experimental feature? They are installed in /usr/lib64, so maybe they need to be moved as I commented above.

Thank you for pointing it out. I installed the package, followed the steps above and can confirm that it is now possible to use the virtual files feature with the Fedora build of Nextcloud.

[Neshura87]

Fedora 38, Non Flatpak Version (3.9.0-1)

Installed nextcloud-client-devel, tried symlinking and copying nextcloudsync_vfs_suffix.so and nextcloudsync_vfs_xattr.so to /usr/lib64/qt5/plugins/ but isVfsEnabled keeps resetting to false. Am I missing something obvious here? Do I also need to set showExperimentalOptions = true for it to work?

[afzl-wtu]

Any roadmap?

[mgallien]

having a proper working VFS implementation on linux is definitely a big amount of work needing support from current linux desktop implementation (think of files explorers, virtual files stack, ....) gathering common interest on the feature from a broad set of people would help a lot

[mgallien]

vfs_suffix has way too much limitations to really want to use this as the solution even if all bugs were fixed

[KaeTuuN]

@mgallien The missing VFS implementation for Linux is one of just a few Blockers left for switching over to Linux as the primary Client OS. Is there any realistic guess on how much work it would be to implement it? Maybe there are other companies out there willing to let one of their devs do the work or pay a dev for it!?

[tomassplatch]

For the Solus Linux packaging I had to move the plugin files /usr/lib64/nextcloudsync_vfs_suffix.so and /usr/lib64/nextcloudsync_vfs_xattr.so to /usr/lib64/qt5/plugins.

I can confirm that doing this after installing nextcloud-client-devel, and adding showExperimentalOptions=true worked for me in Fedora.

[antrmn]

FUSE is not a good solution when network is involved because the normal file API you end up using to access FUSE filesystems is not able to cope with network errors.

Doesn't rclone use FUSE to mount remote storages? Are they settling with a flawed technology?

[r2evans]

@tobiasKaminsky I'm running Ubuntu 23.04, have 3.9.4-20230908.164710.c8fec0118-1.0~lunar1 installed, and with experimental features enabled, VFS is working for me: when I open the .nextcloud file (generally with xdg-open, optionally with nautilus), it is downloaded/renamed and opened correctly.

However, there are files I want to download and not open. Often this is because I need to open it in a non-standard app, or want to pre-download so that I can reference the files later. (Examples: duckdb databases, data files to process with R or python, etc). The only way I've found to do this is to do a "full open" and then immediately exit the default application. In some cases this is just inconvenient, for others it might have consequences (large cpu/memory blooms, "on-open" macros in the file itself, etc).

I see your first post mentioned the suggestion of

nextcloudcmd myVirtualImage.png.nextcloud -> downloads and opens it

In the documentation, in addition to the credentials required to connect, it looks as if it is expecting to sync to a different directory. Is it your intention with this plan to be able to do this inside a directory being serviced by the full-up nextcloud desktop app on individual files?

cd ~/my-nextcloud-syncdir/some-subdir/
nextcloudcmd myVirtualImage.png.nextcloud

and expect that it would download it locally? Perhaps with --no-open as an argument to short-circuit the default-app usage.

I've seen other comments against implementing per-file selection of auto-sync in the app. Personally, I don't think the nextcloud desktop settings modal/app is the right place for individual files, I could easily see the nextcloudcmd (or some other companion utility) communicate with the full app to trigger some actions (such as make-available-locally).

[darkdragon-001]

Linking the discussion in https://gitlab.freedesktop.org/xdg/xdg-specs/-/issues/83 for visibility. I think a Freedesktop spec would be the right place.

[mgallien]

Additional context about things happening related to this topic https://www.volkerkrause.eu/2023/09/23/nextcloud-conference-2023.html https://dragotin.codeberg.page/posts/openvfs/

[darkdragon-001]

FUSE Passthrough Mode supported since Linux 6.9 could potentially solve the performance gap.

[ados8]

Really hoping Linux catches ground with Windows features. I started transitioning Windows desktops to Linux and was shocked by lack of Nextcloud features. It's generally recommended to run the server on the Linux platform, so I assumed it would function better on Linux desktop. There's no virtual files, no thumbnail previews, no right click options etc. I enabled the experimental option and it appeared to work, rebooted and then it starting syncing everything. Can see that virtual files is enabled, double checked the config but it was syncing everything. Really want to daily drive Linux but might have to stick with Linux servers and Microsoft Windows.