search

nextcloud / desktop

đź’» Desktop sync client for Nextcloud
https://nextcloud.com/install/#install-clients
GNU General Public License v2.0
2.99k stars 788 forks source link

Support FileProvider API (VFS) in macOS #1337

Closed helmut72 closed 10 months ago

helmut72 commented 5 years ago

https://www.apple.com/macos/catalina-preview/features/

Third-party cloud service integration An all-new FileProvider API for cloud storage providers delivers a new way to seamlessly integrate their services into the Finder without requiring a kernel extension, helping to maintain the security of your Mac. Cloud storage providers can now deliver their apps through the Mac App Store.

marcotrevisan commented 1 year ago

Thanks @claucambra and Nextcloud !!!

helmut72 commented 1 year ago

@claucambra Also thank you 🚀

Nusephived commented 1 year ago

@misku @marcotrevisan @helmut72 @claucambra Thanks guys for this new amazing feature! Is there a guide to use the virtual file system? I don't find it in settings.

melMass commented 1 year ago

Awesome, which client version has it?

jesusdeveloper commented 1 year ago

Awesome, which client version has it?

I want know this also :)

marcotrevisan commented 1 year ago

@misku @marcotrevisan @helmut72 @claucambra Thanks guys for this new amazing feature! Is there a guide to use the virtual file system? I don't find it in settings.

Lol.. Don’t credit me for this, I only did a bit of beta testing :) For what I’ve seen it’s been merged and it MAY be added to the 3.9.0 milestone (you might want to watch for it under the Issues section). I don’t know if it will actually land in 3.9.x.

The beta build used here has no other documentation than this: https://github.com/nextcloud/desktop/issues/1337#issuecomment-1478386641

melMass commented 1 year ago

Lol.. Don’t credit me for this, I only did a bit of beta testing :)

That was still quite useful and helped to complete this PR so they are right: Thanks everyone involved :)

helmut72 commented 1 year ago

@misku @marcotrevisan @helmut72 @claucambra Thanks guys for this new amazing feature! Is there a guide to use the virtual file system? I don't find it in settings.

Also no credit for me. I just opened the feature request. ;)

misku commented 1 year ago

@misku @marcotrevisan @helmut72 @claucambra Thanks guys for this new amazing feature! Is there a guide to use the virtual file system? I don't find it in settings.

I just thanked the original developer for the great work ;-D

mamboone commented 1 year ago

A personal thanks to all of you from my side as well. Great job!!! Love to see this feature finally arriving within Nextcloud which I looked for eagerly after being used to it within OneDrive Business for a long time. I just upgraded my NAS to the latest 25 version yesterday (25.6) (I cannot update to 26 via the integrated updater - don't know why. 26 will just not be offered to me). Therefore the same question as Nusephived had it above: any documentation how to use it - even in beta state? And what the requirements are? I have Mac OS Nextcloud Desktop 3.8.1 and experimental features enabled. But it just shows me the "suffix" based virtual files handling which I do NOT want to use. Which server version is needed? Which client version? Which settings? Thanks a lot!

claucambra commented 1 year ago

Hey all, no need to give thanks, I wanted to use this as much as you did 🙂

Now that the relevant PR has been merged, the file provider module will be included in version 3.9. The module is now also already included in the unstable daily builds of the desktop client.

For the moment, the file provider module is hidden and disabled by default as we have some rough edges still to iron out. It can, however, be enabled by adding macFileProviderModuleEnabled = true to your client's nextcloud.cfg file. Be warned that this module is still not ready for use in production environments!

Finally, we are also providing a new version of the macOS VFS Beta version of the desktop client that can be installed alongside the normal desktop client. You can download it here:

https://cloud.nextcloud.com/s/8JnmYA8a7xDm9tT

Regarding the issues that you guys have brought up:

  1. @marcotrevisan Unfortunately I haven't yet had the time to properly resolve several of the issues you were facing with borked change fetching after system shutdown, or the other issues. This is on the todo list for now and will need to be resolved before considering this feature stable
  2. Initial sync is still very slow and this will need to also be resolved
  3. There is no GUI management of the file provider sync module, this is -- you guessed it -- also on the todo list.
marcotrevisan commented 1 year ago

Hi @claucambra, thanks for your feedback!

As for the open points: since this ticket is closed and is full of comments, if you prefer I may create separate tickets -one for each open point- that link to this. I'll download the new beta later today and will be able to check for possible resolution of some of the open points.

Best regards!

BJKle commented 1 year ago

Hi, thanks for the great work.

a) Is there a minimum server version required? b) Do we need modules or a special configuration on the backend?

marcotrevisan commented 1 year ago

Hi @BJKle, my 2 cents on your questions:

a) Is there a minimum server version required? b) Do we need modules or a special configuration on the backen?

a) NC server v25.x is quite sufficient. Lower versions should work, if I remember correctly @claucambra wrote he developed it against the NC 23 webdav API. b) Notify_push is suggested but not mandatory. It should work with a "defaults" configured server. For the sake of testing I would recommend against external storage folders, but that is to keep things easier to reproduce.

Hope this helps. Cheers!

claucambra commented 1 year ago

Hi @claucambra, thanks for your feedback!

As for the open points: since this ticket is closed and is full of comments, if you prefer I may create separate tickets -one for each open point- that link to this. I'll download the new beta later today and will be able to check for possible resolution of some of the open points.

Best regards!

Separate issues would be great :)

Saifudaullah commented 1 year ago

Hey all, no need to give thanks, I wanted to use this as much as you did 🙂

Now that the relevant PR has been merged, the file provider module will be included in version 3.9. The module is now also already included in the unstable daily builds of the desktop client.

For the moment, the file provider module is hidden and disabled by default as we have some rough edges still to iron out. It can, however, be enabled by adding macFileProviderModuleEnabled = true to your client's nextcloud.cfg file. Be warned that this module is still not ready for use in production environments!

Finally, we are also providing a new version of the macOS VFS Beta version of the desktop client that can be installed alongside the normal desktop client. You can download it here:

https://cloud.nextcloud.com/s/8JnmYA8a7xDm9tT

Regarding the issues that you guys have brought up:

  1. @marcotrevisan Unfortunately I haven't yet had the time to properly resolve several of the issues you were facing with borked change fetching after system shutdown, or the other issues. This is on the todo list for now and will need to be resolved before considering this feature stable
  2. Initial sync is still very slow and this will need to also be resolved
  3. There is no GUI management of the file provider sync module, this is -- you guessed it -- also on the todo list.

Hi @claucambra, I'm unable to access the provided link. Can you provide another link for me to test this beta client?

BJKle commented 1 year ago

@claucambra Any news or timeline?

marcotrevisan commented 1 year ago

I'm using the beta 2 client for a while now, under Mac OS Ventura. I'd like to encourage the team to "do the last mile" asap, because even if it has its own glitches, it's also already showing some great advantages over the existing implememtation (not only the NC Client, but other webdav clients too). Thanks!

kklem0 commented 1 year ago

Can I test it with the daily build since it is still in the master branch? https://download.nextcloud.com/desktop/daily/

claucambra commented 1 year ago

Can I test it with the daily build since it is still in the master branch? https://download.nextcloud.com/desktop/daily/

We had a major regression with the Finder integration caused by the File Provider module so we had to disable building of the File Provider module by default. So the dailies do not have the module enabled. If you are brave enough to try building the client yourself, however, you can toggle the compilation flag and build the client with the module enabled

pete1019 commented 1 year ago

Hi, is the M1 support with virtual drive back in the daily? Otherwise where to get one without need to build? Thanks

marcotrevisan commented 1 year ago

I'm just guessing but I'd expect to see this being closed first: https://github.com/nextcloud/desktop/issues/5693 You might want to follow the activity on this feature branch: https://github.com/nextcloud/desktop/tree/feature/file-provider-config-ui

My 2 cents

szaimen commented 11 months ago

Reopening because fileprovider support was reverted for the time being iirc

BJKle commented 11 months ago

Is there a target release?

claucambra commented 10 months ago

Reopening because fileprovider support was reverted for the time being iirc

Reclosing, we are shipping working code in master at the moment ( we just are not enabling by default in normal releases as we are in the process of creating a specific version of the client with the file provider engine )

We also have the VFS betas for use at the moment

Is there a target release?

At the moment no, we are being cautious about deadlines considering we have had unexpected hiccups when it comes it existing integrations, for instance. At the moment we are working on getting the FIle Provider engine better integrated into our GUI and on improving initial sync times; once this is done, we will probably be ready to release something we are happy to call stable :)

Saifudaullah commented 10 months ago

@claucambra How do we test the beta? Is there any link that I can download the beta?

marcotrevisan commented 10 months ago

@claucambra How do we test the beta? Is there any link that I can download the beta?

I see the beta package here (not sure if it's the latest version of it): (folder) https://download.nextcloud.com/desktop/prereleases/virtualdrive/ https://download.nextcloud.com/desktop/prereleases/virtualdrive/Nextcloud-VFS-beta-macOS.pkg

Saifudaullah commented 10 months ago

@marcotrevisan I'm afraid that is not the latest one. The timestamp of the installer is 2023-03-21.

claucambra commented 10 months ago

Hi all, we've built a new version of the File Provider based VFS beta based on the current stable branch of the desktop client.

You may download the macOS VFS Beta 3 from the link below:

https://download.nextcloud.com/desktop/prereleases/virtualdrive/Nextcloud-stable-3.10-fileprovider.pkg

Key changes in this beta include some small stability improvements, as well as all changes in the latest 3.10 desktop client :)

kklem0 commented 10 months ago

Hi all, we've built a new version of the File Provider based VFS beta based on the current stable branch of the desktop client.

You may download the macOS VFS Beta 3 from the link below:

https://download.nextcloud.com/desktop/prereleases/virtualdrive/Nextcloud-stable-3.10-fileprovider.pkg

Key changes in this beta include some small stability improvements, as well as all changes in the latest 3.10 desktop client :)

Do you think the problematic issue you mentioned earlier is fixed?

marcotrevisan commented 10 months ago

Hi @claucambra, thanks very much for the new beta! I noticed that now hitting the space bar on a not-already-materialized item will cause materialization and subsequent preview, nice! I still can't get precisely under what conditions (maybe filename patterns?), but looks like the previews that available on the server aren't actually used for the icons of unmaterialized files, except for some of them in my private folders. That's weird... is it a known issue?

Thanks!

Saifudaullah commented 10 months ago

Thank you so much for the new beta @claucambra. I've installed it, and I assume to enable the file provider, we need to add macFileProviderModuleEnabled = true to the client's nextcloud.cfg.

I can say that this beta seems promising and quite nice to be used. Now it really feels like a proper implementation of virtual file system.

Below are my feedback:

  1. The initial sync is really slow.
  2. The loading indication spinning forever when I open empty folder
  3. Folders not showing size
  4. Folders that are already synced (downloaded) still showing dotted cloud icon

I also has a question regarding Nextcloud folder. Is it normal that we have two Nextcloud folders? One under "/Users/"username"/Nextcloud" and another one under "/Users/"username"/Library/CloudStorage/Nextcloud-"nextcloud_username"@domain/"?

marcotrevisan commented 10 months ago

@Saifudaullah I didn't have to set macFileProviderModuleEnabled = true in my config.

This build is still including both sync engines, so I'd suggest you to uncheck the root folder to practically disable the legacy engine. The legacy engine is responsible for "/Users/"username"/Nextcloud" (it will still be there but it won't sync).

As you noticed the initial sync is slow and until it's finished you will experience delays in the UI. Your point 2 is not showing up in my client so I'd suggest to wait a bit more before starting your test. I can also confirm your points 3. and 4. Point 3 is a known issue and in to-do list if I remember correctly.

Regards,

claucambra commented 10 months ago

Thank you so much for the new beta @claucambra. I've installed it, and I assume to enable the file provider, we need to add macFileProviderModuleEnabled = true to the client's nextcloud.cfg.

I can say that this beta seems promising and quite nice to be used. Now it really feels like a proper implementation of virtual file system.

Below are my feedback:

1. The initial sync is really slow.

2. The loading indication spinning forever when I open empty folder

Points 1 and 2 are related. At the moment we require a full scan of the server folder hierarchy during initial sync. We are considering how to address this

3. Folders not showing size

Yeah, aware of this bug -- thanks for raising

4. Folders that are already synced (downloaded) still showing dotted cloud icon

Can reproduce, still not 100% sure why this is the case. Will investigate

claucambra commented 10 months ago

As an aside, I've opened a Discussions thread to continue any discussion on the VFS beta as discussing on a closed thread is not ideal :)

https://github.com/nextcloud/desktop/discussions/6267

claucambra commented 10 months ago

Do you think the problematic issue you mentioned earlier is fixed?

I can confirm it has been fixed, but there is unfortunately no way we are going to be able to ship both the Finder Sync extension and the File Provider module alongside each other. 3.11 is going to be released imminently and I can confirm we will not be including the File Provider module with this release either.

When this module is considered stable we will be shipping a new client edition alongside the normal client which foregoes the Finder integration for the conventional sync engine in favour of the File Provider module. It is likely we will also strip out the traditional sync engine from the VFS-focused edition of the desktop client.

None of this is final and is subject to change, though! Thoughts welcome.

marcotrevisan commented 10 months ago

forgot to mention: that glich is happening to me since the first beta. If I remember correctly (not 100% sure) beta2 made it partially work.

By the way, I noticed in my system (M1-Sonoma) the command “fileproviderctl ls domainname” seems not be implemented anymore, but the man page is unchanged… ?? Any clues on new ways to accomplish that are very welcome! Thanks!

I still can't get precisely under what conditions (maybe filename patterns?), but looks like the previews that available on the server aren't actually used for the icons of unmaterialized files, except for some of them in my private folders. That's weird... is it a known issue?

funkemunky commented 10 months ago

When I download a file pressing "Download Now", the file does indeed download. However, the "Remove download" button is missing when I want to set it to cloud-only.