search

nextcloud / android

📱 Nextcloud Android app
https://play.google.com/store/apps/details?id=com.nextcloud.client
GNU General Public License v2.0
4.04k stars 1.74k forks source link

[TOP] Enhance sync folders #285

Open tobiasKaminsky opened 7 years ago

tobiasKaminsky commented 7 years ago

As discussed on conf this is a collaboration between @AndyScherzinger and me. Purpose:

aparcar commented 7 years ago

Does this include a constant sync of a todo.txt file on change locally and periodically checked on server?

tobiasKaminsky commented 7 years ago

@aparcar currently this will only replace instant upload. But this approach is designed to also handle a real two way sync and a constant sync of files.

AndyScherzinger commented 7 years ago

This issue/implementation is to be considered the top prio since we now already have > 1,300 Android 7 installations according to the play store :exclamation:

tobiasKaminsky commented 7 years ago

Camera should be top of the list as this is the most wanted "folder", I guess. At least I was searching a while where it is ;-)

tobiasKaminsky commented 7 years ago

Uhh, and caching is not working properly, right?

tobiasKaminsky commented 7 years ago

I know it is kind of reduntant, but I think that it should be possible to enable/disable sync also in the setting of a folder (and not only in the folder sync view). [folder sync looks fantastic! :+1: :tada: ]

AndyScherzinger commented 7 years ago

Yeah, we can put camera on top, need to think how to put this into the comparator. Doable :) As for the caching, don't know actually. The queries need to be tweaked also since they need to order by date.

AndyScherzinger commented 7 years ago

And yes, the enable/disable can also be put into the settings. UI is close to feature complete. Then I will publish some screenshots for the discussion with the designers.

AndyScherzinger commented 7 years ago

Camera folder at the top (of the inactive): DONE

AndyScherzinger commented 7 years ago

Let's move v1 discussions to the newly created PR #358

tobiasKaminsky commented 7 years ago

V1.1:

jasonbayton commented 7 years ago

Do version numbers and milestones need to be adjusted here?

AndyScherzinger commented 7 years ago

Yes, removed since this is a long term roadmap

rugk commented 7 years ago

Is there any ETA for this?

ShaunCurrier commented 7 years ago

Auto upload is also not working for me on Google Pixel XL with Android 7. Release version of Android app on Play Store.

jasonbayton commented 7 years ago

@shauncurrier please create a new issue. Ensure you add the troubleshooting steps you've taken also and for how long files have been waiting to upload (or if they don't make it to upload at all).

ShaunCurrier commented 7 years ago

My mistake! I was in a hurry and thought I was posting in a forum thread I had open in another tab. I'll take it elsewhere, apologies.

madorian commented 7 years ago

At what increment are we now at? Shouldn't we have a separate bug for each of them?

AndyScherzinger commented 7 years ago

@madorian I just updated the top description and we are actually implementing what I described in Increment 2 now - #797. We try to keep the discussion here in general but it would make sense to still create separate issues for the different functions but keep this one as an umbrella issue referencing the detail issues and to reflect the complete road map of this feature in one place.

Tormen commented 7 years ago

Hi, Any idea when the "upload existing files" feature might become user testable ? Tormen

Bullnados commented 7 years ago

Yes, and also slowly start including android 8 support 😃

W-Hamra commented 6 years ago

First off, thanks a lot for your great work on this app, it's awesome. On which increment are we standing now?

AndyScherzinger commented 6 years ago

Increment 2 has been implemented and will ship with the next stable release

mario commented 6 years ago

Well, we also have:

show jobs in upload pane track jobs get job result to update upload pane if job fails: allow user to restart

mario commented 6 years ago

Do note that this roadmap/issue will be changed once @tobiasKaminsky gets the time to put notebook notes in the issues :) cc @AndyScherzinger

Orybon commented 6 years ago

Does the work on increment 3 has started ?

AndyScherzinger commented 6 years ago

@Orybon not yet. Priority is bugfixing 2.0 and e2e encryption.

Orybon commented 6 years ago

Thank you for your answer @AndyScherzinger

rgarrigue commented 6 years ago

About increment 3, autoupload existing : is there a workaround to force upload of the existing ? maybe moving out / in via an android file manager ?

mario commented 6 years ago

Yes, that will work.

On Wed, Jan 3, 2018 at 2:44 PM, Rémy G. notifications@github.com wrote:

About increment 3, autoupload existing : is there a workaround to force upload of the existing ? maybe moving out / in via an android file manager ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nextcloud/android/issues/285#issuecomment-355014915, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAWsvuGVv9GX_fBSERTpZeaiO4Tlufxks5tG4RUgaJpZM4KDF9B .

dan0xii commented 6 years ago

Increment 4 - folder upload

allow folder hierarchies

Does this mean that "auto upload" will also be able to upload sub-directories and their contents? I'm looking for any issues related to that functionality and this is the closest I can find.

I have an application that writes files to dir x and numerous sub-directories under that path so I pointed auto-upload at x and it only uploads files in that directory, not sub-directories and their contents.

mario commented 6 years ago

This should already work really.

On Mon, 15 Jan 2018 at 18:11, dan0xii notifications@github.com wrote:

Increment 4 - folder upload

allow folder hierarchies

Does this mean that "auto upload" will also be able to upload sub-directories and their contents? I'm looking for any issues related to that functionality and this is the closest I can find.

I have an application that writes files to dir x and numerous sub-directories under that path so I pointed auto-upload at x and it only uploads files in that directory, not sub-directories and their contents.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nextcloud/android/issues/285#issuecomment-357741956, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAWsp6cznIuDvDVhN-Gzivn1qcVVO2Kks5tK4bXgaJpZM4KDF9B .

Bullnados commented 6 years ago

Is old folder syncing now possible?

AndyScherzinger commented 6 years ago

Hi @Bullnados, the implementation for the upload of already existing files hasn't been started yet. Sorry for the sad news. I hope that this can be achieved with 3.1 or 3.2 though.

Bullnados commented 6 years ago

@AndyScherzinger No prob. Nice to know that u are working on that feature 👍

holocronweaver commented 6 years ago

Assuming work hasn't begun, can I implement increment 3? Just need some broad strokes on what changes need to be made.

mario commented 6 years ago

First to clarify that we'll (hopefully) never support touching outside folders (e.g. we won't delete or replace things outside the Nextcloud data folder) via 2-way-sync.

That being said, increment 3 has in part started by doing this via this PR: https://github.com/nextcloud/android/pull/2253

The next step would be to completely remove the need for Sync adapter (check the code in syncadapter folder).

If you can do that, while keeping existing functionality (but please let me know you're working on it!), then I can take over and finish increment 3.

Awesome that you want to contribute @holocronweaver btw! :)

holocronweaver commented 6 years ago

First to clarify that we'll (hopefully) never support touching outside folders (e.g. we won't delete or replace things outside the Nextcloud data folder) via 2-way-sync.

Yeah, that sounds like asking for trouble. Though maybe we could support creating symlinks for users so they get the same effect?

As for removing dependence on Sync adapter, I will start looking at the code this week and will try to get out a WIP PR this weekend.

flohall commented 6 years ago

I wait for this better auto upload feature for some time. Currently I use FolderSync Pro for 2-way-sync of important images and data which is handy somehow, but makes it necessary to have two separate applications and also is not free software.

What I am not sure about so far, if you also considered to not just sync media files, but also any other folder with files (like chat backup files etc.). This would make it possible to backup all important data on the phone in a very handy way.

First to clarify that we'll (hopefully) never support touching outside folders (e.g. we won't delete or replace things outside the Nextcloud data folder) via 2-way-sync.

But if you don't want to implement it in that way, how the following is meant?

Increment 6 - sync two way

Somehow files outside nextcloud folder must be replaced/deleted so someone can order/delete photos on the computer on a bigger screen without having the need to do the same again on the device. Unlike on a desktop/laptop not all applications can work with files just stored in nextcloud. The application FolderSync can do the 2-way-sync of any folder in android.

Symlinks sound like a reasonable solution - just not sure if it works.

Thanks for the great work so far - looking forward to use the new features.

mario commented 6 years ago

@TheCod-e you can already sync custom folders if you enable expert mode in settings.

As for why I don't want to touch outside folders - it's a bad, bad thing :D Android is going the way of restricting access to your own folder (which will happen sooner or later), and it's increasingly hard to manage outside folders.

flohall commented 6 years ago

@mario thanks for the tip with expert options

ok you say no touching of outside folders - but is there maybe another way of avoiding redundancy of files? just believe it will be confusing to manage files twice (on the phone) and (in nextcloud)

tobiasKaminsky commented 6 years ago

First to clarify that we'll (hopefully) never support touching outside folders (e.g. we won't delete or replace things outside the Nextcloud data folder) via 2-way-sync.

As far as I know we never really talked about this, so please do not rush and let us talk about the overall sync concept before trying to start implementing things.

holocronweaver commented 6 years ago

@tobiasKaminsky @mario To clarify, this has nothing to do with increment 3, which I am working on, right? Seems like an increment 4+ issue.

mario commented 6 years ago

@tobiasKaminsky we talked about this between you, @AndyScherzinger and me months ago. Or did I mistake this for something else? cc @AndyScherzinger

mario commented 6 years ago

@holocronweaver it doesn't. While removing sync adapter isn't a real requirement for stage 3 either, it helps with cleaning up overall sync code.

mario commented 6 years ago

Btw. to clarify ... even one part of increment 4 is already implemented - we already support folder hierarchies in MASTER.

AndyScherzinger commented 6 years ago

That being said, increment 3 has in part started by doing this via this PR: #2253

Just my 2 cents...

2253 doesn't implement the upload existing files option as fare as I can tell while I'd also say that whenever the users activate the auto upload via the cloud icon for a folder the client should check for the existing files' existence on the server and if not present simply upload them. That would imho be the simplest solution for probably >80% of the cases where users are reporting "issues". And if this then works we could then also integrate it into the 3-dot menu to retrigger such logic.

While I am aware that this isn't a real 1-way sync (which isn't defined for increment 3 anyways) it would in my personal opinion still solve most peoples reports and as such help with the overall satisfaction.

And if this would be in place I'd say the next steps afterwards should be the "free up space" feature since this + the "free up space" will solve most use cases people are having (e.g. for the reports of "delayed deletion" because that could than me handled by manually triggering the "free up space" whenever one likes or needs to do that).

mario commented 6 years ago

Well, even 1-way-sync is damn near impossible for outside folders. You never know if server file == local file if it doesn't have the damn etag, and files in random folders won't have them :)

AndyScherzinger commented 6 years ago

true, while for the "simple upload existing files" we can simply upload them, no harm done to the original file on the client :)

korelstar commented 6 years ago

Other sync apps (e.g. Synchronize Ultimate) solve this problem by building a local database which tracks meta data of files in the sync folders when synchronizing. They compare files using file size and modified time (which is what an ETag is often generated from). In nearly all cases, this is sufficient for synchronizing files outside the app dir.

It would be very cool, if a full two-way sync of any folder could be a mid-term goal of the Nextcloud app. This would be nice in order to create backups from local files with the possibility to archive files using Nextcloud.