Offline files on other apps without opening Nextcloud app

ktuulos commented 4 years ago

On Dropbox, it is possible to mark certain files to be available offline. Those files can then be opened on other applications, when Dropbox is offline, without opening the Dropbox application. Those files are also updated automatically whenever Dropbox gets online.

On Nextcloud, when a file is marked as available offline, it can not be opened from other application. It can be opened only through Nextcloud application. This makes it very difficult for opening those files, compared using on e.g. desktop, where a file can be opened just by navigating to file and opening it.

One great use case of these is that on Dropbox, it is possible to create shortcuts to certain files, which are available offline, and place those shortcuts to home screen. Then, when these shortcuts are clicked, the application is directly opened without opening the Dropbox application. Similar behaviour would be needed for Nextcloud also.

tobiasKaminsky commented 4 years ago

You can open any downloaded file via a file browser, or if the other app supports it via document storage system (like an open).

So e.g. if you have keepass marked as available offline, you can use keepass2android to browse for it. There is no need to open Nextcloud for it.

ktuulos commented 4 years ago

OK, now I realized, what the problem is. Nextcloud browser gives such path, which can not be opened outside of Nextcloud browser.

When a file is opened directly from Nextcloud browser, the 3rd party application will get file name as follows: content://org.nextcloud.files/external_files/emulated/0/nextcloud/.... This path can not be opened directly from 3rd party application.

When the application navigates itself to the file, using system navigator, the path will be as follows: content://com.android.externalstorage.documents/document/primary¤3Anextcould%2F .... etc

So, the Nextcloud browser should be modified to give the permanent path, not a temporary path pointing to Nextcloud browser's cache.

pstorch commented 3 years ago

I use https://github.com/Kunzisoft/KeePassDX which uses the Nextcloud App as Document Provider. My keepass file has this green checkmark, which indicates it is available offline, but KeePassDX can't open the document URI if I'm offline.

t-m-w commented 3 years ago

This issue has been a major annoyance for me ever since I started using KeePassDX instead of Keepass2Android, which has native Nextcloud support.

Oddly, this problem was not present in 3.10.0 (January 17, 2020), which was the latest version available when this GitHub issue was opened. However, it exists today, and it has been around since November, affecting all versions 3.14.0 and up.

(Re-?)Introduction of Issue

The last unaffected stable release is 3.13.1 (September 15, 2020). The issue appeared (or returned?) in 3.14.0 (November 18, 2020), which touted a "Document storage enhancement by @trgrote" -- I'm not sure if that's related.

I am not sure if the issue was present prior to 3.10.0; I did not test that.

Expected Behavior

As written by @tobiasKaminsky,

You can open any downloaded file via a file browser, or if the other app supports it via document storage system (like an open).

So e.g. if you have keepass marked as available offline, you can use keepass2android to browse for it. There is no need to open Nextcloud for it.

Versions 3.10.0 - 3.13.1 allowed apps to successfully open the file and displayed a toast with message "File could not be synced. Showing latest version available." If the file was in fact not downloaded, it would try to download it and eventually time out.

Actual Behavior

Whether or not the file is downloaded, if the device is offline, the file cannot be opened using the file browser or any app using the documents UI / storage access framework. Most apps will immediately return an error: "Error downloading file: [file name]"

Testing

Apps affected

Tested with a few apps and my phone in Airplane Mode, and they all demonstrate this problem.

Editor (org.billthefarmer.editor)
- Dialog box showing "Error downloading file: [file name]"
KeePassDX (com.kunzisoft.keepass.libre)
- Produces the following error: java.io.FileNotFoundException: Error downloading file: [file name]
File browser (com.android.documentsui)
- If you try to copy the file, the notification eventually shows "Couldn't copy 1 item" and "Tap to view details" which displays only a generic error saying "This file wasn’t copied" and listing the file.
Termux (com.termux):
- Tried opening a file from file browser with Termux. Dialog box: "Unable to handle shared content:" (two newlines) "Error downloading file: [file name]"

Nextcloud versions affected and device details

Nextcloud app versions tested and affected: 3.14.0, 3.14.1, 3.14.2, 3.15.0, 3.15.1, 3.16.0, 3.16.1, 3.17.0 RC3, Dev 20210811, Dev 20210814 Nextcloud app versions tested and NOT affected: 3.10.0, 3.11.0, 3.12.0, 3.13.0, 3.13.1 Testing method: Opening a Nextcloud file with Editor (org.billthefarmer.editor) while in Airplane mode. Behavior is the same across all affected app versions tested. OS: Android 11 / CalyxOS 2.8.0 Device: Pixel 4a

Workarounds

If you have not already updated the data path for Nextcloud to a more secure location, you can open the file by browsing (via the file-handling app) to Internal Storage > Android > media > com.nextcloud.client > ...
Open the file directly from the Nextcloud app. This has a poor user experience with KeePassDX because it will eventually complain about access to the file being revoked, and the file will not show up in the app's recents list. There may be other quirks: e.g., I seemed to be prompted excessively to unlock the database, but I'm not sure how to replicate that.

pstorch commented 3 years ago

On the other hand I guess it might be a challenge to handle updates to those offline cached files. Handling conflicts properly, not just let the user choose between one of the versions, but merge.

newhinton commented 3 years ago

@t-m-w Interestingly i have checked the files that i assume are responsible for SAF (DocumentStorageProvicer.java)

The codepath's for DocumentStorageProvider.queryDocument(String documentId, String[] projection) haven't changed between 3.13.1 and now. So in theory it should work, but instead it throws nullpointer.

It seems that the FileDataStorageManager.getFileCursorForValue(String key, String value) returns an empty cursor when in offline mode (or the server is not reachable)

However, nothing has changed in those files, so im stumped why the ContentResolver does not work properly. This is the suspected faulty function: getFileCursorForValue

@tobiasKaminsky Any idea what could go wrong there?

Edit: Specifically this fails: https://github.com/nextcloud/android/blob/5eba647fb5f1654663ac6552e37d9db62a98b197/src/main/java/com/owncloud/android/datamodel/FileDataStorageManager.java#L949-L955

t-m-w commented 3 years ago

If it helps, I started building the dev- tags and testing until the problem began. It looks like the problem appeared in dev-20200903. dev-20200901 was fine. I haven't yet dug down into all the changes between those tags, but it looks like there are plenty, including in src/main/java/com/owncloud/android/providers/DocumentsStorageProvider.java and src/main/java/com/owncloud/android/files/services/FileDownloader.java

In index 6359b4da0..0f045e0eb 100644 for DocumentsStorageProvider.java, we even see the introduction of "Error synchronizing file", which is what I started to get after installing dev-20200903; this is when it stops even trying to access the already-downloaded file. Also, prior to this version, if the file was not downloaded, Nextcloud would try to download it even if you were offline (a notification would appear). Starting in dev-20200903, no attempt is made. So this seems like a good place to start looking.

I'm not an expert at any of this (including git), but I may try to take a look if I get time. Coming up on a trip where I won't have internet access at times and would love to have this fixed. (I guess I could just install v3.13.1, really.)

t-m-w commented 3 years ago

My log contains this error from the android-library when trying to access a downloaded file while offline:

2021-09-01 19:28:42 E CheckEtagRemoteOperation
Error while retrieving eTag

The behavior of CheckEtagRemoteOperation when an eTag cannot be retrieved (i.e. when you're offline) is to return ETAG_CHANGED. And I think this may be at the heart of the problem.

The hasServerChange function of the DocumentsStorageProvider returns true when this happens. https://github.com/nextcloud/android/blob/b2e8a35d87824aab903d88bd3a8c4238cacbd597/src/main/java/com/owncloud/android/providers/DocumentsStorageProvider.java#L257-L258

And when hasServerChange returns true, the openDocument function's check for whether a file needs to be downloaded will be true, too: https://github.com/nextcloud/android/blob/b2e8a35d87824aab903d88bd3a8c4238cacbd597/src/main/java/com/owncloud/android/providers/DocumentsStorageProvider.java#L200

I added some debug lines that show that hasServerChange is always true for my downloaded files when I'm offline.

As a test, I tried changing the code so that needsDownload is always false, and this did allow downloaded files to open when offline again (and give an empty error when they're not downloaded).

So I think the issue is as simple as Nextcloud coming to the conclusion, based on the incorrect idea that the eTag for a document has changed (whatever that means), that a file has been updated on the server and needs to be downloaded again. Is this, then, ultimately a problem with the android-library, or...? I don't know.

Since this issue hasn't gotten much attention, I attempted a fix myself in this PR and it appears to work for me: https://github.com/nextcloud/android/pull/8925

newhinton commented 3 years ago

@t-m-w Exceptional detective-work!

You are right, it seems that between da2352974cda69cf1125475f3b85c94c4154f955 (27.08.2020) and 1fdea37c2b745c72d8c2900c8dfbe540deef226d (20.08.2020?) The DocumentStorageProvider was rewritten, and didn't include your check for offlinefiles.

AndyScherzinger commented 3 years ago

Implemented via #8925

nextcloud / android