All files access switch greyed out when trying to grant files permissions

[theredbaron05]

EDIT BY CATFRIEND1; TL;DR: To everyone reading this issue. You are advised to switch to the GitHub release (APK can be downloaded from here) or the F-Droid release to get the full features of the Syncthing-Fork app without G's restrictions.

If you're stuck on the welcome assistant because of Android restrictions, please read https://github.com/Catfriend1/syncthing-android/pull/747 and use the emergency config export button to export your existing Syncthing config to "/storage/emulated/0/Android/media/com.github.catfriend1.syncthingandroid[.debug]/" for migration to another release channel of the app.

======== ORIGINAL POST ==============

Description of the issue

all files access switch greyed out when trying to grant files permissions

Reproducer

open a fresh install of syncthing fork and try to grant files permissions

Version Information

App Version: 1.9.0.2
Syncthing Version: v1.9.0
Android Version: Android 11 full release
Device manufacturer: Google
Device model: Pixel 2 G011A

``` ### Device platform info [ro.product.board]: [walleye] [ro.product.brand]: [google] [ro.product.build.date]: [Wed Jul 29 23:26:21 UTC 2020] [ro.product.build.date.utc]: [1596065181] [ro.product.build.fingerprint]: [google/walleye/walleye:11/RP1A.200720.009/6720564:user/release-keys] [ro.product.build.id]: [RP1A.200720.009] [ro.product.build.tags]: [release-keys] [ro.product.build.type]: [user] [ro.product.build.version.incremental]: [6720564] [ro.product.build.version.release]: [11] [ro.product.build.version.release_or_codename]: [11] [ro.product.build.version.sdk]: [30] [ro.product.cpu.abi]: [arm64-v8a] [ro.product.cpu.abilist]: [arm64-v8a,armeabi-v7a,armeabi] [ro.product.cpu.abilist32]: [armeabi-v7a,armeabi] [ro.product.cpu.abilist64]: [arm64-v8a] [ro.product.device]: [walleye] [ro.product.first_api_level]: [26] [ro.product.locale]: [en-US] [ro.product.manufacturer]: [Google] [ro.product.model]: [Pixel 2] [ro.product.name]: [walleye] [ro.product.odm.brand]: [google] [ro.product.odm.device]: [walleye] [ro.product.odm.manufacturer]: [Google] [ro.product.odm.model]: [Pixel 2] [ro.product.odm.name]: [walleye] [ro.product.product.brand]: [google] [ro.product.product.device]: [walleye] [ro.product.product.manufacturer]: [Google] [ro.product.product.model]: [Pixel 2] [ro.product.product.name]: [walleye] [ro.product.system.brand]: [google] [ro.product.system.device]: [walleye] [ro.product.system.manufacturer]: [Google] [ro.product.system.model]: [Pixel 2] [ro.product.system.name]: [walleye] [ro.product.system_ext.brand]: [google] [ro.product.system_ext.device]: [walleye] [ro.product.system_ext.manufacturer]: [Google] [ro.product.system_ext.model]: [Pixel 2] [ro.product.system_ext.name]: [walleye] [ro.product.vendor.brand]: [google] [ro.product.vendor.device]: [walleye] [ro.product.vendor.manufacturer]: [Google] [ro.product.vendor.model]: [Pixel 2] [ro.product.vendor.name]: [walleye] ``` ### Android Log [android log.txt](https://github.com/Catfriend1/syncthing-android/files/5190810/android.log.txt) 

[cryobry]

there is no version 0.9.13

Sorry, phone mangled the version number. Versions 1.9.0.3 and 1.9.0.4 work fine, but v1.10.0 still shows a greyed out storage option.

[Catfriend1]

Please be sure to download and install the github version (debug or gplay).

[cryobry]

This is when using v1.10.0 from the play store (I'm installing the legacy versions manually from your links in this thread).

[Catfriend1]

Just don't consume too much Google ..... !

[qorron]

got hit by this too. version on the phone is '1.11.1.0' Updated on 'Nov. 3. 2020' from gplay. when trying to install: https://github.com/Catfriend1/syncthing-android/releases/download/v1.9.0.3/com.github.catfriend1.syncthingandroid_gplay_v1.9.0.3_304261d3.apk it fails with "App not installed" it also fails to give any further reason on why it failed.

following this guide: https://github.com/Catfriend1/syncthing-android/wiki/Switch-between-releases_Verify-APK-is-genuine I can not get to step 1.1 'Open the drawer on the left side > Import & Export > Export configuration' because the app right after starting immediately requests the storage permission and I can not get past that screen to export the config.

what to do? shall I wait till next year? I'd rather not redo all my configuration and realize that I should have exported the config before upgrading to A11. unfortunately there is no easy way of knowing wich app requires its config to be exported before an OS upgrade.

[Catfriend1]

@qorron Pls try https://github.com/Catfriend1/syncthing-android/releases/download/v1.12.0.0/com.github.catfriend1.syncthingandroid_gplay-full_v1.12.0.0_8496068a.apk ?

[Catfriend1]

@qorron Did you work it out?

[qorron]

yes, it did! thank you very much! so, the non forked version was unaffected because it used an older API level and the fork version along with other apps that use the latest api are denied the permission until next year. also no exception was made by G because the function and purpose of the app was not understood.

well.. technically, as I currently understand it, syncthing may indeed not need access to the whole tree but to all the sub-dirs that are to be synced. unless of course one, for whatever reason, wants to sync the whole fs. I may be wrong with above assumption but if this is the train of thought G boarded with their decision then one can understand why they arrived at this conclusion.

[Catfriend1]

yes the most seems right like you did explain it. I don't have my free time for google, so I won't care anymore what they say , cease or whatever.

[auanasgheps]

Hey, I've tried all 1.11 versions available on GitHub, but on my new OnePlus 8T I am unable to choose the root of my storage. I have enabled All File Access, Am I missing something? EDIT: Tried 1.9.0.4, same thing. All File Access can be enabled but then can't select the root storage.

[Catfriend1]

Is it Android 11? Then you might need at least one folder below root because Google wants it?! Anyways, restrictions not caused by us should be reported to Google so they'll learn :-).

[auanasgheps]

Yes it's Android 11, forgot to mention. Do I really need to use a subfolder? I used to sync the whole storage as backup. I though giving All File Access or using F-Droid version would have made the trick... So now I'm unable to make full backups.

[Catfriend1]

Well I have no 11 device yet, but it was the same restriction on the emulator. It was not my idea to do that to your backups, sorry for being hit by Google. Maybe a custom ROM with 10, but that is too much of a workaround I think.

[auanasgheps]

I understand it's not your fault, but I read this thread in the past and this issue was not discussed, that's why I'm disappointed.

I have root and I can see this fork has root features, maybe it could be used to circumvent the issue?

[Catfriend1]

Sure, I'll appreciate your feedback if root works to circumvent in 11 :-) 👍

[auanasgheps]

I enabled the superuser option and granted rights, but this did not make a change. I guess the app has to request storage access in a different way. I'm not a dev but a technical user. I can't give any help as of now, but If you want to make some tests please let me know.

[auanasgheps]

Update: the official app with works because they are

requesting legacy storage handling

Would you consider this?

[Catfriend1]

We also request legacy storage, but Android 11 ignores the flag on API 30. Thanks for your feedback, I'll ping you when I continue working on this some day.

[auanasgheps]

Not on my device, that's why I am always available for testing.

The difference is that the debug version of the official app targets API 29, yours API 30.

(Sadly) I'm switching to the official app for the time being.

[Catfriend1]

fyi: Google will request sooner or later official to go to API 30 as they always did before regarding play publishers.

[auanasgheps]

We've still got a year and hope this gets fixed or properly implemented.

[Catfriend1]

great :-))

[tomasz1986]

Sure, I'll appreciate your feedback if root works to circumvent in 11 :-) 👍

I am sorry if this has been mentioned already, but with root it should be possible to simply set the paths using the Web GUI (or even config.xml) and completely ignore the OS limitations, right? Am I wrong here?

I personally have only had a chance to use Syncthing on Android 7 and older, so I have no experience with the newer versions, but in those running as root seems to be able to circumvent any OS limitations regarding the file system access.

[auanasgheps]

Sure, I'll appreciate your feedback if root works to circumvent in 11 :-) 👍

I am sorry if this has been mentioned already, but with root it should be possible to simply set the paths using the Web GUI (or even config.xml) and completely ignore the OS limitations, right? Am I wrong here?

I personally have only had a chance to use Syncthing on Android 7 and older, so I have no experience with the newer versions, but in those running as root seems to be able to circumvent any OS limitations regarding the file system access.

I'm not a dev, but if you actually want to make use of root, the app should not request access via the standard Android API but make use of a system call or something else that is not really official

I haven't tried setting up the path in the webgui, I'll give it a try

[qorron]

now that 2021 is here, has something changed with play-store and filesystem permissions? what would be the suggested way to use STF on A11 without root? (getting updates automatically, etc.) at the time of writing the newest version of STF in playstore is from 17.12.2020

[Catfriend1]

I did not try to re-submit the app including the all files permission to play yet. Maybe they now accept the APK or at least put it into policy review forever or the 403 upload error still applies F-Droid is a very good and updat-able source.

[auanasgheps]

Can you please update the status of this issue? I am still unable to use this fork on Android 11 because I can't select the root of the user storage.

[Catfriend1]

@auanasgheps Hi, there is no update. My phone is still on the most recent vendor update which means Android 10. Did entering the path to your user storage root via the web UI work? Do you want to use /storage/emulated/0 ?

Few days ago I looked through Google Developer docs - they still say all files access cannot be requested yet but later this year.

[Catfriend1]

@auanasgheps

I've made a new gplay build including the ALL FILES ACCESS permission but Google still refuses the upload to the play store.

Build log:

Uploading App Bundle: 21% complete
Uploading App Bundle: 41% complete
Uploading App Bundle: 62% complete
Uploading App Bundle: 82% complete

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:publishGplayBundle'.
> A failure occurred while executing com.github.triplet.gradle.play.tasks.PublishBundle$BundleUploader
   > 400 Bad Request
     PUT https://www.googleapis.com/upload/androidpublisher/v3/applications/com.github.catfriend1.syncthingandroid/edits/XXXX/bundles?uploadType=resumable&upload_id=YYYY
     {
       "code" : 400,
       "errors" : [ {
         "domain" : "global",
         "message" : "Cannot use MANAGE_EXTERNAL_STORAGE permission",
         "reason" : "badRequest"
       } ],
       "message" : "Cannot use MANAGE_EXTERNAL_STORAGE permission",
       "status" : "INVALID_ARGUMENT"
     }

[auanasgheps]

Thanks for showing the log but I already trusted you :) I don't know the exact behaviour of this permission, but could this build it be uploaded to F-Droid?

Did entering the path to your user storage root via the web UI work? Do you want to use /storage/emulated/0 ?

I finally made this test using the latest version from F-Droid. Adding a folder in the WebUI with the path /storage/emulated/0 works!!! I still can't access /Android/databut that's okay, at least I can backup my storage!

Please note I only tried "Send Only" and not receiving/writing to Android storage.

If you need more stuff please let me now. This is a great news.

[Catfriend1]

@auanasgheps Android/data is locked on 11, Android/media probably still works

[auanasgheps]

Android/data is locked on 11

That's what I thought but Solid Explorer just updated with the ability to browse Android/data without root on Android 11. It must be magic but works

[Catfriend1]

@auanasgheps There's no magic. We use the NDK and native methods to build Syncthing and Android limits natives in a way different how it limits a java-app.

[rodrigoswz]

Finally good news to definitely resolve this issue?

https://www.xda-developers.com/android-11-all-files-access-permission-form/

I currently use the F-Droid version to work around this problem.

[Catfriend1]

Maybe, but they currently blocked publishing new releases because the app description full text is not compliant (did not change this between releases and it was rejected from one apk to the other)

Ref. https://github.com/Catfriend1/syncthing-android/issues/774

[leo60228]

Finally good news to definitely resolve this issue?

As far as I can tell, Syncthing doesn't fall within the allowed usages.

[leo60228]

My understanding is that Android 11 is, as far as Syncthing is affected, applying the same restrictions to internal storage that were applied to external storage years ago. My understanding was also that the external storage restrictions could be solved, though it would require a large amount of effort due to Go having no good way to utilize the necessary Android APIs. However, the discussion here sounds like it's not possible to fix things on Syncthing's end. What am I missing?

[Catfriend1]

it would require a large amount of effort due to Go having no good way to utilize the necessary Android APIs.

That is an alternative solution, but no one is working on it as far as I know. So we head for the "shortcut" and use all files access to solve the readwrite access.

[imsodin]

As far as I can tell, Syncthing doesn't fall within the allowed usages.

How so? The following permitted usaged all apply to syncthing in my opinion (https://support.google.com/googleplay/android-developer/answer/10467955#zippy=%2Cpermitted-uses-of-the-all-files-access-permission):

File management

App’s core purpose involves the access, editing and management (including maintenance) of files and folders outside of its app-specific storage space

Backup and restore apps

App must have a need to automatically access multiple directories outside of its app-specific storage space for the purpose of backup and restore

Device Migration / Phone transfer

App’s core purpose is to help the user migrate to a new device

[leo60228]

"File management" and "Device Migration / Phone transfer" both specify "core purpose." While you could argue that the literal definition applies to Syncthing, I think you'd have a hard time arguing that Syncthing's core purpose is as a file manager, or moving permanently to a new device.

On the other hand, "Backup and restore apps" would fit, except it specifies that the app must need access to all files. From my understanding, there are several ways that Syncthing could avoid needing all files access or any other workarounds, but they would all require large rewrites.

On Fri, Apr 16, 2021, 7:00 AM Simon Frei @.***> wrote:

As far as I can tell, Syncthing doesn't fall within the allowed usages.

How so? The following permitted usaged all apply to syncthing in my opinion ( https://support.google.com/googleplay/android-developer/answer/10467955#zippy=%2Cpermitted-uses-of-the-all-files-access-permission ):

File management

App’s core purpose involves the access, editing and management (including maintenance) of files and folders outside of its app-specific storage space

Backup and restore apps

App must have a need to automatically access multiple directories outside of its app-specific storage space for the purpose of backup and restore

Device Migration / Phone transfer

App’s core purpose is to help the user migrate to a new device

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Catfriend1/syncthing-android/issues/704#issuecomment-821096133, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7X32KR5CHQ2RTUIZQ2SDTTJAKDJANCNFSM4RAOMVEA .

[imsodin]

You'll have to expand on that: I honestly don't understand how you could even argue that the following is not Syncthings core purpose: "access, editing and management (including maintenance) of files and folders outside of its app-specific storage space". Backup and restore is a bit less obvious to me, but it still holds: When I switch a device, I setup Syncthing and it restores all the files I need. Then it continuously syncs the files, such that they can be backed up on my server, the combination being backup. Having supported "a few" of our users, this is a very common use-case.
So again, how would you argue this is not Syncthings "core purpose"? I am not being defensive here, I truly don't understand. And understanding your concerns might make future communications with Google simpler/more likely to be successful. I am definitely not saying it will be easy to manage to get the permissions - nothing every is easy with google.

On the implementation part: It can't. Even if we had code to access the storage access framework, the nature of Syncthing (sync stuff, for most users almost everything, continuously), does not allow it to query for access on every operation. We do need access.

[leo60228]

My point is that the core purpose of Syncthing is syncing files. That requires accessing files, and is useful for device migration, but neither of those are themselves the core purpose.

On Fri, Apr 16, 2021, 7:30 AM Simon Frei @.***> wrote:

You'll have to expand on that: I honestly don't understand how you could even argue that the following is not Syncthings core purpose: "access, editing and management (including maintenance) of files and folders outside of its app-specific storage space". Backup and restore is a bit less obvious to me, but it still holds: When I switch a device, I setup Syncthing and it restores all the files I need. Then it continuously syncs the files, such that they can be backed up on my server, the combination being backup. Having supported "a few" of our users, this is a very common use-case. So again, how would you argue this is not Syncthings "core purpose"? I am not being defensive here, I truly don't understand. And understanding your concerns might make future communications with Google simpler/more likely to be successful. I am definitely not saying it will be easy to manage to get the permissions - nothing every is easy with google.

On the implementation part: It can't. Even if we had code to access the storage access framework, the nature of Syncthing (sync stuff, for most users almost everything, continuously), does not allow it to query for access on every operation. We do need access.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Catfriend1/syncthing-android/issues/704#issuecomment-821111109, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7X32PZTT2B57XTHEY6TJ3TJANVHANCNFSM4RAOMVEA .

[mpalermo73]

The "core purpose" is indeed "syncing files" for management, and backup/restore. I just did that this weekend.

Long story short myh Pixel 3 is stuck in a FastBoot boot loop. Working fine one day, doing that the next. Whatever. The point is I ran out and bought the Pixel 5 installed Syncthing and RESTORED 4.5G of MP3 onto the Pixel 5 so I didn't miss a beat. (pun intended ;-) )

The backup of my "mobile MP3 repo" is on my NAS (syncthing running in docker on Synology) which I restored to my new phone.

backup / restore / file management. All 100% syncthing's purpose to exist. I'm totally baffled that someone can think otherwise.

And just to say it, this comment isn't directed at any specific individual reading these words. I'm even including Google on this. I truly don't understand how it's arguing even exists.

M.

[imsodin]

My point is that the core purpose of Syncthing is syncing files.

Ok thanks for clarifying. Then I am good here, as there's no new/unknown to me argument: Syncing is managing files - case closed :)

[Catfriend1]

... just enjoying the feedback :-)

[Catfriend1]

Google still prohibits API 30 apps being uploaded with the MANAGE_EXTERNAL_STORAGE permission. I also see no form to apply for the permission on the fork's gplay console while the offer is showing up on official app's console.

Tried uploading the fork APK including that permission and it was rejected.

> A failure occurred while executing com.github.triplet.gradle.play.tasks.PublishBundle$BundleUploader
   > 400 Bad Request
     PUT https://www.googleapis.com/upload/androidpublisher/v3/applications/com.github.catfriend1.syncthingandroid/edits/XXX/bundles?uploadType=resumable&upload_id=XXX
     {
       "code" : 400,
       "errors" : [ {
         "domain" : "global",
         "message" : "Cannot use MANAGE_EXTERNAL_STORAGE permission",
         "reason" : "badRequest"
       } ],
       "message" : "Cannot use MANAGE_EXTERNAL_STORAGE permission",
       "status" : "INVALID_ARGUMENT"
     }

[auanasgheps]

Well, it's Google, what were you expecting? :D

[imsodin]

https://support.google.com/googleplay/android-developer/answer/10467955

Preview article (effective May 5, 2021)

Today == April 18 < May 5 ;)

[Catfriend1]

@imsodin : I mean this one. I didn't get this from Google yet, but official app received the inbox message.


[Catfriend1]

View of the fork's perspective: