Fix file access problems on Chromebooks

gogins commented 10 months ago

This from Art Hunkins:

Especially for Michael G.:

Everything outlined below works just fine on earlier Chromebooks. But my more recent Samsung Chromebook Plus (v2) displays the following critical issue. All insight much appreciated.

FWIW the problem shows up with both Csound for Android on the Play Store, and Michael's latest github version. (No problems with the "alternative" Android Csounds on my website. I doubt whether this relates directly to the issue, however.) Below is what happens on Michael's two Android Csounds and the recent Samsung Chromebook Plus (v2) only.

Playing Xanadu from the Examples folder in Csound for Android: fine, as expected

Copy Xanadu from the Examples folder to the Google Drive folder; Xanadu plays fine from there as expected (this turns out to be the only other folder from which Xanadu plays)

Copy Xanadu from Google Drive to Play Files/Music folder; Xanadu doesn't play, but displays properly in Edit Mode; console Message = "setCsoundText..." and stops; important clue: filename is displayed in top bar of Csound as "Csound:audio:xxx (a number)"

Copy Xanadu back to Google Drive (after deleting previous Xanadu); Xanadu plays fine as expected - as usual with its proper name within Csound.

I suspect the .csd extension is leading Chromebook Plus astray somehow. It is being interpreted as an audio file.

A confounding factor: aside from Google Drive, the Music folder is the only place Csound can seemingly read from, and especially is the only place its Editor can save to.

And thoughts? (Chromebook Plus is the future of Chromebooks.)

gogins commented 10 months ago

https://developer.android.com/topic/arc/manifest https://chromeos.dev/en/android https://www.xda-developers.com/chrome-os-settings-guide/ https://developer.chrome.com/docs/extensions/reference/fileSystemProvider/

The above are useless. Maybe this is better:

https://chromeos.dev/en/android-environment

gogins commented 10 months ago

Did Hunkins get his Chromebook into Developer Mode?

https://chromeos.dev/en/android-environment/developer-options

What I gather so far is that this problem arises from Google pushing almost all storage into the cloud (e.g. Google Drive). It may be that the developer options and/or the Android settings can be tweaked to provide access to the file system.

gogins commented 10 months ago

https://chromeos.dev/en/posts/chromeos-lint-rules-in-android-studio

gogins commented 10 months ago

Describes "side-loading:" https://www.xda-developers.com/how-sideload-apps-chromebook/.

Does Art Hunkins do side loading of the Csound for Android app? If so, does he use developer mode, or does he use ADB?

If with ADB, then in the terminal window, one can use adb to query for permissions and to grant permissions, e.g.

grant package_name permission | Grant a permission to an app. On devices running Android 6.0 (API level 23) and higher, the permission can be any permission declared in the app manifest. On devices running Android 5.1 (API level 22) and lower, must be an optional permission defined by the app.

Possibly, the Chromebook is overriding the app manifest's grant of all files access; it may be possible to restore all files access in this way.

gogins commented 10 months ago

I tried saving Music/xanadu.csd as Music/xanadu.csound; I could open it and save it from the app, but it wouldn't play.

gogins commented 10 months ago

Art: Focus on loading and saving from and to the Music folder.

gogins commented 10 months ago

https://stackoverflow.com/questions/62782648/android-11-scoped-storage-permissions https://medium.com/@kezzieleo/manage-external-storage-permission-android-studio-java-9c3554cf79a7

gogins commented 10 months ago

From https://developer.android.com/training/data-storage/manage-all-files:

When an app has the MANAGE_EXTERNAL_STORAGE permission, it can access these additional files and directories using either the MediaStore API or direct file paths. When you use the Storage Access Framework, however, you can only access a file or directory if you can do so without having the MANAGE_EXTERNAL_STORAGE permission.

If this is the case with the Csound for Android app, that might well explain Art's problems.

The implication is that if I use the MediaStore API, access with MANAGE_EXTERNAL_STORAGE permission works for all MediaStore files that would work without MANAGE_EXTERNAL_STORAGE permission.

A number of questions arise:

[ ] Is this the cause of Art having his app complain about .csd files being media files?
[ ] If not can I open and read or write a regular file with MediaStore?

This appears to be an attempt to deal with this situation: https://github.com/SNNafi/media_store_plus.

gogins commented 10 months ago

I have two physical devices:

[ ] Smartphone running Android 13 (samsung SM-G998U).
[ ] Tablet running Android 10 (samsung SM-T830).

This should help me clear this up.

gogins commented 10 months ago

Whoa! Maybe this will work for all required access within a subset of directories:

https://developer.android.com/reference/android/provider/MediaStore.Files

I'll hold off on trying this as it would complicate my life....

gogins commented 10 months ago

Upgraded Android Studio.
Upgraded MacBook and all brew apps.
Modified manifest permissions. CsoundAndroid and CsoundApplication now have identical permissions.
File access permissions are up to latest possible SDK level.
I can run examples, save them to device storage, quit the app, restart the app, load the saved examples from the app, and run them on both Android 10 (tablet) and Android 13 (phone).

I just don't get Art's problems unless he hasn't actually granted permissions.

gogins commented 10 months ago

Notes for my zoom with Art....

Agenda:

[x] Have him start the app, I will review messages. The filesystem reported by the app on Chromebook is the filesystem of the emulated Android OS. These are visible on the Chromebook with the older app, but not with the newer app.
[x] Have him run a piece, I will review messages. Nothing new, no help.
[x] Have him show me ChromeOS permissions for the app. Doing that now... I can't read the Chromebook screen if he holds it up to his computer camera, we'll try taking a photo and emailing it to me. He did that, but it's the Android permissions. I think he also accessed the permissions from the Chromebook side, but they were the same.
[ ] If that's no help, have him uninstall and reinstall the app, I will review every step.
[x] It seems that loading shows only the Android filesystem, and saving shows only the Chromebook filesystem. EACCESS for saving the Csound text.
[ ] Save as... tries to save to the Chromebook filesystem. There's an option to save to My Files but that includes only Downloads that one can't read from.

Possibly, I ignore the external storage directory in the app and start from as high up in the filesystem directory as I can.

Takeaways:

It seems that the "all files" permission granted during installation is not observed at run time.
It seems that the app can only load from the Android filesystem, and only save to the Downloads folder or to Google Drive.

gogins commented 10 months ago

This is relevant: https://chromestory.com/2018/06/how-to-access-android-app-files-on-a-chromebook/

gogins commented 10 months ago

Got the Chromebook, set it up for myself, following instructions here.

gogins commented 10 months ago

It looks like there might be another way to do this, and that is to set up the Chromebook in Chrome developer mode, which allows direct setup of .apk files not from the Play Store. But for now I am continuing with the Linux-based side loading.

The request for permission to access all files did appear when I started the Csound for Androd app, as it does when installing on a phone, so that's not a cause of this problem.

I ran Csound Koan I, and saved it to Downloads as i.csd.

I quit the app, started it again, and loaded i.csd. But it would not run.

I then saved i.csd to Music, and opened it from Play files, Music. It would not run.

gogins commented 10 months ago

I have found a workaround (not a solution, a workaround).

After you save or create a new piece, use the Files app on the Chromebook to copy that file to the Linux files directory. You can then open that file from the Csound for Android app, and it will play.

It's clear that, in spite of enabling "All files" permissions in ChromeOS, that permission is not actually granted, and the app is continuing to use the MediaStore system for reading and writing files, which is what is causing these problems. I suppose the MediaStore system knows nothing about the Linux files directory, and that's why this workaround works.

I will see if the permission can be granted using adb, or if there's some change I can make in the Csound for Android app's code to enable access to Linux files.

gogins commented 10 months ago

In the CsoundApplication manifest I changed

    <uses-permission android:name="android.permission.MANAGE_EXTERNAL_STORAGE" />

to

    <uses-permission android:name="android.permission.MANAGE_EXTERNAL_STORAGE" tools:ignore="ScopedStorage" />

However, this did not help.

gogins commented 10 months ago

When I open a Linux terminal and run the adb shell, I can't find android.permission.MANAGE_EXTERNAL_STORAGE. That's interesting!

gogins commented 10 months ago

I issued the command in the adb shell: pm grant com.csounds.Csound6.CsoundAppActivity android.permission.MANAGE_EXTERNAL_STORAGE and it appeared to complete without error.

However the permission is still restricted.

gogins commented 10 months ago

Here at last an answer (in the adb shell):

pm grant com.csounds.Csound6 android.permission.MANAGE_EXTERNAL_STORAGE                                                                                     

Exception occurred while executing 'grant':
java.lang.SecurityException: Permission android.permission.MANAGE_EXTERNAL_STORAGE requested by com.csounds.Csound6 is not a changeable permission type
        at com.android.server.pm.permission.BasePermission.enforceDeclaredUsedAndRuntimeOrDevelopment(BasePermission.java:453)
        at com.android.server.pm.permission.PermissionManagerService.grantRuntimePermissionInternal(PermissionManagerService.java:1458)
        at com.android.server.pm.permission.PermissionManagerService.grantRuntimePermission(PermissionManagerService.java:1411)
        at com.android.server.pm.PackageManagerShellCommand.runGrantRevokePermission(PackageManagerShellCommand.java:2300)
        at com.android.server.pm.PackageManagerShellCommand.onCommand(PackageManagerShellCommand.java:251)
        at android.os.BasicShellCommandHandler.exec(BasicShellCommandHandler.java:98)
        at android.os.ShellCommand.exec(ShellCommand.java:44)
        at com.android.server.pm.PackageManagerService.onShellCommand(PackageManagerService.java:21831)
        at android.os.Binder.shellCommand(Binder.java:929)
        at android.os.Binder.onTransact(Binder.java:813)
        at android.content.pm.IPackageManager$Stub.onTransact(IPackageManager.java:4668)
        at com.android.server.pm.PackageManagerService.onTransact(PackageManagerService.java:4325)
        at android.os.Binder.execTransactInternal(Binder.java:1159)
        at android.os.Binder.execTransact(Binder.java:1123)
255|nautilus_cheets:/ $

Art Hunkins tried developer mode for the device, I tried the Linux development environment on the device, and the results appear to be the same.

So, there seems to be a difference between Android running on a Chromebook, and Android running a whole device, with respect to this permission.

Right now, there is no way to do what Art wants except the workaround I discovered: Save a new piece to My Files, use the Files app to copy the piece to Linux files, open the piece from Linux files, and it will run.

gogins commented 10 months ago

I will try chasing this a bit further.

[ ] Inspect my code for saving and loading files, to see if I can enable saving to Linux files.
[ ] Think about using MediaStorage for all access, as recommended by Google. Based on what I'm seeing here, I am not at all sure how to do this. But it might be possible.

gogins commented 10 months ago

It turns out that any pieces that are copied to, saved into, and loaded from the My Files, Play files, Documents directory will play. Not so, they open as Audio:nnn.

I believe that this is caused by the Csound for Android app using the MediaStore system to load and save files. The MediaStore system allocates different types of files to different directories. Not so.

If this holds up, then I will try removing the All Files permission from the app, and re-installing it on various devices. If it's still possible to load and save pieces from the Documents directory, I can re-submit the app to the Google Play store. Not yet!

However: As Art said, Save as... to one of the accessible directories will result in a playable piece in the app.

I'll have to read the code again....

gogins commented 10 months ago

To do:

[ ] Change where the examples in the app are copied to the Documents directory.
[ ] Test that soundfiles can be written to the Documents directory; if not, update the app's startup messages to reflect which directories are for what.
[ ] Test that things still work on Android 10.
[ ] Update documentation.
[ ] Re-submit to Play Store.

gogins commented 10 months ago

https://medium.com/@wanxiao1994/android-storage-permission-adaptation-and-reading-writing-media-files-6b85121c9461 https://developer.android.com/training/data-storage/shared/media This is the Bible here: https://developer.android.com/about/versions/11/privacy/storage and https://developer.android.com/training/data-storage/use-cases

gogins commented 10 months ago

I think that Google has all apps from Android 11 on actually contact Google to find out if an app has MANAGE_EXTERNAL_STORAGE permission (or maybe there's a magic number in the .apk or something). That means I can't rely on granting this permission to work. Right now, doing so works on my phone and tablet, but not on my Chromebook.

My app uses the Storage Access Framework but, for current Android, this can't provide enough scope for storage.

This is key:

When an app has the MANAGE_EXTERNAL_STORAGE permission, it can access these additional files and directories using either the MediaStore API or direct file paths. When you use the Storage Access Framework, however, you can only access a file or directory if you can do so without having the MANAGE_EXTERNAL_STORAGE permission.

gogins commented 10 months ago

This is an unbelievable mess.

I'm throwing in the towel. I will re-target the maximum SDK level to 29, which is the highest level that still supports our desired behavior. That means the app can't get into the Play Store, but it should work on Chromebooks, phones, and tablets. I'm trying this now. But this move will not work from Android 14 on.

If there's a way forward after Android 14, either Google will introduce new policies/APIs, or I will have to try to make MediaStore.Files work for all needs.

Guess what! Android Studio won't let me set the maximum SDK level to 29.

gogins commented 10 months ago

Back to square one. I have installed Linux and enabled adb debugging on the Chromebook, and created an adb connection to my MacBook. Android Studio can now debug an app on the Chromebook. And look what I get for the file URI:

content://com.android.providers.media.documents/document/audio%3A371

I wonder if I get the filepath from this URI, if I can then go ahead and open the file as a .csd. No surprise, doesn't look that easy. But, what's the URI on my phone? Well, a similar URI, a .csd file in Documents anyway:

content://com.android.externalstorage.documents/document/primary%3Ai.saved.csd.

The first has a provider android.providers.media.documents and the second has android.externalstorage.documents. And I think the first form is caused by MANAGE_EXTERNAL_STORAGE not being enable-able on the Chromebook.

So, Art's workaround works because Save as... creates a file URI. I don't think that there is a way to do this programmatically that would not already involve getting the filepath from the content URI, and I suspect that's not possible.

gogins commented 10 months ago

It's possible to root the Android system on a Chromebook perhaps, see https://www.xda-developers.com/root-android-subsystem-chromebooks/.

gogins commented 10 months ago

I've put my Chromebook into developer mode, and installed the Linux shell. To debug an Android app running on the Chromebook from my MacBook:

Find IP address of Chromebook.
In the MacBook terminal, enter:

michaelgogins@Michaels-MacBook-Pro ~/csound-android % ~/Library/Android/sdk/platform-tools/adb connect 192.168.201.101

The Chromebook should then appear as a debuggable device in Android Studio on the MacBook.

gogins commented 10 months ago

I will try MediaStore.getDocumentUri on all of my devices and compare results and permissions.

[x] On the smartphone with Android 13, only the content://com.android.externalstorage.documents/document/primary%3Acsd%2Fsaved.i.csd Uri yields anything usable as a filepath.
[x] On the tablet with Android 10, same as the smartphone with Android 13. The MediaStore Uri is something like /external_primary/file/20967.
[x] On the Chromebook with Android 11, none of the three Uris looks anything like a filepath; however, the original Uri from the picker can be read to produce valid text. This implies the problem lies in rendering that text.

gogins commented 10 months ago

When the piece is an HTML file, then without a real filepath, it doesn't seem possible to derive a base URL.

For a .csd piece, it's a bit mysterious now for me how this works.

gogins commented 10 months ago

Possibly, I could refactor the code to render pieces only in internal storage, but then how would a user get a rendered soundfile out of the device? Or get samples into the device?

gogins commented 10 months ago

So, let's see what happens to the Uri when I do the Save as... workaround on the Chromebook.

gogins commented 10 months ago

I think this means the following things:

If I can't find a way to enable MANAGE_EXTERNAL_STORAGE on the Chromebook, we may be stuck with Art's workaround.
Getting this to work on the Chromebook without the MANAGE_EXTERNAL_STORAGE permission by refactoring the code may be impossible, and if not impossible, would require a lot of new work.
Google is gradually tightening the screws in an obvious effort to (charitably) protect the security of the computer illiterate and (uncharitably) have all Android content pass through their cloud. I think it rather likely this will mean the end of Android devices as general purpose computers.

gogins commented 9 months ago

On the Chromebook, the URI for rendering a .csd file is e.g.: /document/audio:228. The URI for rendering an .html file is e.g.: /document/document:332. So, there is a way to tell if a piece is a .csd piece (/document/audio:) or an .html piece (/document/document).

However, the .html pieces run into this problem:

2023-11-12 18:37:19.916   767-1290  MediaProvider           com.android.providers.media.module   W  Using FUSE for /storage/emulated/0/Documents/Gogins/minimal.html
2023-11-12 18:37:19.992  4971-4971  Csound:                 com.csounds.Csound6                  D  baseUrlString: file:/storage/emulated/0/

This causes the base URL for the .html page to be incorrect, so required resources are not loaded and there is an exception in running the piece. In the above, the base URL really should be something like /storage/emulated/0/Documents/Gogins.

gogins commented 9 months ago

I have pushed a partial solution for Chromebooks, Csound will now run off a content URI. Still not clear if .html pieces can be gotten to work.

gogins commented 9 months ago

What does the base URL look like on my smartphone? For Scrims:

baseUrlString: file:/storage/emulated/0/examples/Gogins/

gogins commented 9 months ago

This might be relevant: https://android-developers.googleblog.com/2023/11/increasing-trust-for-embedded-media.html.

gogins commented 9 months ago

I don't currently see a solution to the base URL problem. The whole point of the content URI idea is to make a flat table of resources, and the whole point of a Web server/Web browser system is to root a hierarchical tree of resources at the base URL.

The Save as... workaround does not work for .html pieces, for the same reason.

gogins commented 9 months ago

Possible solution to .html pieces on Chromebooks: Add a Setting field for the base URL.

gogins commented 9 months ago

Android 13 on phone loading example:

2023-11-14 09:24:59.292 31888-31888 System.err              com.csounds.Csound6                  W      at com.csounds.Csound6.CsoundAppActivity.getFilePathFromContentUri(CsoundAppActivity.java:645)
2023-11-14 09:25:05.639 31888-31888 Csound                  com.csounds.Csound6                  D  getFilePathFromContentUri: filepath: 
2023-11-14 09:25:54.285 31888-31888 Csound                  com.csounds.Csound6                  D  uriToFilepath: filepath: /root/storage/emulated/0/Music/examples/Gogins/scrims.html
2023-11-14 09:26:14.022 31888-31888 Csound                  com.csounds.Csound6                  D  baseUrlString from filepath: file:/storage/emulated/0/Music/examples/Gogins

Loading from storage:

2023-11-14 09:30:06.620 31888-31888 Csound                  com.csounds.Csound6                  D  getFilePathFromContentUri: filepath: primary:Music/Csound6AndroidExamples/Gogins/Message.html
2023-11-14 09:30:06.621 31888-31888 Csound                  com.csounds.Csound6                  D  uriToFilepath: filepath: /storage/emulated/0/Music/Csound6AndroidExamples/Gogins/Message.html
2023-11-14 09:30:06.622 31888-31888 Csound                  com.csounds.Csound6                  D  baseUrlString from filepath: file:/storage/emulated/0/Music/Csound6AndroidExamples/Gogins/

Android 10 on tablet loading example:

2023-11-14 09:39:32.247 25682-25682 System.err              com.csounds.Csound6                  W      at com.csounds.Csound6.CsoundAppActivity.getFilePathFromContentUri(CsoundAppActivity.java:645)
2023-11-14 09:39:32.278 25682-25682 Csound                  com.csounds.Csound6                  D  getFilePathFromContentUri: filepath: 
2023-11-14 09:39:32.278 25682-25682 Csound                  com.csounds.Csound6                  D  uriToFilepath: filepath: /root/storage/emulated/0/Music/examples/Gogins/message.html
2023-11-14 09:39:32.289 25682-25682 Csound                  com.csounds.Csound6                  D  baseUrlString from filepath: file:/storage/emulated/0/Music/examples/Gogins

Loading from from storage:

2023-11-14 09:40:46.836 25682-25682 Csound                  com.csounds.Csound6                  D  getFilePathFromContentUri: filepath: primary:Music/examples/Gogins/message.html
2023-11-14 09:40:46.838 25682-25682 Csound                  com.csounds.Csound6                  D  uriToFilepath: filepath: /storage/emulated/0/Music/examples/Gogins/message.html
2023-11-14 09:40:46.841 25682-25682 Csound                  com.csounds.Csound6                  D  baseUrlString from filepath: file:/storage/emulated/0/Music/examples/Gogins/

Android 11 on Chromebook loading example:

2023-11-14 09:44:52.009 11857-11857 System.err              com.csounds.Csound6                  W      at com.csounds.Csound6.CsoundAppActivity.getFilePathFromContentUri(CsoundAppActivity.java:645)
2023-11-14 09:44:52.061 11857-11857 Csound                  com.csounds.Csound6                  D  getFilePathFromContentUri: filepath: 
2023-11-14 09:44:52.062 11857-11857 Csound                  com.csounds.Csound6                  D  uriToFilepath: filepath: /root/storage/emulated/0/Music/examples/Gogins/message.html
2023-11-14 09:44:52.088 11857-11857 Csound                  com.csounds.Csound6                  D  baseUrlString from filepath: file:/storage/emulated/0/Music/examples/Gogins

Loading from storage:

2023-11-14 09:46:41.390 11857-11857 Csound                  com.csounds.Csound6                  D  getFilePathFromContentUri: filepath: document:135
2023-11-14 09:46:41.393 11857-11857 Csound                  com.csounds.Csound6                  D  uriToFilepath: filepath: /storage/emulated/0/135
2023-11-14 09:46:41.397 11857-11857 Csound                  com.csounds.Csound6                  D  baseUrlString from filepath: file:/storage/emulated/0/

gogins commented 9 months ago

On the Chromebook, running the HTML examples works. But running the same example that is loaded from storage does not work (though it does on the other devices). This appears to be caused by a truncated base URL.

I'm stepping through the loadWebView function and examining all URIs, URLs, and filepaths to see if something is hiding in there that I did not notice before. No help.

I'm stepping through the code that loads the sample, my assumption is that it does the equivalent of Save as... First, the piece is copied from the assets directory using a complete filepath built by adding the asset filepath to the external storage directory path. Then, the app loads the piece as a text file in the same way as for all other pieces. After that, the app creates the URI for the piece thusly:

       csound_uri = FileProvider.getUriForFile(getApplicationContext(), BuildConfig.APPLICATION_ID, file);

Here, csound_uri has the complete and correct filepath /root/storage/emulated/0/Music/examples/Gogins/scrims.html.

gogins commented 9 months ago

Here we go again: https://developer.android.com/reference/androidx/core/content/FileProvider.

As far as I know, I have done these things. But they're complex enough that I might have made a mistake.

gogins commented 9 months ago

I have not been able to figure out how to get the base URL from a content URI other than the one available after copying an asset into the examples directory.

Google's content URI idea is fine for loading and saving files, copying files, and sharing files, but it is not designed for files that need to know about other files or filepaths, which a Web browser does need to know.

Review of the code, stepping through the code, review of Android documentation, and googling forums has not told me how to do this.

One of these things might work and would be acceptable:

[x] Copy all the assets into the external storage root directory. Then the code for deriving the base URL should work, or can be made to work. I'll try to copy the contents of the Gogins examples into the root. No help.
[x] Provide a Settings field where the user may enter the base URL directly. This won't be necessary on devices that actually permit MANAGE_EXTERNAL_STORAGE. I hard coded the correct base URL to the external storage directory, where all required files existed, but it did not help.
[x] It would be enough to simply create a single directory that would present filenames and a workable base URL. Not likely, works the same as first two options.
[x] It would be enough to stick in a USB stick or micro SD card. _I can mount a USB drive, copy files onto it using the Files app on the Chromebook, and load a piece, but an HTML piece has the same problems as the first three options.

gogins commented 9 months ago

Bottom line: without MANAGE_EXTERNAL_FILES permission on Android 11 or later, I seem to absolutely need one of the following:

[ ] Find some way of getting the complete filepath from the content URI. I'm not sure this would work for HTML pieces.
[ ] Better would be to find some way of enabling the permission after all.

gogins commented 9 months ago

https://copyprogramming.com/howto/uri-to-filepath-get-file-path-from-uri

gogins commented 9 months ago

I tried denying the All Files permission on my chromebook, then granting it again. No help.

gogins commented 9 months ago

I did get the "display id" out of the ContextResolver, which is the filename, i.e. the title of the piece.

gogins commented 9 months ago

I think there is only one possible solution here, and that is to set up a special directory in a known location, then put all files in there in a flat directory structure, and simply stick the display id onto the path of that special directory to get the filepath.

gogins / csound-android

Fix file access problems on Chromebooks #29