Android-for-Python / Android-for-Python-Users

An unofficial Buildozer users' guide
135 stars 17 forks source link

File-path, POSIX file processing _is_ still possible on Android #4

Closed quixotely closed 1 year ago

quixotely commented 1 year ago

Not sure if this doc is still maintained, and I commend its authors for providing some sorely needed Kivy/buildozer info. But its coverage of Android storage seems misleading, and arguably incorrect. The sections on shared storage and Kivy's file explorer in particular state that Android does not support file-path, POSIX access to storage; this is untrue.

I suggest that the authors explore Android's All Files Access permission (a.k.a. MANAGE_EXTERNAL_STORAGE). This was added in Android 11 to reinstate the file-path and POSIX file access that was more readily available earlier. With this permission, Python-coded apps can still use Python's portable file tools to process files on Android in shared storage, as well as app private (both "internal" and "external") storage. For a first look, try the dev page.

As an example, I recently published a backup/sync app that's coded in Python and Kivy; uses Python's built-in file tools to process content; and runs Kivy's file explorer to navigate shared and app storage by path names--on all Androids 8 and later, including 13. It obtains All Files Access on 11 and later, and legacy storage permission earlier.

You're invited to fetch this app's trial version to see this in action, at the Play Store. This app's PC versions use the same portable Python and Kivy code, and can be had at its main website.

It's true that All File Access requires Play-store approval, but it's been widely granted for content-processing tools, and this doesn't apply to side-loaded tools. It's also true that accessing shared storage by file paths and POSIX calls incurs some overhead, but this stems from Android's FUSE implementation and its added security checks, and the penalty is a minor price for being able to use existing POSIX code.

More to the point, Python file-path/POSIX processing can be used on Android. This is a huge win for people seeking to use portable Python code on Android, and seems better called out as an option than swept under the rug. With permissions, the only storages truly off limits to Python/Kivy app developers today are other apps' private ("internal") storage, and shared storage's Android/data folder (though some file explorers manage to open this as well).

RobertFlatt commented 1 year ago

Thanks for your feedback, if you'd like to write a subsection on how and when to use MANAGE_EXTERNAL_STORAGE that would be great.

The current omission is deliberate, and that is OK because this is a Users' Guide and not a Reference Manual.

Its not clear that we could create supportable documentation. Sometimes is best not to document the features of other peoples software, at some point you are guaranteed to fail.

The major reasons for omitting this are (perhaps these are things you could address):

quixotely commented 1 year ago

Thanks for your reply; I'll get back to you with some ideas asap (I'm busy pushing out an update to my app that works around a new batch of Kivy glitches).

-----Original Message----- From: RobertF @.> To: Android-for-Python/Android-for-Python-Users @.> Subject: Re: [Android-for-Python/Android-for-Python-Users] File-path, POSIX file processing is still possible on Android (Issue #4) Date: Tue, 06 Jun 2023 17:55:05 -0700

Thanks for your feedback, if you'd like to write a subsection on how and when to use MANAGE_EXTERNAL_STORAGE that would be great.

The current omission is deliberate, and that is OK because this is a Users' Guide and not a Reference Manual.

Its not clear that we could create supportable documentation. Sometimes is best not to document the features of other peoples software, at some point you are guaranteed to fail.

The major reasons for omitting this are (perhaps these are things you could address):

  • If we tell users how to use it we also have to tell users when they can use it. Its not clear how to prevent surprises at the Store https://support. google.com/googleplay/android-developer/answer/10467955?hl=en. Surprises would mean unreliable documentation. Too many users do not have the context to know what "Core functionality" means. This document has to be written for the lowest common denominator Python script kiddie, and MANAGE_EXTERNAL_STORAGE breaks the security model. I think 'script kiddie' and 'security model' do not belong in the same sentence. �

  • Interaction with the MediaStore is unclear to me. Can we say the two apis will have consistent state (I'm looking at you MediaStore)? If not there will be unexpected behavior. Can we document this? If we don't we get the why... questions that we don't know the answer to. We don't want to create support issues.

-- Reply to this email directly or view it on GitHub: https://github.com/Android-for-Python/Android-for-Python-Users/issues/4#issuec omment-1579674374 You are receiving this because you authored the thread.

Message ID: @.*** ithub.com>

quixotely commented 1 year ago

This appears to have been closed without any changes. If this page is a personal-opinion blog, then you're of course welcome to say whatever you like, and best wishes and never mind.

But if it's meant to be an open-source doc for the Kivy and Buildozer projects, then its storage coverage is misleading enough as is to discourage Python people from giving Android, Kivy, and Buildozer a try, and that doesn't seem the goal. If so, I suggest an insert like the following after the existing coverage of storage rules (which I'm not adding myself, because the scope and ownership of this doc is unclear):

=snip=

Sidebar: The All Files Access Exception

All that being said, you can use Python's POSIX file tools, Kivy's FileChooser GUI, and traditional file paths to process content in shared storage and removable drives on all Androids. You simply need to obtain the required permissions first:

With these permissions, Python's open() call, file objects, and os-module file tools will all happily read, write, and process shared storage, app storage, and USB-drive storage on Android using normal file paths. This is ultimately facilitated by the Android native libraries underlying Python on this platform. As a side effect, Kivy's Python-coded file chooser works to browse and select usable file paths as well.

For proof of concept, see the PC-Phone USB Sync backup/sync app on the Play store. This app processes content using portable Python code that runs the same on both Android and all PCs. This is huge interoperability win for both existing libraries, and programs intended to be used across a platform mix that includes Android.

Tradeoffs: the Play store won't accept apps using MANAGE_EXTERNAL_STORAGE unless they are in a category that qualifies, but has allowed this permission broadly for most apps that processes content, including file managers, backup/sync tools, and media and text editors. Naturally, this permission can also be freely used outside Play, for side-loaded apps made available on other sites (e.g., F-Droid) or built for in-house or personal use.

It's also worth noting that shared storage runs slower than app storage in Android 11 and later today when using portable file paths, but this is an implementation defect with open bug reports. Moreover, shared storage is fast enough for a variety of use cases and may grow faster with UFS-4.0 chips, and is simply required for cross-app content creation and usage tasks—of the sort required to build apps in the first place.

=snip=

Android-for-Python commented 1 year ago

Opinion noted and ignored.

quixotely commented 1 year ago

You're welcome? People who can't handle feedback probably shouldn't ask for it.