[Bug]: Cannot read or execute files under /sdcard without root privilege

[erhenjt]

Problem description

After executingtermux-setup-storageand granting storage permissions, Termux is unable to read, execute or list files under /sdcard or any subfolders of /sdcard (using ls). (However it was able to list the file when using ls /sdcard/filename.) Granting root privilege with su or sudo circumvents this, but it will require users to root their phones which is not possible with some modules.

Steps to reproduce the behavior.

1. create a file1under internal storage or any subfolder of it such asDownloadfolder
2. in Termux, runtermux-setup-storageand grant storage permissions
3. in Termux, run /sdcard/1,/sdcard/Download/1,/storage/emulated/0/1,/storage/emulated/0/Download/1,/storage/self/primary/1,/storage/self/primary/Download/1 returns "Permission denied"
4. addingnanobefore above commands prompt "Permission denied" in nano editor
5. Addinglsbefore above commands was able to list the specific files
6. Removing/1in above commands lists no files but only folders under that that directory

What is the expected behavior?

Termux should be able to read, write and execute files in internal storage and its subfolders

System information

• Termux application version: 0.127
• Android OS version: Android 13.0 stock rom
• Device model: Over multiple devices

[fornwall]

@erhenjt Thanks for reporting and providing clear information!

On newer Android versions this requires the MANAGE_EXTERNAL_STORAGE permission - see https://developer.android.com/training/data-storage/manage-all-files.

Unfortunately, when submitted to Google Play with that permission, it was not accepted, with the motivation that it did not fulfil the linked requirement above:

Your app's usage of the permission must fall within permitted uses and must be directly tied to the core functionality of the app

I'm trying to convince the Google play reviewers to grant this, but so far it looks unlikely that they will budge. If so, the second best thing is probably to provide built-in utility scripts to fetch and insert files to shared storage using the Storage Access Framework.

[erhenjt]

@fornwall Thank you for clarifying.

Unfortunately, when submitted to Google Play with that permission, it was not accepted, with the motivation that it did not fulfil the linked requirement above

There are many file management and archive management apps (such as Total Commander, X-plore File Manager, ZArchiver and RAR) that actually claimedMANAGE_EXTERNAL_STORAGEpermission and were able to get on Play Store. All of them have target SDK 34. It will be interesting to know how they convinced Google Player viewers and whether Termux can use the same reason or a similar reason to do the same, since Termux also provides features like managing and modifying files and archiving and extracting archives.

By the way, do you mind explaining why Termux can list the directories under /sdcard but not the files, and when directly ls the file name, it is able to list that specific file? Thanks.

[erhenjt]

Actually just found that there are many IED apps like Pydroid 3, Jvdroid and Cxxdroid also have theMANAGE_EXTERNAL_STORAGEpermission, and the UserLAnd app, a Linux compatibility layer haveREAD_EXTERNAL_STORAGEandWRITE_EXTERNAL_STORAGEpermissions, but not MANAGE_EXTERNAL_STORAGE. All of those 4 apps have target SDK 33.

Maybe it's worth asking these developers how did they manage to pass the Google Play review?

[fornwall]

Actually just found that there are many IED apps like Pydroid 3, Jvdroid and Cxxdroid also have the MANAGE_EXTERNAL_STORAGE permission

Thanks for looking that up! Those indeed seems like apps quite similar to Termux, so can be used as "if they can get it, so should we" argument - even though the Google Play review process is quite non-deterministic/varies over time and between apps, unfortunately.

Have now created a 0.128 build with the MANAGE_EXTERNAL_STORAGE permission added back and submitted it for review (and mentioned your findings above, about similar apps), and we'll see what happens - will post an update here as soon as I hear anything!

[fornwall]

Just an update that the build with MANAGE_EXTERNAL_STORAGE added is still in Google Play review, haven't heard anything yet.

[fornwall]

It passed Google Play review, and the current version 0.129 of Termux on Google Play should now give read/write access to /sdcard using MANAGE_EXTERNAL_STORAGE 🥳. Thanks for the help here @erhenjt !

Note that files stored on /sdcard cannot have the execution permission.

Leaving this issue open to monitor and follow up issues or questions for a while.

[erhenjt]

Congratulations on passing Google Play review!

An idea for executing files in /sdcard: Since Termux can execute on its own data directory, how about copying file that user wants to execute to somewhere in Termux's data directory (for example usr/tmp)and execute that file instead?

[erhenjt]

@fornwall Just tested the commands cp /sdcard/1 /data/data/com.termux/files/usr/tmp/ && chmod a+x /data/data/com.termux/files/usr/tmp/1 && /data/data/com.termux/files/usr/tmp/1 with a 100MB binary file 1. On an old phone, the command finished almost instantly, so it's certainly feasible since in reality most executables will be much smaller.

So a possible workaround will be:

1. During launching, clear any files under /data/data/com.termux/files/usr/tmp/
2. When user tries to execute a file under /sdcard/folder1/folder2/executable, Termux checks whether it already exists in /data/data/com.termux/files/usr/tmp/folder1/folder2/, if exists, execute the file with working directory being user's current working directory, ignoring step 3,4
3. If the file is not in /data/data/com.termux/files/usr/tmp/folder1/folder2/, copy that specific file with its directory relative to /sdcard to /data/data/com.termux/files/usr/tmp/
4. Add executable permission to the file for all users
5. Execute the file under /data/data/com.termux/files/usr/tmp/ with working directory being user's current working directory
6. Monitor changes to the original file under /sdcard, when it is changed or removed, remove the corresponding file under /data/data/com.termux/files/usr/tmp/ as well (Keep something like a "last changed time" file for each original file?)
7. When exiting Termux, remove all files under /data/data/com.termux/files/usr/tmp/

Is this possible to do? And are there any downsides to this solution?

[fornwall]

@erhenjt Something like copying away the file for execution as you are doing should work, and you could check set up some shell script tool to automate that for you, perhaps

• `execute-sdcard # copy to temp dir if file doesn't already exist there, chmod +x it, then execute it
• cleanup-sdcard-executables # Cleanup stored temp dir with copied executables

Should perhaps be good enough? Perhaps it's hard to make some generic mechanism as people would like to tweak the exact behaviour, but some custom shell script might be good enough.

Out of interest if you want to share, which executables do you have on /sdcard that works on Android?

[erhenjt]

Personally I don't have a need to run executable in Termux (for now), but I think the ability to run executable files is one of the most basic features for a Linux terminal environment and it will enable users to run varies softwares that are only in binary format and not being packed into deb/rpm on their devices.(such as many GitHub projects, and especially since more projects are adding support for aarch64) And if user replaced the binaries, it can be runned without having to manually copy-paste/setup permission/change directory. So I think it will be better that theexecute-sdcardcommand is one-time, and after setup, Termux will automatically do the things like copying the file to tmp directory, changing permission and tracking whether the file was modified, and user can revok those and restore to default with a parameter (maybe likeexecute-sdcard -restore?)

[erhenjt]

And if you were asking about the testing file, it was a static curl, and I appended a 100M file to it just to test whether it would be feasible and fast enough to copy the file to Termux data directory with larger files.