Inaccessible userdata after using chroot-distro

[nickelberg]

I can't be the only one, so it should be pretty self explanatory. Happened on two my different phones - common features: Xiaomi, rooted, Android 14. After exiting shell from distro's chroot and then exiting Termux, phone apps don't have access to phone storage.

Examples:

• Gallery not able to display photos
• Files (app) not able to display internal memory files
• Material Files, here I got more informative error message: "java8.nio.file.FileSystemException: /storage/emulated/0: opendir: Transport endpoint is not connected"

Edit: Important to add - a simple reboot fixes this, but it's cumbersome.

[jjkola]

Thanks for the report and the extra information. I did notice your bug report a few days ago. Unfortunately, I have been really busy, so I haven't been able to react to your report.

Just using chroot should not cause this kind of behavior. Also, as chroot-distro uses bind mounts, it should not cause the described behavior.

So, I would like to have more information about the situation so that I can try to reproduce the issue.

First, what distribution did you use? Be as exact as possible.

Second, while using the distribution, what did you do? If you can provide clear steps to reproduce the issue, then all the better.

Third, after exiting the jail, did you just close the Termux, or did you issue some commands?

Also, if you could provide the version of the chroot-distro, that would be helpful.

[nickelberg]

1. Debian 12 and Alpine 3.19
2. Doesn't matter. I can install whole desktop enviroment and vnc server or just do sudo apt update and then quit. It will happen anyway.
3. Steps: 3.1 chroot-distro:$ exit 3.2 Termux#: chroot-distro unmount debian 3.3 Termux#: exit 3.4 Termux$: exit 3.5 *app closes*
4. It has happened every time I used chroot-distro. Originally in March 16th version and even in the most recent version v1.1.3.

And most importantly I forgot to add: I'm not using Magisk, I have KernelSU on both of my phones. There's a high possibility, this is the bug culprit.

[jjkola]

Unfortunately I don't have any devices with KernelSU so I can't test this. @YasserNull, do you have any devices with KernelSU?

[YasserNull]

No, I don't have any devices with KernelSU.

[roninspin]

chroot-distro unmount <distro> (ubuntu in my case) really unmounts /storage/* for android instead of chroot distro. Maybe chroot_distro_unmount_system_points receives empty distro_path somehow. Why do we touch /storage after all? Isn't /sdcard sufficient? Btw. I have KernelSU and I use chroot-distro as a standalone script (without installed module).

[defectly]

it's not kernelSU problem i guess, i've been using tmoe with magisk for chroots, and there's same problem with unmount

[jjkola]

I have checked, and distro_path should not be empty. @nickelberg, @roninspin and @defectly, can you run command -v umount and then with the path run ls -l <path returned by command>. I want to check what is providing the umount command. These needs to be run as a root, and I want the output of both commands.

[defectly]

I have checked, and distro_path should not be empty. @nickelberg, @roninspin and @defectly, can you run command -v umount and then with the path run ls -l <path returned by command>. I want to check what is providing the umount command. These needs to be run as a root, and I want the output of both commands.

like this?

[jjkola]

@defectly thanks, that was what I wanted to see

@roninspin Using standalone version of chroot-distro is not a problem, only matter of convenience and preference. If you want, you can run mksh -xv <location of chroot-distro> unmount <distro> after normal login, and redirect the output to a file. The output will tell exactly what commands were run, and with what parameters. If you don't mind would you like to share the output for example through pastebin (or some similar service) if you did run the debug run?

[roninspin]

@roninspin Using standalone version of chroot-distro is not a problem, only matter of convenience and preference. If you want, you can run mksh -xv <location of chroot-distro> unmount <distro> after normal login, and redirect the output to a file. The output will tell exactly what commands were run, and with what parameters. If you don't mind would you like to share the output for example through pastebin (or some similar service) if you did run the debug run?

Here you go https://pastebin.com/MNdaLxJw

P.S. After unmount:

ls -lR /storage
/storage:
total 3
drwxrwx--- 4 media_rw media_rw 3452 2023-06-26 20:26 emulated
drwxr-xr-x 2 root     root       60 2024-09-21 14:23 self
ls: emulated: Transport endpoint is not connected

/storage/self:
total 0
lrwxrwxrwx 1 root root 19 2024-09-21 14:23 primary -> /storage/emulated/0

[jjkola]

Based on debug log, the script has worked as it should. I'm starting to think that there may be a problem with toybox implementation of umount in this particular situation, or something else which affects umount calls.

@nickelberg, @roninspin and @defectly: Can you provide as much information about your device as possible?

At least device model, android version, firmware version, twrp version (or if you didn't install twrp, then what did you install, or if you didn't install custom recovery at all), kernel version, busybox version (should not matter but does no harm).

Also, can you guys run toybox --version? Also, if you have custom rom, then what rom, and what version? Also, just in case, whether the device has kernelsu or magisk, and what is the version?

One more thing, can you run mount (and redirect the output to file) before distro login, and after exiting the distro but before unmount command, and lastly after unmount, just so that it is easier to see what happens to mounts. And if you could attach the outputs to your reply (one reply per device)?

[defectly]

Based on debug log, the script has worked as it should. I'm starting to think that there may be a problem with toybox implementation of umount in this particular situation, or something else which affects umount calls.

@nickelberg, @roninspin and @defectly: Can you provide as much information about your device as possible?

At least device model, android version, firmware version, twrp version (or if you didn't install twrp, then what did you install, or if you didn't install custom recovery at all), kernel version, busybox version (should not matter but does no harm).

Also, can you guys run toybox --version? Also, if you have custom rom, then what rom, and what version? Also, just in case, whether the device has kernelsu or magisk, and what is the version?

One more thing, can you run mount (and redirect the output to file) before distro login, and after exiting the distro but before unmount command, and lastly after unmount, just so that it is easier to see what happens to mounts. And if you could attach the outputs to your reply (one reply per device)?

Xiaomi Redmi Note 10 Pro model: M2101K6G codename: sweet Android 13 Cherish OS 4.8 (custom) PitchBlack Recovery Project 4.0 KernelSU version 11872 BusyBox for Android NDK version 1.36.1

mount.txt mount2.txt mount3.txt

this problem was even in stock miui rom with tmoe and magisk

[jjkola]

If somebody could also provide their device information if this is working properly, then that may help providing direction to where this bug may lay. Based on the mount data from @defectly, it should work without problems, but clearly it isn't.

I checked toybox change logs from news page and the last time umount command was changed is for version 0.8.0, and the change was to ignore -c parameter. So, most likely toybox is not the cause for the bug unless it is a bug which has been there always/very long time.

P.S. @nickelberg and @roninspin, I would still like to get the device information from you guys, so that we have more data available to help pinpoint where the problem lays.

[roninspin]

Nubia Red Magic 6 (NX669J) ROM: stock android 12(L) kernel: 5.4.249 (lots of backports) TWRP 3.7.0 KernelSU: 11928 toybox: 0.8.4-android BusyBox for Android NDK version 1.36.1

Turns out that the reason for the problem (at least for me) is the use of umount with -f (forced) for /storage/emulated. Without that parameter no problems at all. So, @jjkola, I think you should omit it, at least for /storage/*.

[jjkola]

@roninspin Thanks, good catch. We will have to investigate this but it sounds like something has changed on kernel side which makes it unusable on newer kernels (my device is Asus Zenpad 10 (Z301M) with 3.18.35 kernel, and it works). Bind mounts should never affect their originator mounts but clearly this has been broken at some point.

@YasserNull Do you happen to remember why -f flag has been added? I tried digging through the version control but umount -lf <path> seems to be even in the first uploaded version of the file.

[nickelberg]

Sorry for not much response from me, I'm kinda busy irl lately.

---

@jjkola

Phone 1:

• Phone: Xiaomi Mi 9T Pro (codename raphael)
• ROM: Android 14 - LineageOS 21 (Unofficial build)
• Kernel: 4.14.350
• Recovery: OrangeFox R11.1_4 Stable
• KernelSU: 11872
• Toybox: 0.8.10-android
• BusyBox: 1.36.1

Phone 2:

• Phone: POCO F4 GT (codename ingres)
• ROM: Android 14 - HyperOS 1.0.4.0.ULJCNXM (xiaomi.eu)
• Kernel: 5.10.192
• Recovery: OrangeFox R12.1_1 Unofficial
• KernelSU: 11928
• Toybox: 0.8.9-android
• BusyBox: 1.36.1

[YasserNull]

@jjkola In fact, I don't remember, but what I do remember is that I used ChatGPT to learn how to use this command.

[jjkola]

I did a test on my raspi device where I did multiple successive bind mounts, and forced unmount (-lf) worked properly with kernel 6.6.47. I did the test with normal mount/umount as well as with toybox (0.8.9) mount/umount. I did use loopback device as the source so that is not the same as with block devices.

[roninspin]

Anyway, -f shouldn't be used all the time (by default).

[jjkola]

Fix released.

[defectly]

Fix released.

so how do i shutdown it?

[jjkola]

I'll try to add more output to error message to make it easier to pinpoint the problem. Most likely I will have time for that next weekend.

[jjkola]

In the mean time, you can try following: mount | grep <mount_point_path> losetup | grep <mount_point_path> lsof | grep <mount_point_path>

If those don't help (and there is no running processes) then you will have to check for anonymous inodes and inotify watchers.

[defectly]

In the mean time, you can try following: mount | grep <mount_point_path> losetup | grep <mount_point_path> lsof | grep <mount_point_path>

If those don't help (and there is no running processes) then you will have to check for anonymous inodes and inotify watchers.

can it be automatic? tmoe has feature to kill chroot, and it also kills chroot if you use unmount in it

[jjkola]

Sure, it can be added. But it can't be default action as it can leave the jail in inconsistent state as it will not bring it down in managed/consistent way. At least, I would prefer bringing my stuff down in a way that it will not cause problems for me in the future.

Also, just to be clear the unmount command just unmounts the known system mount points, it does nothing more, nothing less.

[jjkola]

I have now a PR to add force option to unmount and uninstall commands. Sorry, that it took time to create this PR. I have had the code ready for some time but wanted to do final test round for it, but unfortunately I have been quite busy IRL.

[jjkola]

@defectly Do you have a chance to test my changes from PR? Since you had problems with storage mounts it would be nice to get some input how the new PR is doing. Also, it would be nice if you could include the output here (if the unmount/uninstall command does not run successfully without force option).

[defectly]

@defectly Do you have a chance to test my changes from PR? Since you had problems with storage mounts it would be nice to get some input how the new PR is doing. Also, it would be nice if you could include the output here (if the unmount/uninstall command does not run successfully without force option).

maybe if you give instructions

[jjkola]

Just download the code (either as a zip, or with git) from the PR by going to the branch in my repo. Then create development version of the chroot-distro with the help of hacking guide from README. After installing the package and rebooting the device, just use normally as you would. I'm mostly interested in the output of unmount command (or uninstall if you can test that also).

[defectly]

Just download the code (either as a zip, or with git) from the PR by going to the branch in my repo. Then create development version of the chroot-distro with the help of hacking guide from README. After installing the package and rebooting the device, just use normally as you would. I'm mostly interested in the output of unmount command (or uninstall if you can test that also).

so, in first case i tried unmount with ubuntu when it says device or resource busy, it takes terminal i/o until i press ctrl + c and it doesn't affect running ubuntu

:/ # chroot-distro unmount ubuntu
umount: /data/local/chroot-distro/ubuntu/storage/emulated: Device or resource busy

^C
130|:/ # chroot-distro unmount
chroot-distro - misssing distro
try 'chroot-distro help' for more information
2|:/ # chroot-distro help
chroot-distro : install linux distributions

usage :

chroot-distro help - for more information
chroot-distro env - output debug information about environment
chroot-distro list - list of available linux distributions

chroot-distro download <distro> - download rootfs
chroot-distro redownload <distro> - redownload rootfs
chroot-distro delete <distro> - delete rootfs

chroot-distro install <distro> [-a|--android] - install distro
    '-a|--android'  Installs also /data and /system folders, by default not installed
chroot-distro reinstall <distro> [-a|--android] [-f|--force] - reinstall distro
    '-a|--android'  Installs also /data and /system folders, by default not installed
    '-f|--force'    Force closing open files (closes the process accessing them)
                    and/or unmounting active mounts. Use with care.
chroot-distro uninstall <distro> [-f|--force] - uninstall distro
    '-f|--force'    Force closing open files (closes the process accessing them)
                    and/or unmounting active mounts. Use with care.

chroot-distro unmount <distro> [-f|--force] [-a|--all] - unmount (system) mount points from distro
    '-f|--force'    Force closing open files (closes the process accessing them)
                    and/or unmounting active mounts which prevents (system) mount points
                    from unmounting. Use with care.
    '-a|--all'      Instead of unmounting only system mount points, will try to unmount
                    all found mount points, be it a normal mount point or a loopback mount point.

chroot-distro backup <distro> [<path>] - backup distro
    <path>          Custom path for backup location
chroot-distro restore <distro> [-d|--default] [--force] [<path>] - restore distro
    <path>          Custom path for backup location
    '-d|--default'  restore default settings (note: only those set during install)
    '--force'       Force restore even if may cause unintended side-effects
chroot-distro unbackup <distro> - delete backup from default location

chroot-distro command <distro> <command> - run command
chroot-distro login <distro> - login to distro

:/ # chroot-distro unmount -f ubuntu
umount: /data/local/chroot-distro/ubuntu/storage/emulated: Device or resource busy

in second case i tried unmount and uninstall with debian because i dont want delete ubuntu and it doesn't affect distro too

:/ # chroot-distro unmount debian
:/ # chroot-distro uninstall debian
distro is potentially running - if needed shutdown the distro, and unmount mounted folders

You can try running uninstall again with -f(--force) option to automatically close/unmount files/folders mentioned below.
Please note, that using --force option may close more programs than intended, so use it with care.

open files under /data/local/chroot-distro/debian:
12001   /data/local/chroot-distro/debian/usr/bin/su    0/dev/pts/1
12001   /data/local/chroot-distro/debian/usr/bin/su    1/dev/pts/1
12001   /data/local/chroot-distro/debian/usr/bin/su    2/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  0/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  1/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  2/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  255      /dev/pts/1
end of open files

mount points under /data/local/chroot-distro/debian:
Not found
end of mount points

loopback devices under /data/local/chroot-distro/debian:
Not found
end of loopback devices

If there is no open files, mount points (outside of system mount points) or loopback devices,
then check for anonymous inodes and inotify watches
1|:/ # chroot-distro uninstall -f debian
distro is potentially running - if needed shutdown the distro, and unmount mounted folders

You can try running uninstall again with -f(--force) option to automatically close/unmount files/folders mentioned below.
Please note, that using --force option may close more programs than intended, so use it with care.

open files under /data/local/chroot-distro/debian:
12001   /data/local/chroot-distro/debian/usr/bin/su    0/dev/pts/1
12001   /data/local/chroot-distro/debian/usr/bin/su    1/dev/pts/1
12001   /data/local/chroot-distro/debian/usr/bin/su    2/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  0/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  1/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  2/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  255      /dev/pts/1
end of open files

mount points under /data/local/chroot-distro/debian:
Not found
end of mount points

loopback devices under /data/local/chroot-distro/debian:
Not found
end of loopback devices

If there is no open files, mount points (outside of system mount points) or loopback devices,
then check for anonymous inodes and inotify watches
1|:/ # chroot-distro unmount -f debian
1|:/ # chroot-distro uninstall -f debian
distro is potentially running - if needed shutdown the distro, and unmount mounted folders

You can try running uninstall again with -f(--force) option to automatically close/unmount files/folders mentioned below.
Please note, that using --force option may close more programs than intended, so use it with care.

open files under /data/local/chroot-distro/debian:
12001   /data/local/chroot-distro/debian/usr/bin/su    0/dev/pts/1
12001   /data/local/chroot-distro/debian/usr/bin/su    1/dev/pts/1
12001   /data/local/chroot-distro/debian/usr/bin/su    2/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  0/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  1/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  2/dev/pts/1
12002   /data/local/chroot-distro/debian/usr/bin/bash  255      /dev/pts/1
end of open files

mount points under /data/local/chroot-distro/debian:
Not found
end of mount points

loopback devices under /data/local/chroot-distro/debian:
Not found
end of loopback devices

If there is no open files, mount points (outside of system mount points) or loopback devices,
then check for anonymous inodes and inotify watches
1|:/ #

[jjkola]

Thanks. I'm interested in knowing what is happening with that Ubuntu unmount. Can you run /system/bin/sh -xv /system/xbin/chroot-distro unmount ubuntu > debug.log 2>&1, and attach the debug file here? I want to ensure that it isn't a bug in the chroot-distro implementation.

[defectly]

Thanks. I'm interested in knowing what is happening with that Ubuntu unmount. Can you run /system/bin/sh -xv /system/xbin/chroot-distro unmount ubuntu > debug.log 2>&1, and attach the debug file here? I want to ensure that it isn't a bug in the chroot-distro implementation.

there's one line only

/system/bin/sh: system/bin/sh: inaccessible or not found

[jjkola]

Sorry, I wasn't in front of my computer and remembered the invocation incorrectly. The correct one is: /bin/sh -xv /system/xbin/chroot-distro unmount ubuntu > debug.log 2>&1

[jjkola]

Hmm, that's wierd. I have /system/bin/sh (this one is mksh) and /bin/sh (and this one is ash, at least based on whatshell.sh). Can you run /bin/sh and from within that shell run that script? I would like to know what it is. Also, what does command -v sh say? And, just for posterity, can you run /system/bin/sh just for posterity?

Also, if you can include the output from chroot-distro env?

[defectly]

Hmm, that's wierd. I have /system/bin/sh (this one is mksh) and /bin/sh (and this one is ash, at least based on whatshell.sh). Can you run /bin/sh and from within that shell run that script? I would like to know what it is. Also, what does command -v sh say? And, just for posterity, can you run /system/bin/sh just for posterity?

Also, if you can include the output from chroot-distro env?

/system/bin/sh returns nothing

command -v sh returns "/system/bin/sh" and maybe i wrote command wrong because now it writes this to file

/system/bin/sh: /system/xbin/chroot-distro: No such file or directory

and same does

 /bin/sh -xv /system/xbin/chroot-distro unmount ubuntu > debug.log 2>&1

chroot-distro env returns

Version: 1.2.3
Versioncode: 11
md5 hash: 8101eb5286a8b5c647a409debccb0da1
Toybox version: toybox 0.8.6-android
busybox => '/system/bin/busybox'
Busybox version: BusyBox v1.36.1-osm0sis (2023-09-02 19:01:41 ADT) multi-call binary.
chroot => '/system/bin/chroot'
lsof => 'busybox lsof'
KernelSU installed (version not known)
Busybox for Android NDK Magisk module installed
Linux 4.14.349-openela~SLEEPY #1 SMP PREEMPT Wed Jul 31 14:08:05 UTC 2024 aarch64 Android
PATH=/debug_ramdisk:/sbin:/sbin/su:/su/bin:/su/xbin:/system/bin:/system/xbin:/data/adb/ksu/bin

[jjkola]

Did you accidentally drop the first row that contains the script location (from the chroot-distro env output)? You need to use the same location for sh invocation.

[defectly]

Did you accidentally drop the first row that contains the script location? You need to use the same location for sh invocation.

yes

Script: /system/bin/chroot-distro
Version: 1.2.3
Versioncode: 11
md5 hash: 8101eb5286a8b5c647a409debccb0da1
Toybox version: toybox 0.8.6-android
busybox => '/system/bin/busybox'
Busybox version: BusyBox v1.36.1-osm0sis (2023-09-02 19:01:41 ADT) multi-call binary.
chroot => '/system/bin/chroot'
lsof => 'busybox lsof'
KernelSU installed (version not known)
Busybox for Android NDK Magisk module installed
Linux 4.14.349-openela~SLEEPY #1 SMP PREEMPT Wed Jul 31 14:08:05 UTC 2024 aarch64 Android
PATH=/debug_ramdisk:/sbin:/sbin/su:/su/bin:/su/xbin:/system/bin:/system/xbin:/data/adb/ksu/bin

debug.log

[jjkola]

Thanks, and I forgot that the script is installed at /system/bin directory. Just after you posted your latest output I checked my own device only to find out that you had already found the correct location. There is some fishy going on based on debug output. I will have to debug it. Unfortunately, I'm not sure when I will have time for that, but I will do it as soon as possible.

[jjkola]

I'll close this after the investigation.