Huavei Mate 10 lite and Proot -Simlink issue

[JanuszChmiel]

Dear specialists, I AM user of Termux terminal and I Am intensively using Proot package to run The various Linux distributions. Unfortunately, Huavei 10 Mate lite contain very probably some kernel issue related to Symlinks getting information and creation. Because no distribution based on Proot work here. Do you think, that you could help Me to debug this issue and may be, that you could recode Proot, since it is too complex C app for advanced user and I can not modify so complex code which call Kernel routines and function without deep Android kernel knowledge my self. It is not toy for children. But I Am patient and I can debug with you. Now, I Am runnin Android 7.0 and I Am ready to send you some technical information, I Am attaching them as A log file. Do you think, that you could help Me, Or I must accept The fact, that kernel od this device model will never allow Me to run Linux distributions by using Simlinks and Proot? The following command causes errors and file system creation can not continue.

proot --link2symlink tar -xf $cur/debian.tar.xz --exclude='dev'||:

Here is my Log about my Android device.

setupTermuxArchSysInfo7cf5.log

[michalbednarski]

We'd need to see what's going on, so please try running above command with few extra options and share results

proot -v 9 --link2symlink tar -xvf $cur/debian.tar.xz --exclude='dev' &> log-extract.txt
ls -Rla &> log-list.txt

Added -v 9 switch to proot and -v to tar; alternatively you can use PROOT_VERBOSE=9 env variable. Probably easiest way to do so is modifying installer script but you can also run them manually (note that in that case you should be in new empty directory and fix path to archive).

[JanuszChmiel]

Dear Mr Bednarski, Thank you very much for yours professional answer. I have done my best to find The cause myself, but it is too complex for Me, so I will be very glad that I can follow yours advice. I will send you logs. One important remark!!!

I Am even getting warning while installing packages by using Termux by using apt even if I do not use Proot. I even think, that may be, that Termux do not successfully complete initial download stage. Because when comparing with Android 6.0 device, Modem download much less data. But The fact is, that I Am getting warning, that symbolic link have been changed from one value to other. Do you think, if is this correct? If it is normal behaviour, or if this would never happen at all. I Am sad, because This mobile phone model work and I had A dream, that I will use much more available of RAm than while working with older devices, which had 2 or 1,5 GB of RAM. I know, that Android will not allow Me to use 1,9 or 2 GB for Proot tasks, but if it will allow me to use 800 MB or 750, it would be excellent. I had experiences with Orca screen reader and speech engines, which are working fluently in combination with Pulseaudio for Termux. It is stable solution and work even while working randomly with sounds after 1 hour of phone inactivity. So I will try to debug with Proot according to yours advice. The positive is, that Android kernel is so safe, that allows its users to run Linux apps in combination with Android apps.

[JanuszChmiel]

Because command which executed proot did not created log for Me, I have decided, that I will begin from scratch. From The stage, when termux have been executed for The first time. Here is The log. Do you think, that there are some problems while Termux download some data from Internet and need to install something? Because if yes, Proot code have no blame on The issues which I AM experiencing. Here is The big log file.

termux.log

I will also upload The next one when I will perform Termux distribution upgrade by typing apt update apt dist-upgrade Because I always get warning about sim link change from one number to another.

[JanuszChmiel]

Here is The simbolic link related issue. Please is this behaviour normal? Or is it really The signal, that some Android background process is doing somethink not good?

Progress: [ 94%] [######################################################....] dpkg: warning: symbolic link '/data/data/com.termux/files/usr/var/service/telnetd/log/run' size has changed from 66 to 62

dpkg: warning: symbolic link '/data/data/com.termux/files/usr/var/service/ftpd/log/run' size has changed from 66 to 62

Progress: [ 96%] [#######################################################...] Progress: [ 98%] [########################################################..] And here is next log after performing apt dist-upgrade log-list.txt

[ghost]

I have observed that behaviour on Android 10 but did not found any issue with created symlink (they exist and point to correct file).

[JanuszChmiel]

OK. So I can be quiet, that problem do not reside in The process, when Termux install packages, when DPKG create files. I will try to enable 9 value for Proot loggin while using this perfectly prepared script.

https://github.com/sp4rkie/debian-on-termux The process of installation produces fully usable Debian on my Lenovo Tab 3 8 device including DBUS inter process communication support. Orca screen reader and at-spi cooperation. I think, that we are near to The solution for this specific phone model, because many apps, which Debootstrap install are being configured correctly without errors. This is not true for other simpler scripts such as Ubuntu script for Termux.

[JanuszChmiel]

Please, who of us would have some time and would be so kind that could modifi this script so all possible loggint output for Proot, debbootstrap and tar can be involved so it will be chance to find out, why The installation of this system can not be finished successfully? Script is perfect and bug is not inside its code. I have confirmed, that on Lenovo 3 8 tablet and Android 6.0 it work perfectly and I can login as root by running /bin/.enter_deb So script is here.

https://raw.githubusercontent.com/sp4rkie/debian-on-termux/master/debian_on_termux.sh

I Am running Windows and Putty, I have also logging in Puty enabled, so in conjunction with The logging of Proot and Tar and Debootstrap we have A real chance to find out The issue. I would never bother you, but I Am sure, that script is working and that there is really some hydden complex issue on Huavei 10 Mate lite. It also work with Blackview A5. But this model uses only 1 GB of RAM.

Thank you very very much for yours patience with Me. I know, that I Am among real experts and that if it will be possible to find The solution, you will find it with Me.

[michalbednarski]

Use following commands for installation (first is to clean old install; second is running actual installation, note that with such verbose logging it might take more time than standard install, please attach log-setup.txt.gz when finished)

rm -rf deboot_debian debootstrap debootstrap-1.0.116
PROOT_VERBOSE=9 bash -x debian_on_termux.sh |& gzip -9 > log-setup.txt.gz

[JanuszChmiel]

Thank you very very much for yours kind access to my plea and for The command. I will just begin and I will be patiently waiting, till The installation process will finish.

[JanuszChmiel]

Here is log. Fortunately, it have been really created.

log-setup.txt.gz

I will hope, that it is not complex kernel related issue, which will not be possible to overcome without rooting or without Because if yes, I will have to be deeply sad and I will not be able to use this device model to run Linux distributions by using Proot. Really thank you very much for yours time, patience and for The analysis of The so long log.

[michalbednarski]

Looks like installation failed due to case insensitive filesystem

You could probably perform installation if you'd exclude problematic packages by modifying installation script:

--- debian_on_termux.sh 2019-12-06 08:42:32.917835977 +0100
+++ debian_on_termux.sh        2019-12-06 13:43:29.973774933 +0100
@@ -123,7 +123,7 @@
     -r "$PREFIX/.." \
     -0 \
     --link2symlink \
-    ./debootstrap --foreign --arch="$ARCHITECTURE" "$VERSION" "$HOME/$ROOTFS_TOP" \
+    ./debootstrap --exclude=cron,init,libpam-runtime,login,mount,systemd,systemd-sysv,sysvinit-utils,udev,util-linux,iptables,logrotate --foreign --arch="$ARCHITECTURE" "$VERSION" "$HOME/$ROOTFS_TOP" \
                                                                 || : # proot returns invalid exit status
 } # end DO_FIRST_STAGE

If package you need has contains problematic file you can configure apt to exclude some files after finishing system installation

---

Your previous ls -Rla lists were made on /sdcard (or ~/storage or /storage)? In general files cannot be executed from that locations so you'll need to stay in Termux $HOME. Your later debian_on_termux.sh installation seems to be done from correct location, just looks like filesystem is case-insensitive (but symlinks and file execution work there)

[JanuszChmiel]

Thank you very much for yours analysis. Why do you think, that kernel developers from Huavei have decided to recode Android kernel so my model uses case insensitive filesystem? Do you think, that it is The good idea, which have been created because of security? Or it is AN mistake while script format internal flash chips and create this kind of file system type? I know, that I can not simply change this behaviour so I must accept it.

Thank you for yours advice related to package exclude. It is sad, that manufacturers have decided to set The file system by this way. Interesting is, that Termux related package installation is working.

And I will always start every script outside /home/storage/downloads

[ghost]

@JanuszChmiel Can you post here the content of /proc/mounts ? That will explain which options were used for mounted file systems.

[JanuszChmiel]

It is sad, that non rooted Android kernels do not allow to use file container. In this case, file container could have a file system type other than file system type, which is being used internally for formatting and data storage inside flash chips. But it is very unlikely, that it would be possible to use Proot by this way on non rooted Android. I have read some articles and I have found out, that some file systems are really case insensitive filesystems and some no. I also know, that using one big file for storing Linux distros is also not good idea. It good idea for USB external harddisks with The rrotating plates or for fast SDD disks. But when I would have The opossibility to use this big file container, it would cause very slow operations and my chips would be very probably overburned quickly.

[JanuszChmiel]

When I execute /proc/mounts I Am getting

Permission denied error message. I execute from The home folder.

[JanuszChmiel]

And when I want to run this command after executing enter_deb It produces Bas-5.0 line. And When I execute /proc/mounts
It also return The Permission denied.

Sure. Because installation of Debian fail and there is no possibility to use su or sudo. To overcome this, I will use more simple script such as little script for installing Ubuntu. But sure, ideal would be to solve Debian installation by using longer script which produces usable system also on other devices.

[ghost]

@JanuszChmiel You need cat /proc/mounts.

[JanuszChmiel]

Here is log produced by this command outside system which run Proot so directly from Termux shell, not from Debian Bash shell. I will send The next log if partially installed Debian allow Me to produce The output.

MOUNTS_LOG.txt

[corbinlc]

Something similar was reported here: https://github.com/Neo-Oli/termux-ubuntu/issues/87

@JanuszChmiel sorry you are seeing this issue in UserLAnd too. It would be the same cause. I did some work on solving case insensitive issues to allow installing on the sdcard but 1) the required the internal file system to still be case sensitive and 2) I never finished solving all the corner cases. I can think of a way to modify that work to solve this, but it would be quite a bit of work and would impact performance significantly. As such, I don't think it will be worthwhile given this doesn't seem to happen on a wider set of devices and android version.

[JanuszChmiel]

I have thought, that this will not be easy to fix. It is sad, that it is impossible to use kernel loops on non rooted Android devices to simply use file container, which would not reside on AN sd card but directly inside flash chips of Android device. But I know, that The performance would also be very degradet even if big file would reside inside internal flash chips. In some cases, advanced user must simply give up too complex programmers issue. But if Android kernel do not allow to use some techniques, which will allow users to use file container to create file system here, I will have to accept this fact. Instead of to be sad, I will use other ammazing features of my phone because to be honest, its kernel is very sstable including sound chip drivers, UI is also stable so I will focus on other thinks which are working. Termux packages are working, but I Am not C or C++ professional programmer to recode all Dependencies of Orca and other apps. It would be heavy and long task even for professionals.

[JanuszChmiel]

I Am wondering if is it possible to get Android kernel log without rooting The device. Standard Debian support dmesg. Is it possible to use Termux for The similar task? Or Android kernel have no loggind enabled? I Am still thinkink about my issue. I have also found information, how to 100 % verifi, is file system is case insensitive or no. Here is The command how to achieve this. touch a A && ls I will firstly test it before running some Linux distro Bash shell by using Proot and I will try this command also after executing some Linux distro by using Proot. If The command will create two different files with two different file names, so only case characters difference will exists and there will be two files? Is there any chance to modify Proot source code to support my phone model?

If is it possible to analyse also original Android kernel I will do my best to also put this log to be stored while Proot will work, because we allready have very detailed Proot log while installing Debian.

I Am very sorry that I Am still intruding you. My source os my motivation resides in Pandroid big project, which is being capable to compile Android apps from Pascal sources. Its author have done its best to collect all legally available opensource modules so even programmer who do not see at all can really program. I want to use this bundle from Linux on Android, because computer CPU have getting hot while building by using Apache ant. And it would be much more easier to code directly on The phone than combining The computer with The phone.

My other devices have too few internal storage space and too little RAM available. There is no other Pascal language based development technique. And thanks to Pandroid, I can create GUI with no need to put numbers to specifry every object position. So this is The reason why I want to use Orca and Lazarus. Sure, I know, that I can not pretend The same speed like when I could run it on standalone running Linux, but for my little apps such as radio players with simple GUi for visually impaired users it is enough.

[michalbednarski]

I don't think kernel log will contain useful information in this case

Depending on system it might be possible to turn case insensitivity off, try running following:

pkg install e2fsprogs
mkdir testdir
lsattr -d testdir
chattr -F testdir
cd testdir
touch a A
ls

Even if above wouldn't work I believe that you could have usable Linux distro running with minor configuration changes even if filesystem is case insensitive

[JanuszChmiel]

Oh MR bednarski, You have made Me big joy. I will try The commands which you have adviced to Me. I Am very glad, that I would have some chance to run some Linux distributions. FOrtunately for Me, Pulseaudio, which is The part of Termux have no hydden internal issue similar to PUlseaudio which is The part of latest Available Xserver XSDL. Because pulseaudio which is The part of Termux repositoryes work perfectly. Sound is smooth, no distortions and even if I use sound after several minutes, some modules do not denie Pulseaudio from working and sound is still perfect. So thank you again for yours advices.

[JanuszChmiel]

Here is The result of our test. Do you think, that it have worked? or unfortunately for Me no?

$ mkdir testdir
$
$ dir
bin deboot_debian debootstrap-1.0.116 testdir
buildAPKs debootstrap storage
$ lsattr -d testdir
-------------------- testdir
$
$ chattr -F testdir
$
$ cd testdir

$
$ dir
$ touch a A
$

$ dir
A a
$ ls
A a
$

[ghost]

@JanuszChmiel Your FS is case-sensitive and is ok. Otherwise command touch a A would not have created 2 files.

[JanuszChmiel]

Thank you very much for yours time. So we are in The beginning. I have other possible source of The problem. What if The cause of my problem is really inside build in antivirus based on Avast antivirus engine. But those proceses can not be reliably stopped for A longer time. But I will give it A try.

[JanuszChmiel]

Really strange issue. I have disabled on cloud files checking in build in antivirus because I have thought, that it can make some bad activity in The background. But no success, grep can not still load shared library. And The most sad think is, tht problem do not only exist for Debian, but for all distributions available. Is there other log which I could create to find The real cause of The problem? Or we have done maximum what can we do?

[michalbednarski]

Could you also try running following commands and sharing result

cd ~/deboot_debian/usr/share/man/man7
touch a A B
ln -s B b
ln -s C c
ls -l

[JanuszChmiel]

And do I had to run those commands after executing enter_deb script so bash 5.0 will be executed by using Proot or should I only execute those commands from Termux environment?

[michalbednarski]

Directly (without enter_deb)

[JanuszChmiel]

Here are commands and their results. I have executed them after running enter_deb Because .so library can not be loaded, I AM only running Bash without rooting mode and system is very strange.

bash-5.0# touch a A B
touch: cannot touch 'A': File exists
bash-5.0# ln -s B b
ln: failed to create symbolic link 'b': File exists
bash-5.0# ln -s C c
bash-5.0#
bash-5.0# ls -l
ls: error while loading shared libraries: libpcre.so.3: cannot open shared objec t file: No such file or directory

[JanuszChmiel]

$ cd ~/deboot_debian/usr/share/man/man7
$
$ touch a A B
$
$ ln -s B b
$
$ ln -s C c
$
$ ls -l
total 64
-rw------- 1 u0_a104 u0_a104 0 Dec 10 21:32 A
-rw------- 1 u0_a104 u0_a104 0 Dec 10 21:32 B
-rw-r--r-- 1 u0_a104 u0_a104 2276 Feb 14 2019 PAM.7.gz
-rw------- 1 u0_a104 u0_a104 0 Dec 10 21:32 a
lrwxrwxrwx 1 u0_a104 u0_a104 18 Dec 10 21:33 b -> B
-rw-r--r-- 1 u0_a104 u0_a104 508 Apr 18 2019 bash-builtins.7.gz
lrwxrwxrwx 1 u0_a104 u0_a104 18 Dec 10 21:33 c -> C
-rw-r--r-- 1 u0_a104 u0_a104 1635 Feb 14 2019 pam_env.7.gz
-rw-r--r-- 1 u0_a104 u0_a104 1842 Feb 14 2019 pam_selinux.7.gz
-rw-r--r-- 1 u0_a104 u0_a104 4160 Nov 2 19:16 term.7.gz
$

[JanuszChmiel]

I think, that I Am running very interesting variant of Android kernel. Really thank you very very much that you are doing yours best to find The cause of my issues. The reality is, that some files can not be simply extracted. On arch Linux but even while running very simple Ubuntu script. The interesting fact is, that all Termux packages will be very probably working without issues. Only warning about simlinks, but files aare on its place.

[JanuszChmiel]

What do you think about The kernel? Does The results of commands show somethink unusual? Or no and problem is very probably deep in other part?

[michalbednarski]

Looks like there's partial case-insensitivity while running inside proot (or possibly any other debugger, logs show that case-insensitivity isn't directly caused by proot) and yet filesystem is case-sensitive outside proot.

Another unusual thing in results is touch a A (with names differing in case) reporting an error, which I didn't expect as normally touch a a (with exactly same filename) doesn't report error, just opens and closes same file twice. (thats why I'm calling file system partially case-insensitive as it seems it disallows creating files with different case, but only while debugging (so I'd guess this behavior is some leftover after device kernel development))

At this point I think it'd make most sense to use modified installation script for Debian installation and use system despite filesystem case insensitivity, as further experiments probably would likely give trivia but not actually go

Below is script for modified Debian installation, base installation will have some packages omitted but you should be able to use apt to install them. Note that this script will erase your current Debian installation (which wasn't working)

#!/data/data/com.termux/files/usr/bin/bash
set -xe
cp -f debian_on_termux.sh debian_on_termux2.sh
patch <<'END'
--- debian_on_termux2.sh    2019-12-06 08:42:32.917835977 +0100
+++ debian_on_termux2.sh    2019-12-06 13:43:29.973774933 +0100
@@ -123,7 +123,7 @@
     -r "$PREFIX/.." \
     -0 \
     --link2symlink \
-    ./debootstrap --foreign --arch="$ARCHITECTURE" "$VERSION" "$HOME/$ROOTFS_TOP" \
+    ./debootstrap --exclude=cron,init,libpam-runtime,login,mount,systemd,systemd-sysv,sysvinit-utils,udev,util-linux,iptables,logrotate --foreign --arch="$ARCHITECTURE" "$VERSION" "$HOME/$ROOTFS_TOP" \
                                                                 || : # proot returns invalid exit status
 } # end DO_FIRST_STAGE
END
rm -rf deboot_debian debootstrap debootstrap-1.0.116
bash debian_on_termux2.sh
mkdir -p deboot_debian/etc/dpkg/dpkg.cfg.d
echo path-exclude /usr/share/man/man7/pam.7.gz > deboot_debian/etc/dpkg/dpkg.cfg.d/01_bad_case

[JanuszChmiel]

Dear Mr bednarski, Thank you very very much for yours very big patience and for yours free time, which you have iinvested for this non easy issue. Really very very well done. MR bednarski, you are really expert and you can be very prout on yourself. Because you are not only expert, but you have A good will to give yours energi for such complex project. Thanks to people like you are, Opensource software will exist for ever. I wish you many programmers achievements and much positive personal achievements. I AM really deeply appreciating you.

[JanuszChmiel]

I have confirmed, that modified patched script work like A charm. I have root account available and all commands including dir are working. And I can try to install my favourite packages. So my dream will come true thanks to yours team.

[JanuszChmiel]

It would be A good idea if Termux integrated shell would create some environment variable which would contain exact Phone name and its model so Bash and other script developers would create a specific code for such situations like Huavei 10 mate lite has caused. BUt I do not know, if it would not bee too complex to integrate such code to The termux. And if is it A good idea from Me.

[ghost]

some environment variable which would contain exact Phone name and its model so Bash and other script developers would create a specific code for such situations

@JanuszChmiel Already done (by Android):

$ getprop ro.product.manufacturer
samsung
$ getprop ro.product.model
SM-G930F

[JanuszChmiel]

Oh, I have been overjoied too soon. System work but when installing more complex group of packages such as LXDE, I AM getting errors. So it looks, that kernel behaves unpredictably and that I will never have full freedom to install what I will need. But thank you I know, that you have done yours best of all what you could do for Me. I will try newer Huavei if this kernel behaviour also exist on newest Android release. If yes, I will try to kindly ask Huavei programmer for An repair, but I Am rather skeptic, that they will do that, because using Proot and Linux distribution is not standard user task with Android device so may be, that kernel developers will simply ignore my kind plea. But thank you very very much. We were very near to The functioning Debian system.

[JanuszChmiel]

Oh I found The issue and its cause. It is necessary to add The following line to /etc/dpkg/dpkg.cfg when running The system by typing ./enter_deb Is this line correct?

path-exclude /usr/share/man/man7/pam.7.gz > deboot_debian/etc/dpkg/dpkg.cfg.d/01_bad_case Can be this line like a last line of dpkg.cfg?

[JanuszChmiel]

So this code

mkdir -p deboot_debian/etc/dpkg/dpkg.cfg.d echo path-exclude /usr/share/man/man7/pam.7.gz > deboot_debian/etc/dpkg/dpkg.cfg.d/01_bad_case

Are those characters 
``` also The part of The script code? Or those chars are not its part?

I have found out, that Espeak support Pulseaudio which run from Termux like A charm. I can also install it rom Debian. But for now, I must forget on installing some packages such as Aptitude. Mr Rausty is convincing me that I could use his APK builder project, but I do not know, which other module than Proot can safely and reliably translate some Binary code to kernel so Linux app which uses LibC will work on Android on non rooted device. I do not assume, that it is possible to use other free and stable other tool than Proot for running Linux distributions.
Proot is good invention and without that, there would not be too much possibility to run Linux distros with terminal emulation on Android devices.
Sure, it is true, that if I were absolvent of informatic sciences, I would even go by using other way. Creating more and morepackages directly for Termux, but I do not assume, that this task only involves to make only cosmetic changes in The C or C++ source code And making big projects such as LXDE will very probably require many many changes of many source files before building it for Termux. 

But next true is, that if I want to make my dream to come true, that I must do something for that. So I will start to read Termux packagers development wiki and I will see on simple C apps, which changes are necessary to build them for Termux environment directly.

[JanuszChmiel]

I have finally found special script and fully functioning Linux distribution, which have some internal magic, so all packages without differences, every packages can be installed and Proot is working perfectly even with my specific Android kernel. No special tricks to make The system work as it should. No errors while upgrading whole distro. But it would be too complex for Me to tune Debian by this way and I Am also aware, that I can not intrude you so much because of it. The distribution, which is working without file system case insensitivity problems is Fedora. But I could not found functioning script before. Now I have The one and I Am now installing LXDE. I only hope, that developers of Fedora did not changed some .so libraryes so I will not be able to stream sounds to Pulseaudio which run from Termux. If yes, I will try to discuss on Fedora ARM mailing lists about it. But for now, it looks very promise. I could apply full distribution upgrade and I Am using Fedora 31 for ARM64 BIT CPU. And phone do not produce much heat and The system is stable. So I Am very sorry MR Bednarski and others that I have intruded you so much about Debian distribution.

[ghost]

Ok, then I guess issue can be closed.