search

RalfWerner / integrated-process

sources for check X11
3 stars 0 forks source link

API 29 example #2

Open RalfWerner opened 4 years ago

RalfWerner commented 4 years ago

Install APK

What happens if my t*.apk will installed on a Q/10 device and opened (bash) or is it not possible: type bash is non-native(~/u)? Four shots to install and use of example check see README: grafik

The action is simple with Android 9. To do the same in 10/API-29 would have to be u/bin u/lib (ln -s ~/../usr ~/u) and ~/check.exe ($s) in native /system. @xeffyr said Yes to Could the APK solution be tested on Andriod-9? How does it work?

Github has a limit that my-t77.apk (63Mb) can not be saved - is also here and created with apksigner. There are also the versions 0.73 (smaller but needs package update) and 0.82 to find. An exchange of the 'my-t*.apk` (same signing) has no restriction of the check function to follow.

@fornwall made a (proot less) suggestion in dev - where do I find (and how to use) the 'termux-package-binary' and do you've an example of a script?

ghost commented 4 years ago

What happens if my t*.apk will installed on a Q/10 device and opened (bash) or is it not possible: type bash is non-native(~/u)?

Android prohibits usage of native code where it can be modified. This restriction is enforced for all applications targeting at Android SDK level 29.

Currently Termux targets on SDK 28 and therefore it is not affeced, but will be once we bump used SDK version.

To do the same in 10/API-29 would have to be u/bin u/lib (ln -s ~/../usr ~/u) and ~/check.exe ($s) in native /system.

No. It has to be in /data/app/${random dir name}/lib. This directory is extracted from APK during application installation process.

To place binary into /system you need root.

@xeffyr said Yes to Could the APK solution be tested on Andriod-9? How does it work?

Assume that binary can be executed only from /data/app/${random dir name}/lib and its execution causes permission denied error if doing this from /data/data/${package name}.

Nothing dependent on specific Android versions. Just native code must be placed at paths where it should be.

The closest implementation to one that will be used for Termux is https://github.com/xeffyr/alpine-term/tree/master/alpine-term-app - this one has 100% Android 10 compatibility (qemu/socat binaries in jni lib dir and data in dynamic $PREFIX).

where do I find (and how to use) the 'termux-package-binary' and do you've an example of a script?

Such script is not exist. But generally, it will be a wrapper around utilities zip and apksigner for bundling user's binary inside prebuilt APK.

This approach will be the lowest priority target for Termux as soon as proot can be used.

RalfWerner commented 4 years ago

This directory is extracted from APK during application installation process.

In my example 400Mb created in u/bin and u/lib etc. with virgin start and package update (also contains bash). In /system/bin are 8.5Mb and in */lib 185Mb. At least part of the 400Mb would have to be in the t*.apk _right?

A: To place binary into /system you need root. B: But generally, it wants to be a wrapper, for bundling user's binary inside prebuilt APK. C: This approach wants to be the lowest priority target for term as soon as possible.

A: This does not apply if the APK already contains everything or? B: What would the termux-build look like to do this (+bash etc)? C: As I understood last Sunday's discourse, the priority should be higher for it to be sustainable.

ghost commented 4 years ago

At least part of the 400Mb would have to be in the t*.apk _right?

Right.

This does not apply if the APK already contains everything or?

Application cannot modify the system partition which is created by manufacturer and is read-only.

User applications are installed only to /data partition.

What would the termux-build look like to do this (+bash etc)?

It will just append a file to APK, which in fact just a zip archive. Then it will be signed with apksigner.

Steps will look like these:

cd $TMPDIR
cp $PREFIX/share/user-bin-wrapper/base.zip termux-bin-wrapper.zip
mkdir -p lib/armeabi-v7a/
cp $HOME/your_binary lib/armeabi-v7a/
zip -r9u termux-bin-wrapper.zip lib
apksigner keyfile.jks termux-bin-wrapper.zip $HOME/termux-bin-wrapper.apk

You will need to install termux-bin-wrapper.apk and open it by clicking on icon to start service exposed to Termux.

Then

termux-bin-wrapper your_binary [set of args]

Neither wrapper or script are available or being developed now.

That method of executing user binaries has some cons and will never replace proot or direct binary execution.

RalfWerner commented 4 years ago

Ok, I understood. /system/* can not be changed when built by t*.apk (only Huawei / Gigaset can do that in my case) so there will never be /bin/bash. The 'termux-bin-wrapper' from your description does not exist. Is this 'termux-package-binary' that @fornwall spoke of?

Probably it will never be developed because you consider the "proot solution" to be "future proof" or should t*.apk be replaced by alpine-term.apk (then I would have more questions)?

ghost commented 4 years ago

Is this 'termux-package-binary' that @fornwall spoke of?

Yes, it is.

or should t*.apk be replaced by alpine-term.apk

alpine-term's parts can be used to execute single binaries https://github.com/xeffyr/alpine-term/blob/master/alpine-term-app/app/src/main/java/alpine/term/TerminalService.java#L319 but whole application cannot.

RalfWerner commented 4 years ago

Stupid question about Q/10/SDK 29: If I wanted to update my Gigaset is missing the "manufacturer update" and in my Huawei the termux app would not work (many others may not) - right?

If you update your device to Android 10, Termux wants work because it uses older sdk.

Ok thanks - so there's no need to adapt termux to the restictions as SDK<29 apps will also work in future releases. So it's just about hypothetical extensions.

My example contains only one single binary s=~/check.exe but a lot of environment. Do you think a test with alpine-term.apk could be successful?

ghost commented 4 years ago

Do you think a test with alpine-term.apk could be successful?

Can you explain what exactly you want to achieve:

Solutions for these 2 cases are different and not comparable.

RalfWerner commented 4 years ago

Obviously my idea of ​​the SDK=29 adjustment was wrong, since termux works without the binary restiktionen as before on Q-devices. The problem only occurs when the APK-built-process with version 29 is done - right?

So comparable to the android-5 branch, this only becomes relevant when the android-7-9 branch is discontinued or if Q extensions are to be used by Termux. Both are not urgent, especially since a termux-sdk=29-version (maybe 1.00) does not exist yet and Q-Update is not yet possible.

Also wrong was my idea of ​​the PATH to native lib. If not /system (I used in the example) then where?

Can you explain what exactly you want to achieve:

All in all, be compatible with future devices and apps, and second, if possible, fix any current issues with it. In my example two APK with the same signing (my-) are used. I built the my-x11.apk (plugin from @twaik) with your help and `SdkVersion 28` on my PC. It contains at least two bugs (SSM and unexpected crash in direct mode) and currently also the described mpv-x bug (also occurs exactly in Xvnc).

I do not know if one of your two solution suggestions can fix the bug - probably not - and which of the two would be used in termux-1.00?

ghost commented 4 years ago

The problem only occurs when the APK-built-process with version 29 is done - right?

Right.

So comparable to the android-5 branch, this only becomes relevant when the android-7-9 branch is discontinued or if Q extensions are to be used by Termux.

Nothing to do with packages ! Android 7 branch will be used for Android 10.

Problem comes from https://github.com/termux/termux-app/blob/master/app/build.gradle#L18 being set to 29. We will have to do this, otherwise Google will delete app from Play market.

Post https://github.com/termux/termux-app/issues/1072#issue-422258684 has the most clear explanation about the problem.

If not /system (I used in the example) then where?

/data/app/package.name-${random characters}/lib

and which of the two would be used in termux-1.00?

See https://github.com/termux/termux-packages/wiki/Termux-and-Android-10. Embedding in apk has 2 unsolved problems. Until these problems will not be solved, no work on porting Termux to Android 10 will be done.

RalfWerner commented 4 years ago

Thanks, my own APK are not affected because they are not offered by google play. On the native lib:

/data/app/package.name-${random characters}/lib - Embedding in apk has 2 unsolved problems...

I could not access also proot (with termux-chroot ls /data ../data) because: drwxrwx - x 55 system system 4096 Nov 25 13:45 data/ belongs system. Maybe if I knew package.name-${random characters} or have to install additional APK / package?

to the mpv-x bug I have in dev still made a further test.

ghost commented 4 years ago

Maybe if I knew package.name-${random characters} or have to install additional APK / package?

echo $(dirname $(pm path com.termux | cut -d: -f2))/lib
/data/app/com.termux-I-EOhD7byPe8v52EoueQxw==/lib

This directory is owned by system and is read-only.

RalfWerner commented 4 years ago

Yes there is */lib/arm64 with two *.so (aarch64) and no write permition except system. On Gigaset (arm) I get only . . . ./lib as PATH to native lib and echo $(pm path com.termux) ended with cmd: Failure calling service package: Failed transaction (2147483646). You got it too?

Embedding ~/check.exe would require write permission (your first suggestion above?) but do not fix any of my current issues above - maybe later, when termux-1.00 exist! Thanks so far.

ghost commented 4 years ago

You got it too?

No. But I expected that. The normal Android way of obtaining that path is from java code only.

Embedding ~/check.exe would require write permission (your first suggestion above?)

No. If check.exe is embedded in APK, it will be extracted to ./lib directory like other *.so files.

RalfWerner commented 4 years ago

From the dev team chat yesterday, I understood that the ./lib - way is currently not- tracked but proot. You have created an apk, which I install after re-signing: apksigner keystore app-debug.apk my-t84.apk

You could save the 64 Mb in Github - I did not succeed because of the 25 Mb limitation! How did you do this? In your *gz I discovered SdkVersion 29 in */app/build.gradle. The X11/Wtermux plugins of my example were created with 28 and I've used my-t82.apk before. Differences:

PID1 (com.termux) as well as PID2 (.termux.wtermux) are again recognizable as processes (X11 service not) and a libproot.so process has been added (sometimes several). The two X-servers and ssh are started with OTS. I can perform all checks as before and get all known errors (SSM, crash in Direct mode of X11 and mpv-x without video). I have to examine several effects in more detail - later here the first:

Review

I test with the full script s, (packages in ~/u and pkg up done), t84 (968Mb) on aarch64 and t82 (798Mb) on armv7l and all plugins. Termux start (OTS with sshd) is the same for both devices.

grafik

Questions about t84 and proot

ghost commented 4 years ago

Should/could the actions tested here above also work on an Andriod-Q device?

From my tests proot'ed Termux appears to work without issues on Android 10. There maybe problems with things like Termux:API but they will be solved later.

Performance is slower under proot by 30-40 % (mostly affects only I/O operations).

You will not see differences in usage experience, binaries can be executed in $HOME in same way as pre-Q Android versions.

Would that be possible without the proot process (only t84 APK with 29)?

Nothing. Proot acts as executable loader.

All *.exe are in$HOME like in t82 - isn't that a problem?

Not a problem while proot is being used.

What can be the reason that sftp does not work anymore after t84 (ssh does it)?

I don't know. This needs testing. But note that proot may not be stable on some devices.

Is there anything to improve the performance of proot processes (as in t82)?

No. Proot uses ptrace() to hook system calls and manipulate their arguments.

You should not execute termux-chroot under proot'ed Termux.

RalfWerner commented 4 years ago

Pleased I discovered @twaik back in dev dialog with fresh ideas. Thank you @xeffyr for the explanations (above) I continue here the test. to:

Performance is slower under proot by 30-40% (mostly affects only I/O operations).

I understood your answer as saying that proot process libproot.so (r='pgrep -n libproot') is already active in your t84, even though ~ is not /home but t=/data/data/com.termux/files; $t/home, while at t82 the proot process has to be created with termux-chroot in order to have comparable proot tests. In addition, the Q-restriction only is bypassed with ptrace() and *.exe must be saved as before in $t/* (sd/ does not works) - as far as right and why $HOME is different in the two versions?

With 30-40% loss, I would have no problem. Now I have the command: date; du -hd 1 u/; date without/with proot tested (seconds) and got: t82=7/20 and t84=25/374 '. With respect to 7, 40% would actually be 7*1.4=10 seconds. My unacceptable test above result with check.c above corresponds to 7/374 although there is hardly any I/O except for make. Now you said:

You should not execute termux-chroot under proot'ed termux.

Remains a loss factor of 2 to 3 compared to 7 sec. I have described this as justifiable but have not made any more extensive tests and you said the plugins could be a problem. I want to do that below.

Navigator, plugins: widget, API and two X11 (Xwayland)

All scripts are included in .bashrc sd/ip/s (134 sloc expanded by $r administration). A light version is api29/t (19 sloc reduced to OTS and one XClient). t=bash .bashrc $@ is used to start all PID4 processes. The processes on my t84 device are unstable (SSM only here), but they were already before in contrast to the t82 device (currently working for 11 days and PID3 since 2 days).

ssh seems only possible if $r and sshd are found. Unfortunately, the pstree function ends with:pstree: error opening uptime file (why?) in both versions, but it is also recognizable with pp that all $r instances with PPID refer to the first one or to PID1 (com .termux see shots above)

The pstree problem above seems to be more complicated. by replacement with cd u/bin; p=pstree; mv $p -$p; In -s busybox $p is the function again available (error in coreutil?), but does not do the desired - or maybe it does. Originally, the entire Termux process should be displayed as a tree. Due to various sevices and libproot this is no longer possible and also alias k = kill $PPID from .bashrc is no longer possible in t84 - option -9 is required. Can the libproot after PID1 be started one time and PID2 (also sshd and Xvnc) again as a process? and solve the _kwown bugs (above)? This would greatly simplify the work, especially if a restart is required (alias k only). Currently several processes have to be stopped at Android level to be successful.

Are you interested in further tests?

ghost commented 4 years ago

Pleased I discovered @twaik back in dev dialog with fresh ideas.

Many ideas that's great. There only one problem - who will implement & maintain that ? So far I haven't seen any working proof-of-concept.

I'd rather go with lower performance (qemu-system or proot) than with regressions (which may take ages to be solved) bringing Termux back to its early stages.

Unfortunately, the pstree function ends with:pstree: error opening uptime file (why?)

That's SELinux. Since Android SDK does not mention /proc as "official" interface, manufacturers are allowed to restrict its access as much as possible (+ for security as /proc has potential for leaking data).

The pstree problem above seems to be more complicated. by replacement with cd u /; p = pstree; mv $ p - $ p; In -s busybox $ p is the function again available (error in coreutil?)

Not an error. Just different implementation.

Due to various sevices and libproot this is no longer possible and also alias k = kill $PPID from .bashrc is no longer possible in t84 - option -9 is required.

libproot.so is the $PPID now. Proot can't be safely terminated as immediately destroys all child processes.

RalfWerner commented 4 years ago

Thanks @xeffyr for your answer, a few questions are still open.

I'd rather go with lower performance (qemu system or proot) than with regression (which may take ages to be solved) bringing Termux back to its early stages.

API loss factor of 6 times of t82 is a bit too much - can this be improved? I agree with you to avoid regression. So pstree in busybox is a working alternative (like ps before). If @twaik had a working test environment in a manageable time with his ideas, I would like to test them as well.

libproot.so is the $PPID now.

Unfortunately in my test t84 often came in a mode that only allows restart termux and services. So I've included kill -9 $PPID in thealias k. Widget is no longer usable at t84/proot2 (above)! Could you find out why Widget/bash behaves differently on t82 and t84 and if this can be fixed?

ghost commented 4 years ago

can this be improved?

No. Proot intercepts all system calls and changes their arguments, that is core functionality.

Widget is no longer usable at t84/proot2 (above)!

My APK is just example with using proot to launch shell. I'm not aware about any compatibility with plugins.

I maybe will submit newer version soon, but it will contain other changes needed for Android 10 including dropped shared storage support.

RalfWerner commented 4 years ago

I'm not aware about any compatibility with plugins

Now you are: API works slowly, X11 as before (same bugs) and Widget does not work as expected.

I maybe will submit newer version soon

That's good - please let me know when the time comes? Once again to one widget (.shortcut): .shortcuts/ls-env-ps. in t82 execute without a new bash the included script: ls -la; env; ps -eo pid, ppid, comm, etime; pstree; exit 1 with a new navigator item, like also if the script is executable in both versions. t84 starts a bash (11745) - without script and generates and holds also a new navigator item. The following processes are active:

31767[~]$ .shortcuts/ls-env-ps
...
XDG_RUNTIME_DIR=u/tmp
SSH_CLIENT=192.168.0.114 64919 8022
PATH=./:/data/data/com.termux/files/home/:./:/data/data/com.termux/files/home/:/data/data/com.termux/files/usr/bin:/data/data/com.termux/files/usr/bin/applets
SSH_TTY=/dev/pts/1
_=/data/data/com.termux/files/usr/bin/env
PID   PPID  COMMAND          ELAPSED    PID   PPID  COMMAND          ELAPSED
 2171 32503 bash              1h41       2171 32503 bash              1h54
 2173  2171 bash              1h41       2173  2171 bash              1h54
 2219  2173 pcmanfm           1h41       2219  2173 pcmanfm           1h54
 2229  2173 aterm             1h41       2229  2173 aterm             1h54
 2236  2229 bash              1h41       2236  2229 bash              1h54
 5376 32503 bash              1h02       5376 32503 bash              1h15
 5377  5376 bash              1h02       5377  5376 bash              1h15
 5404  5377 aterm             1h02       5404  5377 aterm             1h15
 5405  5404 bash              1h02       5405  5404 bash              1h15
 5874 32503 openbox          58:40       5874 32503 openbox           1h11
11442 32503 bash              0:02      11745 31653 libproot.so       2:16
11445 11442 ps                0:01      11751 11745 bash              2:16
31653   529 com.termux        1h44      12066 32503 ps                0:01
31767 31653 libproot.so       1h43      31653   529 com.termux        1h56
31773 31767 bash              1h43      31767 31653 libproot.so       1h56
31842     1 sshd              1h43      31773 31767 bash              1h56
31970     1 Xvnc              1h43      31842     1 sshd              1h56
32158 31773 Xwayland          1h43      31970     1 Xvnc              1h56
32501 31842 sshd              1h43      32158 31773 Xwayland          1h56
32503 32501 bash              1h43      32501 31842 sshd              1h56
                                        32503 32501 bash              1h56
?-+-Xvnc
  `-sshd---sshd---bash-+-bash---bash-+-aterm---bash
                       |             `-pcmanfm-+-{gmain}
                       |                       `-{pcmanfm}
                       |-bash---bash---aterm---bash
                       |-bash---pstree
                       `-openbox
31767[~]$ cat .shortcuts/ls-env-ps
ls -la;env;ps -eo pid,ppid,comm,etime;pstree;exit 1
31767[~]$ pp

--- before 31767[~]$ pp I have started the Widget from start screen! (2nd column) --- Can I do something to start the script here? except manually

Addendum to (s)ftp: The ending of the sftp session (Connection closed) does not take place with only-ftp (tcpsvd). However, if process 31767 (above) is ended, a connection can no longer be established. sshd (31842) and tcpsvd are then "zombie processes" and must also be killed and restarted so that a connection is possible again.

RalfWerner commented 4 years ago

I did a few tests with Ubuntu to possibly fix / work around some of the Xserver bugs. I first installed the packages xterm and nedit there. With the DISPLAY of my PC (Xserver) these also worked fine. After the installation I start Ubuntu (root) with the alias uu, on:

What should the command uu in t84 and its parameters look like here and how could I use the X11/server (PID2+3=Xwayland) there (proot2)?

ghost commented 4 years ago

This should probably be not done, since I am already in proot

This should be done to call proot with different parameters. Proot in my APK is not meant for just execve workaround but for changing file system layout (pending for next prooted Termux version) so we will be able to make Termux prefix-independent and also workaround some filesystem problems in Android 10.

What should the command uu in t84 and its parameters look like here

Normally parameters should be same. Regarding nested proot - nothing can be done as reusing existing proot process is not possible.

and how could I use the X11/server (PID2+3=Xwayland) there (proot2)?

Probably $PREFIX/tmp should be mounted (with -b) to /tmp.

RalfWerner commented 4 years ago

Thank you @xeffyr - I still have a few questions(?) to be successful.

In order to use my make loops in uu (see below), several development packages have to be installed. That takes about 800Mb in u*-fs and script s (OTS in preparation) is adapted to this location (root).

For t82, I was able to start the make loop t :0 c (shot4 in the first commit above), but do not get any output in PID2 and *.exe and aborted at dpy=XOpenDisplay(0). I suspect an Xwayland problem (could not connect .. below). pp shows the processes in termux. The PID2 process (4037) is only visible with wtermux with my-x11.apk the service in termux is not visible and cannot be started from ubuntu - or do you've an idea?

u=uu/uu.sh; echo '#!/data/data/com.termux/files/usr/bin/bash
cd $(dirname $0); unset LD_PRELOAD
command="proot --link2symlink -0 -r ubuntu-fs -b $PREFIX/tmp:/tmp -b /dev -b /proc -b /sdcard -w /root /usr/bin/env -i HOME=/root PATH=/usr/local/sbin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin "
command+="TERM=xterm LANG=C.UTF-8 /bin/bash --login"
if [ -z "$1" ];then exec $command; else $command -c $@; fi'>$u; chmod +x $u; alias uu=$u
[~]$ uu
...
moin IP 192.168.0.132, Xwayland:2002, Xvnc:13094
root[~]# t :0 c
make: '/root/TMP/check.exe' is up to date.
:0 (c) /root/TMP/check.exe P*pm s *.c pet1*m :182834.400
E=> Display not opened! STOP
root[~]# Xwayland :0&
[1] 3984
root[~]# could not connect to wayland server
(EE) Fatal server error:
(EE) Couldn't add screen

The two versions (termux, ubuntu) are processed in parallel. The processes are common, but for me they are only visible in termux ps/pp (ps and psmisc package are missing).

[~]$ pp
PID   PPID  COMMAND          ELAPSED
 2000     1 bash              2d02
 2002  2000 Xwayland          2d02
 3888 29085 sshd             10:19
 3889 29085 sshd             10:16
 3892  3888 bash             10:15
 3924  3892 proot             8:01
 3927  3924 bash              8:01
 4002 29085 sshd              0:19
 4007 29085 sshd              0:10
 4008  4002 bash              0:10
 4037   322 .termux.wtermux   0:06
 4059  4008 ps                0:01
13094     1 Xvnc              1d01
24407  9231 bash              9h21
29085     1 sshd              2d05

All packages (u/bin and u/lib) must be installed twice. Example:

[~]$ ll u/bin/Xwayland
-rwx------ 1 u0_a116 u0_a116 1022380 Nov 18 08:55 u/bin/Xwayland
-rwx------. 1 root root 1329300 Apr  3  2019 u/bin/Xwayland

If I also want to try ap-get install mpv (with mpv-x -vo x11 is not possible), additional 670Mb is needed - but I was unsuccessful (libnss3 404 Not Found) - cannot check.

nested proot -nothing can be done as reusing existing proot process is not possible.

I had already tested and excluded the proot start from t84 above - or mean you something else? To decouple the u*-fs from the termux installation I want to use-r $HOME/sd/uu/ubuntu-fs -b $HOME/sd:/sdcard - is that a problem?

20258[~]$ bash sd/uu/uu.sh
proot error: execve("/usr/bin/env"): Permission denied
proot info: possible causes:
  * the program is a script but its interpreter (eg. /bin/sh) was not found;
  * the program is an ELF but its interpreter (eg. ld-linux.so) was not found;
  * the program is a foreign binary but qemu was not specified;
  * qemu does not work correctly (if specified);
  * the loader was not found or doesn't work.
fatal error: see `proot --help`.
proot error: can't chmod '/data/data/com.termux/files/usr/tmp/proot-19838-VykanY': No such file or directory

All packages must be installed twice (package update pu) if proot is done with uu*). All attempts with ln -s sd/uu uu failed so I did with rm uu; mv sd/uu . continued - unfortunately also unsuccessful. after that the cell phone condition was so unstable that only a reboot helped. uu then has the same termination as above - so with cd uu; rm -r ubuntu-fs; bash ubuntu.sh tested (decompressing ubuntu image). The path /sdcard is unknown in proot2 so I did this with: -b /storage/6533-6333:/sdcard and was able to successfully log in with uu - hallelujah!

*) abbreviation uu=ubuntu (double u) another possibility would be triple u (so many u's contain this linux variant) is used as alias and path, because this can be easily entered on the on-screen KB.

RalfWerner commented 4 years ago

@xeffyr test for termux/ubuntu on aarch64 (t84 with proot2):

The t84 behavior is unacceptable and I gave up starting an ubuntu Xserver there. With t82 X11 server does not work and I could not use the Xvnc server (PID3 = 13094 from OTS with tigervnc in termux) in ubuntu. If I kill the process before uu, four new processes will be created with vncserver, but they will be killed when RealVNC starts - You can see why? Processes and log:

[~]$ pp
PID   PPID  COMMAND          ELAPSED
13094     1 Xvnc              3d21
[~]$ pp
16393     1 Xtigervnc         1:16
16394     1 vncserver         1:16
16395 16394 Xvnc-session      1:16
16396 16395 vncconfig         1:16

root[~]# cat .vnc/localhost\:1.log
Xvnc TigerVNC 1.9.0 - built Dec  1 2018 21:51:29
Copyright (C) 1999-2018 TigerVNC Team and many others (see README.rst)
See http://www.tigervnc.org for information on TigerVNC.
Underlying X server release 12003000, The X.Org Foundation

Sun Jan 12 14:42:52 2020
 vncext:      VNC extension running!
 vncext:      Listening for VNC connections on local interface(s), port 5901
 vncext:      created VNC server for screen 0

Sun Jan 12 15:01:02 2020
 Connections: accepted: 127.0.0.1::48058
 SConnection: Client needs protocol version 3.8
 SConnection: Client requests security type VncAuth(2)
terminate called after throwing an instance of 'rdr::Exception'
terminate called recursively
(EE) Backtrace:
(EE) Fatal server error:
(EE) Caught signal 6 (Aborted). Server aborting
XIO:  fatal IO error 11 (Resource temporarily unavailable) on X server ":1"
      after 173 requests (173 known processed) with 0 events remaining.
Killing Xtigervnc process ID 16393... which was already dead
Cleaning stale pidfile '/root/.vnc/localhost:1.pid'!
RalfWerner commented 4 years ago

Xvnc in proot/ubuntu

@xeffyr Your log handles pu in ubuntu and PID3=vncserver from OTS. Not recognizable are the four processes there (16393 ff above). Here at Xvnc (:1) PID2=RealVNC-App can be called before or after PID3 start (only if it does not yet exist). Since it already exists through PID1 (termux app start with OTS), I get a Open-Display-Error at PID4=check.exe in ubuntu without Xvnc the four processes created and with t c PID4 start normaly but with "tab on app" (PID2) I get the XIO: fatal IO error at the end (all four processes killed), as you get too and how to use PID3=Xwayland (:0) what is to do?

t64.apk instead of t84.apk

I gave up the test of your proot2.apk due to serious defects (no widget and loss factor of 80 times at proot) and replace t84 by your apk above (my-t64 with apksigner) and describe your suggestions in README. As the first steps I did tu=t.tgz update and pu=apt-get ... (ubuntu). Pleasing: Widget and sftp work again and pu is ready after 5 minutes (960Mb). Unfortunately, your links are replaced after tu. The $NL/arm64/*| grep -c lib_ 76 files and ...|grep -c bin_ 149 files are still there (38Mb) and you donated a "brilliant" script for repetition - will experiment a little more. NL=$(dirname $(pm path com.termux|cut -d: -f2))/lib works well for t64 but how to do it for t32?

tu and symlinks in sd paths

The easiest thing would be to completely save the termux/ubuntu data in sd (there I have space for t64 35Gb). The first hurdle is the symlinks in sd (which is not a problem from termux), which you said "could do" - how? currently I have 1.8/1.9Gb (t82/t64) and without ubuntu the last sd t.tzg is 356/309Mb and now 684Mb. This makes the tu (necessary for every vt) and also t tgz t slow and I am thinking about what can be disposed of beforehand - how do you do that?

ghost commented 4 years ago

how to use PID3=Xwayland (:0) what is to do?

Either to launch it in TCP mode or bind-mount (with proot) $PREFIX/tmp directory to /tmp.

NL=$(dirname $(pm path com.termux|cut -d: -f2))/lib works well for t64 but how to do it for t32?

If I correctly understand, t32 is Termux for 32 bit ARM. I currently have APK only supporting AArch64 and i686.

The first hurdle is the symlinks in sd (which is not a problem from termux), which you said "could do"

Which context you are referring to ? I didn't suggested to put environment on /sdcard.

RalfWerner commented 4 years ago

Either to launch it in TCP mode or bind-mount (with proot) $PREFIX/tmp directory to /tmp.

Done! (find backwards $PREFIX/tmp in this issue and checked with ll /tmp. With Xvnc you also had no success (log). Have you been able to start one of the two Xservers (preferably both) on your device?

Unfortunately, my third Xserver (XSDL) now works on neither of my two devices after the typical "6-second-init" shows the following message (both devices)): grafik This is still the case even after "reboot without termux". Trying de/re-install the app is only possible on t82 and on t64 it is no longer possible to "reinstall" (google-play) did you have that too? I did this with nedit -display ... (shot2) and it works well on my PC (with the Windows Xserver). The functions ps and ip (wrapper on /system/bin/ip) are still missing on uu. with cp u/bin/ip uu/ubuntu-fs/bin (termux),-b system (uu.sh) and #!/bin/sh (in /bin/ip) I get /bin/ip: 3: exec: /system/bin/ip: not found - what is wrong @xeffyr? - sorry forgot to restart uu but ps still missing!

I currently have APK only supporting AArch64 ... Which context you are referring to (sd symlinks)?

t64 is sufficient for check now! ... Unfortunately I couldn't find the reference anymore - no matter if you say "it never works" I believe you. I am interested in storing data in sd in order to be independent from the de/re-installation of termux as with t.tgz - to "disposed of beforehand" you've said nothing!?

ghost commented 4 years ago

Have you been able to start one of the two Xservers (preferably both) on your device?

XSDL works fine. No special setup needed, did export DISPLAY=:0 and it worked.

Using GIMP in Termux/Ubuntu: Screenshot_20200115-142024_XServer XSDL

However there are SHM-related errors which cause certain issues. Example:

shmget() failed: Function not implemented
Disabling shared memory tile transportgimp_get_monitor_resolution(): GDK returned bogus values for the monitor resolution, using 96 dpi instead.

Proot does not implement system calls like shmget and makes them no-op. This does not affect programs from Termux environment as here SHM functionality is implemented through libandroid-shmem.so & Android's ashmem.

Trying de/re-install the app is only possible on t82 and on t64 it is no longer possible to "reinstall" (google-play) did you have that too?

No. I have reinstalled Termux multiple times recently to switch between versions, no issues regarding this observed.

Of course all apps & addons signed by one key.

what is wrong @xeffyr? - sorry forgot to restart uu but ps still missing!

-b system is wrong and will lead to following issue

proot warning: can't sanitize binding "system": No such file or directory

Use -b /system instead.

I am interested in storing data in sd in order to be independent from the de/re-installation of termux as with t.tgz - to "disposed of beforehand" you've said nothing!?

/sdcard support will be dropped for API 29 and higher as Android no longer provides access at file system level. Termux will not support to have stuff on /sdcard for this reason and few others.

Binary execution from sdcard is possible despite noexec mount option, just proot built from the branch https://github.com/termux/proot/tree/force-exec is needed.

Storing of symlinks and other special files (directly) is not possible due to file system implementation limits.

RalfWerner commented 4 years ago

XSDL works fine (... been able to start one of the two Xservers ...)

@xeffyr Thank you first! I meant Xwayland/X11 and Xvnc! XSDL is only my 3rd choice and X11 is the 1st - you apparently haven't had any success with that yet - have you? With a workaround (download/installation from an external url) I now have XSDL on t64. The problem was my "play store app" (not your beer) but - surprise - the app works again overnight. Fdroid, no longer offers XSDL but termux (84). Currently, however, I only use my self-signed apk (t64).

/sdcard support will be dropped for API29 ... But this is possible if device is rooted

With sd=/sdcard; s1=/storage/6533-6333; s2=/storage/emulated/0 I have three possible paths that start with [ ! -d $sd ]&& sd=$s1; [ ! -d $sd ]&& sd=$s2 of the device environment. Then in $HOME e.g. set up symlinks with ln -s $sd/api29 i and ln -s i/t .bashrc, so that termux and ubuntu use the same scripts but have different /home paths. This won't work with API29 - correct? Even saving/resetting (t.tgz - tu) to $sd is no longer possible and with every termux de/re-install the complete function set must be installed with pu (very time-consuming). I installed mpv in uu again (one hour) and was a little more successful than above (find mpv backwards here). The function shows the video (-vo), but gives no sound (-ao reversed as with termux), many error messages and just only possible on XSDL. The app is then 3.15Gb and $s2 is almost full - unacceptable! You say: if device is rooted all problems should be solved. How would I do that and is the cell phone still usable apart from temux?

Use -b /system instead.

Typo -b system was only in this issue (correct in the script) what was missing was: crtl-d plus uu (restart to activate). then I added -b /data and NL=$NL to uu.sh, moved it to */i/uu.sh, usealias uu="bash i/uu.sh"and preparing some widgets /.shortcuts with nedit -display ... (shots in preparation) and also on my PC and mpv wrapper. I completely removed the functions ps and ip in uu and added /system/bin in PATH. This means that both functions (for process and display control) are correct. Then I am currently working on your ELF-analyze-loop (/bin, /lib) in i/uu.sh, so that I can also analyze actual packages and my make loop. Are you interested in the result?

Addendum to a ELF-check (from you files / size) by:

u/bin and u/lib(exec) for my current 3.15Gb app come from termux (385+498+47 / 68M+313M+96M) and ubuntu (588+2208 / 91M+873M), which will make your t64 (238 / 38Mb) have to be added to $NL!

ghost commented 4 years ago

had any success with that yet - have you?

I don't use Xwayland. May try with Xvnc though.

This won't work with API29 - correct?

You specified these paths: sd=/sdcard; s1=/storage/6533-6333; s2=/storage/emulated/0:

Both /storage/emulated/0 and /storage/6533-6333 do not allow file execution. Additionally, API 29 disallows execution from /data/data/*.

Should I mention that you can't put environment to these locations ? Perhaps should be obviously that it will not work (it will be even broken due to nonexistent symlinks).

If you want to put environment on /sdcard or its equivalent, you can do this only through EXT4 file system image. Example command sequence for ADB root shell:

setenforce 0
dd if=/dev/zero of=/sdcard/data.img bs=1M count=2048
mke2fs -t ext4 /sdcard/data.img
mkdir /mnt/termux-env
mount /sdcard/data.img /mnt/termux-env
cp -a /data/data/com.termux/files/* /mnt/termux-env/
rm -rf /data/data/com.termux/files/*
ln -s /mnt/termux-env/home /data/data/com.termux/files/home
ln -s /mnt/termux-env/usr /data/data/com.termux/files/usr

The app is then 3.15Gb and $s2 is almost full - unacceptable!

New Termux with API 29 fix will take 2x more space (size of: apk + extracted data to /data/app/com.termux-*/lib).

if device is rooted all problems should be solved.

Rooting by itself doesn't solve problem. Additional actions will be required.

You will be able to turn off SELinux restrictions with setenforce 0 and be able to execute stuff in /data/data/com.termux/files/home/*.

How would I do that and is the cell phone still usable apart from temux?

See commands for ADB above.

RalfWerner commented 4 years ago

/sdcard - a compatibility symlink to shared storage ... do not allow file execution

only on t82 and not your t64 (my Huawei device) - that's why I check the existence. /sdcard is here a symlink on /storage/sdcard1 since 1970 - which doesn't exist but $s1 and $s2 exist. Have I misunderstood that data access (ln -s $sd...) is also possible in API29? No execution has always been this way and I have aligned my processes accordingly: . .bashrc is passed only sourced and i/uu.sh as argument from u/bin/bash. If this is still the case, I am reassured :) What changes is then only that u/bin/bash is a link to $NL/*bash.so (/data/app...) that you created on Aug 18 11:54. Do I understand it correctly now?

New Termux with API 29 fix will take 2x more space

see Addendum to a ELF-check above! The APK should be at least 1441M in size - right? From my make-loop it is only one file with 110K, which is saved with make in s=~/check.exe. With ubuntu then 2x. In the first comit (shot4) the X11/Xvnc display is included. Editing the code there or with QuickEdit (shot2) - requires saving to $sd. If you click on the 2nd icon (blue/yellow barrel) in the check-Xclient, make $s; $s done. That means after 2-0.5 seconds the screen appears again with updated code and without change after 0.2 seconds. The replacement of $s=$NL/lib_check_.so (if I had write permission) would also work. How can I achieve this?

Example command sequence for ADB root shell

Thanks @xeffyr for the description I will test it later with a smaller app. With the exception of setenforce, the process could also run under termux. however, pkg install e2fsprogs would have to be done to have mke2fs, but pkg no longer works with API29, since $NL/libbin_mke2fs.so cannot be generated. How do you free yourself from the dilemma - maybe pm ... - and what about t32?

I don't use Xwayland. May try with Xvnc though.

Pity - our XSDL shots above (Xclients: nedit/gimp) are clearly visible on a 4k screen in github. It gets more difficult on my cell phone and when I try screen tab, rotation or use OnScreen-KB, the horror begins because XSDL doesn't support any of it properly. With Xvnc, unfortunately, it doesn't get much better. A smart-termux on the smart-phone is currently only possible with X11/Xwayland. you tell me if you were successful with Xvnc in proot/ubuntu?

Leonid: Try to imagine that you've only your cell phone (no PC, mouse, KB, glasses) and your fingers and would like to support and develop termux. How should termux work if you've only this tool as an app

ghost commented 4 years ago

Have I misunderstood that data access (ln -s $sd...) is also possible in API29?

No, not possible unless you will use private app directory on such storage:

$sd/Android/data/com.termux

What changes is then only that ubin / bash is a link to $NL/*bash.so (/data/app...) that you created on Aug 18 11:54. Do I understand it correctly now?

Generally yes, but we are going to put everything (not just binaries) to $NL and then setup symlinks in $PREFIX.

The APK should be at least 1441M in size - right?

Assume that APK of perl takes 22 MB:

~/perl-app/app/build/outputs/apk/debug$ du -hs app-debug.apk 
22M app-debug.apk

Size of extracted jni lib directory takes 65 MB:

~/perl-app/app/src/main/jniLibs$ du -hs arm64-v8a/
65M arm64-v8a/

What we have when package is installed:

Reality can be a bit worse: Screenshot_20200117-170334_Settings

Example with perl package is not perfect since it contains a lot of text files which are compressed well inside APK.

How can I achieve this?

You can execute your check under proot.

In discussion at https://gitter.im/termux/dev was suggestion to reimplement termux-exec (https://github.com/termux/termux-exec) to prepend proot to command line of any userspace executable at time when execve() is invoked.

however, pkg install e2fsprogs would have to be done to have mke2fs

No need. My example was for ADB shell. All devices have at least make_ext4fs which can be used too. Some have mke2fs too in /system/bin.

Usage for /system/bin/make_ext4fs:

make_ext4fs -l 1G /sdcard/test.img /path/to/fs/root/dir

but pkg no longer works with API29

In API 29 pkg will become a Termux custom package manager. APT will not be used anymore but its repositories will continue exist.

and what about t32?

t32: https://linx.li/t32-termux-api29.apk

Note that proot-based Termux apk and examples I have posted on Gitter are discontinued -- Fornwall created https://github.com/termux/termux-app/pull/1427 where implementation is better than mine and close to one that will be used. But bootstraps here are not API-29 compatible currently.

APK for PR 1427: https://linx.li/termux-pull1427-debug.apk

you tell me if you were successful with Xvnc in proot/ubuntu?

Xvnc was fixed with

vncserver -SecurityTypes None :1

Screenshot_20200117-180552_VNC Viewer

RalfWerner commented 4 years ago

Emulation of Q10 (Studio/BlueStacks)

worked after various de/reinstallations and "windows features" (10Gb) and BlueStacks (api25 with 5.6Gb) after update. A test is therefore only possible in Studio, as there is no Q10 update planned for my real devices. The following shot shows the different sources with t32 from @xeffyr: grafik So the APK contains next to armeabi-v7a (32bit) also lib from x86 (used here) and arm64-v8a (should be identical to t64 - do not consider further). In all emulations shared storage with Windows is not possible. Unfortunately there is no proot in the APK and only the classic pkg (replaces the symlinks in nl) - you have/plan no new APK with it (below)! like ln -s ../usr u I created~/nl (native libary) as symlink - here with:

[~]$ aapt l i/x/my-t32.apk |grep bin_am
lib/arm64-v8a/libbin_am.so
lib/armeabi-v7a/libbin_am.so
lib/x86/libbin_am.so
[~]$ a=`ls -l u/bin/am`;[ ! -z "$a" ]&& ln -s `$e ${a##*>}|sed "s@${a##*/}@@"` nl
[~]$ ll nl/*bin_am.so u/bin/am
-rwxr-xr-x 1 system  system  208 Aug 18  2019 nl/libbin_am.so
lrwxrwxrwx 1 u0_a134 u0_a134  70 Feb 18 12:02 u/bin/am -> '/data/app/com.termux-BmlXcv6HZ6xjT3cbcxO4ug==/lib/x86/libbin_am.so'
 same on BlueStacks
-rwxr-xr-x 1 system system 208 Aug 18  2019 nl/libbin_am.so
lrwxrwxrwx 1 u0_a61 u0_a61  43 Feb 13 18:02 u/bin/am -> /data/app/com.termux-1/lib/x86/libbin_am.so
 and on Huawei y7 (with *t64.tgz - without "am")
 NL=$(dirname $(pm path com.termux|cut -d: -f2))/lib/*/;ll $NL/*bin_bash.so
-rwxr-xr-x 1 system system 731544 Aug 18  2019 '/data/app/com.termux-gzboyfYIFTpUCAdfJ-k5LA==/lib/arm64/libbin_bash.so'

Manage storage at BlueStacks emulation with t32

The file app is called Media Manager and differentiates between *.apk (to be installed) and Import from Windows. Termux does this with ~/sd/DCIM/SharedFolder. I discovered no sdcard - for that $sd is generously equipped with 63 Gb. Since Q10 is not possible, there are no restrictions on data and functions. The X11 screen shown in the shot above is the OTS result on ip object and in termux I've created tu.tgz for use in studio. All actions are faster than on my real devices. I didn't do any further tests (widget, API, ubuntu / proot) - no relevance for api29. The lib (u/bin, u/lib* and nl) should be identical before pkg (100% symlinks) and can then be used in BlueStacks as on my devices. tu.tgz for backup If reinstall is required, could be created in /storage/emulated/0 so that it will not be disposed of with the termux deinstallation.

Manage storage in Studio-Q10 emulation with t32 and t73

I've little experience with Studio emulation/APK. Tested first *t73.apk (old school with net-traffic-bootstrap) after drag/drop in the emulation (pixel5x). This attempt failed due to insufficient memory (tu.tgz update). There is also the way to start termux without installation from Studio - so could several termux versions (projects) be tested in parallel. Have create therefore an additional device (pixel4 for t73) and enlarged in config.ini path: C:\*\*\.andoid\*\*.avd - the path cannot be changed) by 1 GB each. So Studio needs now 17Gb and is very slow compared with real devices/BlueStacks.

Example use uu.sh, bashrc and tu.tgz in termux: The sdcard in the file-app contains the same folders as in ~/storage except~/sa. However, data from downloads can be copied there via "SDCARD-Android-data-com.termux-files" (ln -s $sa$a sa in uu.sh). The file names are changed (tu.tgz -> tu (1).tgz). These can be changed back in t32 with mv.

The whole process worked only once and after restarting Studio neither Termux nor the file-app could access sdcard and I couldn't find a repair setup. So have uninstalled the 17Gb of the two devices from Studio and installed again one device (only t32) in order to then need 20Gb for it - after 5 hours.

Access (termux-setup-storage) must be allowed allways. The path with the symlinks is superfluous because it can also be reached with $sd=~/sd and at t32 it is not accessible except the additional external-1 (~/sa above). With t73 I usually have the behavior that is familiar from my real devices. However, the functions for transferring to the emulation differ. With drag & drop everything is saved in Download except *.apk (will be installed). This path is accessible from t73 and the google-file-app but not from t32. The file app is surprisingly also familiar with Termux's $HOME (no api29 restrictions apply to google-apps), but does not contain any function to move the downloaded data there, except share single files. During the attempt, the data is copied to ~/downloads and a Termux instance (bash) is opened, from which the files can copied back to $HOME with mv * ...

ghost commented 4 years ago

do you have/plan a APK with it?

I do not plan any new apks. We already know all properties of Android 10 we need. No more tests from me until some implementation will be already done.

https://github.com/termux/termux-app/pull/1427 - is the current progress.

RalfWerner commented 4 years ago

I do not plan any new apks. We already know all properties of Android 10 we need

conlusion of api29 test (above)

Since with t32 no proot function is available in Android Q/10 and obviously you haven't had any success either, I cannot test this important function for ubuntu and make-loops. The storage restrictions have been confirmed for non-google-apps. These must be used to protect themselves against loss and are very complex and time-consuming.

With api28, objects can be easily installed and exchanged, even if they contain complex tasks such as OTS - api29: is the end of this game! If you @xeffyr should create an apk with proot please let me know! I will continue my research at ubuntu/Xserver (above) and widget.

Addendum to ~/sd at api29/30 proot and make-loop.

In uu.sh is an object finder that is looking for a start script (default: ip/bashrc). It is also possible to search for *.tgz or a path if it contains bashrc. api29+Q10 contains: */Android/data/com.termux/files as the only valid paths that are connected to the world outside termux. In the google-file-app: Android SDK build for x86-* or SDCARD), if it exists. ) Studio warning: in the file-app the sdcard-drop-button deletes everything that was there immediately (without confirm). For the repair all configuration files have to be deleted from C:\**\*.Avd, which corresponds to a reset of the mobile phone. Unfortunately, Studio consume a lot of memory - currently a total of 36Gb. Shots and scripts to R11 emulator and proot/make-loop: grafik

[~]$ ln -s ../usr u; p="proot -b /storage -b /apex -b /system -b /data -b /dev -b /proc -b u:/usr -b u/bin:/bin -b u/etc:/etc -b u/lib:/lib -b u/share:/share -b u/var:/var -b u/tmp:/tmp -r .."
[~]$ unset LD_PRELOAD; $p # should start proot and prompt should be rt[~]$
rt[~]$ printf '%s\n' '#include <stdio.h>' "int main(){char c=getchar();return c-'0';}">s.c; i=1; while [ $i != 0 ];do cc s.c;echo stop with 0:;a.out; i=$?; done ; rm s.c a.out

In dev and here is the discussion with @fornwall regarding his *29.apk. I've update studio to 3.6. Effect: the emulator was deleted without confirm inclusive everything I checked there (I'd moved the 10Gb to D:\ because I've more space there). 3.6 allowed also R11 emulators that work without studio (shortcut-icons on my PC-Screen (1st shot) and start R11 (2nd shot) with check of environment), but must be saved in C:\. Both now: 19.6Gb and R11 has a different behavior regarding storage. Q10 applies roughly: google (file app) can do almost anything and termux almost nothing. With R11: google allow write to $HOME without confirming in Termux inclusive read ~/nl and Termux allow nothing (setup-storage without effect) outside of ../ (no sdcard support). With both emulations and `p29.apk` no proot processes are possible but cut/paste into the emulation: For simple check I've put The first two bash-lines start proot (on my real device) and the 3rd line creates and executes a simple make-loop - could you @fornwall please check it on your Q10-Huawei?

Last R11 checks: I've now access to internal data (e.g. sd/Download) but not to sdcard (read only except for google-file-app) - so no more restriction to A*/data/com.termux/file but only by phone setup and not from termux-setup-storage (based on: com.termux.app.reload_style).

Working with termux and the google-file-app/drive is difficult! I've created a backup (*u.tgz) for termux and ubuntu (uu.tgz with 264 Mb) in the Q10 emulation with t73.apk and the pu=package update. I tried to transfer this into the R11 emulation with *29.apk and instead of pu I installed this from uu.tgz (same ARCH=i686). The creation/control (alias u=bash uu.sh) with u tu; u tu uu; ls -l sd/*.tgz is simple. The transfer with google-drive finished after about one hour with a lot of manual work and several errors that were communicated via notifications - or not. With u uu ubuntu would it be unpacked on R11 (sd/uu.tgz exists and ../uu does not) however *p29.apk contains proot not, besides many linux-basics also tar, ps ... So in the meantime I had to install *t29.apk (only for tar). Each APK installation ends termux and then I started again with the new associated command set.

RalfWerner commented 4 years ago

Continue research of ubuntu/Xserver

Currently I need about 45Gb on the PC for three Android emulators and ApkProjects. After the actions in the last commit, I was able to use nedit in emulators according to the pu from the Q10 emulator, but have no functions like ps/pp, grep, sed that are not available in *p29.apk and lead to error messages with scripts. Shot: R11 emulator with nedit (shows first lines of uu.sh) with the PC-Xserver grafik

2nd choice Xserver: Xvnc (:1)

The third choice would be XSDL, whose use is limited to the creation of shots (above). The Xvnc server in termux is not identical to the Ubuntu version (uu). Both are installed with tiger*, use different, mutually exclusive processes, but both have the same real viewer (a change must be confirmed there).

With your/@xeffyr uu-shot above the WM=fluxbox+Xterm is started with the Xserver. It shows the problems why Xvnc is not 1st choice: Many XEvents not get through to the Xclient but are treated by the viewer as if you were on a PC with mouse+KB. e.g. a simple rotation of the phone does nothing on a 1900x1200 screen (config). The real size is not used, but Real-Viewer only tries poorly to compensate for the shortage. Compared to X11/Xwayland, the viewer always works in emulate touchpad mode but correct would be direct mode, which almost all apps use.

1st choice Xserver: X11/Xorg/Xwayland (:0)

To use XClients according to the device properties (mobile phone), this Xserver should used and no WM=Windows-Manager. This works in ubuntu and termux! Many thanks to @twaik and @xeffyr for the support here ff. In checks on my real devices, the Xserver (PID1-3) only had to be started once from termux and support all XClients (PID4). Actions are still open for emulated devices.

godsworkers commented 3 years ago

Is it possible to redirect storage location inside git on termux from remote=/data/data/com.termux/files/.git Redirected into: $s/$a with s=/storage/emulated/0;a=Android/data/com.termux/files?

RalfWerner commented 3 years ago

I have checked for Android 11 and /storage here and here. Your g=.git is in the path../ on termux and can be moved (mv) to your target path (corresponds to $s/$a) and replaced with a symlink (ln -s $s/$a/$g ../$g).

RalfWerner commented 3 years ago

How will you write it ex?

What do you mean by that? Which termux version do you use? I'm treating targetSdk>28 here. With 28 $a is mostly not necessary, with 29 sometimes, with 30 almost always and also depending on the device.

RalfWerner commented 3 years ago

The last google play store version is 101 and pkg up (e.g. liblz4) is still possible. A termux version with level 29+ in fdroid is currently not planned but possibly newer versions with level 28 there. So far correct?

@xeffyr I didn't get an answer to this question so I assume Yes, related to wiki! Termux with target level 29+ is then a private build of *.apk - I've done several times. The packages are then in bootstrap-$a.zip or/and other plugins but you are not serviced these.

Without a little background on how Termux is being implemented, it may be hard to understand

You have a few other ideas for people who understand. in between I count myself among them and have bought such a device, but I ask myself, what is the benefit of target 29+, if the developers have given up - what do you think?

ghost commented 3 years ago

I didn't get an answer to this question so I assume Yes, related to wiki!

Yes, Termux will have target SDK level 28 and be published on F-Droid (v0.102+).

what is the benefit of target 29+

Primary benefit is to be able publish Termux updates on Play Store and also to avoid possibility of delisting application as "deprecated" by Google from this installation source.

Benefits of in-APK packaging are:

RalfWerner commented 3 years ago

Benefits of in-APK packaging

Thanks @xeffyr for the feedback. You repeated the benefits because I didn't ask about limits that led to the development being canceled - OK so far - but "in-APK packaging" should be able to work in 28-Termux as well as in 29+. The almost daily updates of the package servers would then have to be done also in apks or only there if not two Termux versions are exist in parallel and all developers adhere to them - is that realistic? My practical experience with this concept is rather frustrating and the most important 29+ limit is unaffected.

RalfWerner commented 3 years ago

Device (Pad) with PiP (Picture-in-Picture) and floating OnScr-KB

grafik Android 10 benefits include in the app/pictures and transparency management. Here is one shot with Termux (mc+KB above) and X11 (check+touch below). The widget table is partially visible at the bottom left - looks like windows but isn't.

If ls env ps is selected, the action:ls -l;env;ps -eo pid,ppid,comm,etime;pstree;exit 1 should run in termux but isn't. Even if several widget actions are selected, nothing happens. If PiP is not used or minimized for Termux, the action will be execute correctly. I use shortcut.next-session=ctrl+n and described this as a popup to ENTER. To start the missing actions in Termux one ctrl+n is necessary so that all selections are displayed in the DRAWER and processed step by step - example. Another PiP problem are the API dialog boxes because they are hidden behind all pictures (not in full-screen)

PiP+Split+navigation functions on the Galaxy Tab S6:

Almost all apps (VNC viewer not) can work in-picture/split. Termux/X11 is allowed with PiP. This includes over the start/task screen and transparency. The pictures behave similarly to Windows (can be moved and scaled) and the apps for this reduce the display size to the picture. Question: does this apply to all Android 10 devices?

Examples: Shot1 Termux+X11 via VNC (landscape with aterm+pcmanfm+openbox) and Shot2 via the start screen (portait with widget). The floating keyboard is used in both apps. grafik Shot1: To use VNC, first minimize the two apps (above) with a middle tap (movable small circle - here above left) and display the keyboard (with extension) via the VNC menu (here Hacker's). Shot2: In the task screen (split left) the PiP table can be seen in an oval (tap on small circle without open pictures) and on the right (split) the apps (settings) with internal PiP function. grafik

RalfWerner commented 3 years ago

Thanks @xeffyr! The googled PiP documentation does not contain any reference to general valid PiP administration (for almost all Apps), but only for a small list of Google apps with a focus on youtube. My four (Q/R) emulations do not allow PiP for Termux & Co. The management of the PiP list takes place on my Galaxy device in the task list. In addition to the three pictures from my Doc, I've added camera and gallery here. grafik This allows many useful possibilities to operate with termux and the other apps and to follow the logs-on-device in parallel. In shot are six apps/tasks and five of which are in the PiP list. Can you do that on your device too? This is what the 5 apps look like when they are positioned on the display (startscreen). grafik

RalfWerner commented 3 years ago

My current idea is to keep the apt/pkg updates in target 29+ versions

to create new bootstrap$a.zip (test without problem) from the backups after pkg in/up, which will be used for building1) the next *.apk generation but with ELF-Split3) and complete symlink table to nl=../usr/nl. That would also be possible at 28 and would only require one symlink for 29+, files.so would then be unnecessary, with an ARCH-split, the APK size could be reduced for several ARCHs and with aapt in bootstrap$a.zip the Version/targetSdk (e.g. 103/29) could be displayed at Termux startup2)- what do you think of that?

1) Is build APK on-device with non-rooted devices possible and which packages besides the above are required in addition? Should proot fail in the future, would this be an alternative to make-loops have you checked something like that?

2) If cat in ln -s ../usr u; head -n 4 u/bin/login replaced by sed -e s%\!%" $v"%, the version ($v ) can be displayed at termux start. E.g. v="103, target=29, min=24, device=28/arm7l" is a possible value here. The last part is simple: v=...;v+=", device=$(getprop ro.build.version.sdk)/$(uname -m)". ... part requires pm and/or path ($nl) to the installed Termux $apk. Unfortunately, they are inconsistent (known/unknown) on different devices but related. If nl=../../lib exists it's easy. If $nl is known, is: apk=$nl/../../base.apk and revers is: nl=$(dirname $apk)/lib/*. I've no idea, if both are missing -You?

3) I got the first 29+ Version from you @xeffyr in Sept. 2019 (with ELF split - flexible and easy to understand).

check with termux version 103 and 104 (29.12.2020)

For now, I'm just looking at B=bootstrap-i686.zip. The two versions should be identical after pkg up and pkg in aapt proot. To check (30.12) I use find u/|sort>1;pkg list-in>1i before and after (with 2/3 instead of 1) the two pkg actions. With wc -l ? I've got 103: 4974->5019->5096 and at 104: 4993->5018->5095 lines. With diff actions I found differences between thetermux-tool version and xxhash (explainable) and created A=104.tgz backup of ../usr (contains symlink u/nl) to get a next generation $B. Here with bash script and SYMLINKS/ELF-split.

$tar tvf 104.tgz|grep usr/nl
lrwxrwxrwx u0_a138/u0_a138       0 2020-12-29 14:50 usr/nl -> /data/data/com.termux/lib
$ tar xf 104.tgz; S=`find usr -type l`;  ls -l $S>S; wc -l S; rm $S
cd usr; [ ! -e nl ]&& mkdir nl; typeset -i i=0 j=1000; echo "ELF check in `pwd` ..."
while read -r f; do i+=1; if grep -q ELF < <(dd if=$f bs=1 count=3 skip=1 2>/dev/null)
   then j+=1; echo "mv $j.so usr/$f;# $f -> nl/$j.so">>../S; mv $f nl/$j.so; fi
done < <(find bin lib* -type f); j-=1000; echo "$j of $i in $2"; cd ..; exit

Then 3731+279 files and 1363 lines in S remain (have to be syntactically converted to SYMLINKS.txt (three of the 1084 symlinks contain $PREFIX by mistake).

RalfWerner commented 3 years ago

Backups and distros.

Termux backup (tu.tgz) includes ../usr ../home. For Debian pkg in proot-distro; proot-distro install debian-buster install debian but without apt/backup function and is saved in u/ instead of ../, but has the advantage that it is maintained by @termux (could extent/moved/zipped to ud.tgz). The process corresponds to u uu for ubuntu with backup (uu.tgz) If fileApp shows termux, you can also use ln -s ../usr/var/lib/proot-distro/installed-rootfs distro. This shows the debian-buster and $HOME is root (/home is empty and not termux)! grafik Here is an example with debian and my ubuntu distro in ../uu with proot -b $HOME:/home ... and with [~] ln -s ../uu/root uh also uh visible in the fileApp but not the symlinks to /home (th) which is a bind (only allows by ubuntu) and */u/X* are the Xserver of the different distros. grafik With [/home] ln -s /data/data/com.termux/files t in debian all termux/ubuntu data known here too.

Create/Restore process of Backups (*.tgz)

The three *.tgz mentioned above contain all the necessary packages and updates (Create after completion of the action with tar czf ...). They can be restored independently/alternatively, whereby a minimal termux installation is necessary. My tip: everything that is possible with termux should also be done with it in order to avoid performance losses and project data should be managed externally as long as possible (-29). With tu.* the installation of termux (init ist required), to get the last state and bootstrap loop creation and ~ if project data are here, can be restored separately. The distros (u*.tgz) have a separate (/root) and a common (/home) home area, which also includes the project data, /tmp and the common 1st choice Xserver.

RalfWerner commented 3 years ago

check of termux with MANAGE_EXTERNAL_STORAGE and targetSDK=30 on 29 and 30 emulators

grafik above 6 shots (with S-Pen modifications). Because of build in windows (*w) the workaround is required, as well as depending on the OS:git clone https://github.com/termux/termux-app has difference size to linux (*l) in 15770 lines and in 92 of 121 files (diff -r *l *w|grep -c \>;diff -rq *w *l|wc -l here 0.108). The 0.112 update increase files: 121 to 196 and size (Kb +30%): 312 to 448. Of these are 27 modified and 47 only (new/old). 28 Checks will done - question to set nl/$nl for version b.112 is open! Here with 0.98 (includes Android 10/MANAGE*STORAGE ) and is a revised version (copy/paste) for the Termux start.

P=./:$PATH; PATH=/system/bin:$P;echo 'PS1="\w:$ "; e=echo; f="\`find u/"; w="|wc -l\`";s=storage; sd=/$s/emulated/0
alias  ll="ls -al" f="$e f/l $f -type f$w/$f -type l$w" s="$e u/nl $f$w/$f/nl$w" d="du -hd 1" pp="ps -eo pid,ppid,comm,etime"
t=com.termux; a=Android/data/$t/files; l=/data/data/$t/lib;if [ ! -h u ];then ln -s $sd sd; ln -s ../usr u ;fi
if [ ! -f ~/u/bin/bash ];then cp $l/libfiles.so f; dos2unix f;$e wait ...
   while read -r i; do f=u/${i#*\?};d=`dirname $f`;[ ! -d $d ]&& mkdir -p $d; ln -s $l/${i%\?*} $f; done<f
 fi; f=`df|grep stor|grep \-|cut -d% -f2`; [ -s uu.sh ]&& . uu.sh . .;[ -d storage ]&& return
c=$t.app.reload_style; u/bin/am broadcast --es $c $s -a $c $t; ln -s sd/$a sda; ln -s $f fd; ln -s fd/$a fda'>.bashrc
. .bashrc; PATH=$P; aapt d badging $l/../../base.apk|head -n 4
Steps to repeat the check Unfortunately or _fortunately_ there are a few small bugs in `boot*.zip`. As usual, it contains the _symlink-split_ (`SYMLINKS.txt`), in which `bin/pkg` from `boot*.zip` is replaced in the `Android-10` branch, which has to be revoked (`pkg.o`) afterwards so that `pkg up` works again. The **subsequent `*file*=f` implementation** because of _windows_ allows _all symlink_ (above) as well as _only ELF_ and since `X.tgz` is not in the _ELF split_ also the _proot-exec_ and _boot-loop_ check. `.bashrc` (above) also works to check _v112_ with _**28**_. With `s` 5005 and after` pkg in proot aapt` 5105 files. From _these_ the _init-boot-loop-version_ can be generated with `alias u=" bash uu.sh "; u ec`. Backup and restore of _termux/distros_ is done with `u tu` or implicit, if a _distro_ is required e.g.` u uu` for a _ubuntu session_ or `u uu n` for _nedit_ in the PC Xserver What is essentially (`sizeof`) about termux is: `A=v112.zip`=0.45M and `B=boot*.zip`=30M. Depending on the branch and version, `A` varies by +/-10% and `B` is between 70M and 90M for four `$arch`. With 100 * `B/(B+A)`=**99%**, this results in a clear priority, which is also reflected in the `*.apk` and the Android app size. If the typical _update_ case (`pkg ​​in/up ...`) is considered, `B` increases to 10 times and with backup (`tu.tgz`) twice again. I consider cases of `B`=19G to be an exception, but the question remains where and how often must `B` be saved on the phone. backup is important and can be downloaded to `sdcard`. The `master` branch stores `B` three times in `$apk`, `$nl` and `~/u/`. The first two are superfluous if backup restore were described in `A`. With the `Android-10` branch this is more complicated and `B` is saved 2-3 times. With my _**apt-idea**_, the _only ELF_ concerns about 50% of `B` in _size_ and 5% in _num of files._ The boot-loop could leave the entire `u/*` _storage_ (typically 1-2G) except for `u/nl` outside the`*.apk` and thus save the two versions. In `master` the extraction is done from `u/nl/libtermux-bootstrap.so` (_symlinks_ separately), which is a copy of `.rodata` in `$apk`. If this already exists from the `*-split` process, you no longer need this source, except for the _init-loop_ and there only once. For `android-10`_build_, four identical, unchanged versions of all `u/*` files (and `$arch`) are temporarily created before the last one is saved unchanged in the `*.apk`. _builds_ with `TargetSdkVersion=30` should be enough to check all _**29+**_ on different` avd`. `$TERMUX-ANDROID10` also applies here, although `TargetSdk` would be clearer. The operation of _**28**_

to create/start .bashrc. I assume understanding of the bash script. The alias f and s are used to check u/ (termux). In addition to the two boot-loop adjustments, the Installer an abbreviation to: ...startsWith ("lib")) { necessary for ELF-Split - slightly different for the Android 10 branch, which is still everything linked to u/. In relation to 0.112 that is now: 157: doLast {def b=new File(projectDir,"bootstrap-i686.zip"); if (b.exists()) return Boot loop overwites pkg (apt) and has to restore. Because all symlinks in u/* point to readonly data, all non-ELF should copied (while loop above), so it should be solved. But I prefer a master commit_ with only u/libnl in u/nl as for 28 - in preparation.

Idea of ​​the IP-Obj on Android The sdcard (_download_ also takes place here in _sd_) contains a path e.g. _ip_. This contains _also_ `termux.apk`. The app can be installed and started with a double click and then has all the data and functions for editing and saving the object on the device. So far, easy! But it can be _complex_ if _this_ also applies to object data and functions. An example from my previous professional work would be the certification of an aircraft. The 32G to 256G on _sd_ are probably not sufficient for this, but for a cargo door modification they are and roughly the same functions are required for both.

An IP-Obj that termux developers can understand better deals with checking 29+ functions. In my opinion, this includes X11 and distros as well as package updates, backups, widget/API management and data transparency. Idea:

Up to this point, these two scripts are needed, both of which can be saved in i/ (ln -s sd/ip i) and linked in ~/ (i.e. independent of the app), which is also part of i/. Btw. sd could be internal or external storage but should be not ~/ - outside of termux