benibela / xidel

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
http://www.videlibri.de/xidel.html
GNU General Public License v3.0
674 stars 42 forks source link

Internet Error: -4 under any non-system shell when using ssh on Android 4.2.2 #48

Closed MegaCron closed 3 years ago

MegaCron commented 4 years ago

Hi all.

I get the following error under any non-system shell when using ssh on Android 4.2.2

Error: Internet Error: -4 when talking to: ... (any site, http, https)

The site can be any - it is both http and https. I got this error on standalone 4.3 bash and on bash 4.4 from entware.

Linux localhost 3.4.5 #1 SMP PREEMPT Thu Mar 27 16:19:17 CST 2014 armv7l GNU/Linux Xidel 0.9.9 (20200201.7173.9d381d1545ec) dropbear from SSHDroid 2.1.0

P.S. I can use strace on this device

benibela commented 4 years ago

-4 means no connection. Perhaps there is a firewall or no internet permission?

In strace there should be socket(AF_INET, SOCK_STREAM, IPPROTO_TCP), sendto... and recvfrom... (at least on linux)

MegaCron commented 4 years ago

Perhaps there is a firewall or no internet permission?

There is no problem with internet access. Both under sh and under bash

/system/bin/sh

root@highscreen:/data/data/berserker.android.apps.sshdroid/home # ping www.unit-conversion.info PING www.unit-conversion.info (23.227.169.89): 56 data bytes 64 bytes from 23.227.169.89: seq=0 ttl=55 time=288.898 ms 64 bytes from 23.227.169.89: seq=1 ttl=55 time=319.472 ms 64 bytes from 23.227.169.89: seq=2 ttl=55 time=293.829 ms 64 bytes from 23.227.169.89: seq=3 ttl=55 time=288.850 ms

--- www.unit-conversion.info ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 288.850/297.762/319.472 ms

root@highscreen:/data/data/berserker.android.apps.sshdroid/home # nslookup www.unit-conversion.info Server: 8.8.4.4 Address 1: 8.8.4.4 dns.google

Name: www.unit-conversion.info Address 1: 23.227.169.89

root@highscreen:/data/data/berserker.android.apps.sshdroid/home # curl -sI www.unit-conversion.info HTTP/1.1 200 OK Date: Fri, 13 Mar 2020 10:57:45 GMT Server: Apache Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate Pragma: no-cache Set-Cookie: PHPSESSID=qqoiunu82vsi6hkp62kb7r5p6h; path=/ X-Frame-Options: SAMEORIGIN X-XSS-Protection: 1; mode=block X-Content-Type-Options: nosniff Content-Type: text/html; charset=utf-8

standalone bash or entware bash

-bash-4.4# ping www.unit-conversion.info PING www.unit-conversion.info (23.227.169.89): 56 data bytes 64 bytes from 23.227.169.89: seq=0 ttl=55 time=294.376 ms 64 bytes from 23.227.169.89: seq=1 ttl=55 time=290.825 ms 64 bytes from 23.227.169.89: seq=2 ttl=55 time=290.542 ms 64 bytes from 23.227.169.89: seq=3 ttl=55 time=288.395 ms

--- www.unit-conversion.info ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 288.395/291.034/294.376 ms

-bash-4.4# nslookup www.unit-conversion.info Server: 8.8.4.4 Address 1: 8.8.4.4 dns.google

Name: www.unit-conversion.info Address 1: 23.227.169.89

-bash-4.4# curl -sI www.unit-conversion.info HTTP/1.1 200 OK Date: Fri, 13 Mar 2020 10:55:36 GMT Server: Apache Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate Pragma: no-cache Set-Cookie: PHPSESSID=uslfrv00fjil3c22r32dtss3cq; path=/ X-Frame-Options: SAMEORIGIN X-XSS-Protection: 1; mode=block X-Content-Type-Options: nosniff Content-Type: text/html; charset=utf-8

full strace log for sh - https://dl.dropboxusercontent.com/s/rdfvpkftqcksqno/xidel-under-sh.txt full strace log for bash - https://dl.dropboxusercontent.com/s/3epdrihymgojcki/xidel-under-bash.txt

Under bash, I did not find strings containing a "socket"

MegaCron commented 4 years ago

Can anybody help?

benibela commented 4 years ago

It is too weird

Looks like the sh worked, the server answer ("HTTP/1.1 200 OK\r\nDate: Fri, 13 M". ) is there. But otherwise I see no difference between the sh and bash log, except for the timezones in the sh log. But the timezones should not matter for HTTP.

Are there different environment variables (env) set in sh and bash?

MegaCron commented 4 years ago

Sorry for the long delay with my reply. Damn coronavirus.

As benibela said:

Are there different environment variables (env) set in sh and bash? I changed only PATH in this configuration

bash

-bash-4.4# env ANDROID_ASSETS=/system/app LC_ALL=en_US.UTF-8 SSH_CONNECTION=::ffff:192.168.10.230 14174 ::ffff:192.168.10.215 34555 BOOTCLASSPATH=/system/framework/core.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/mms-common.jar:/system/framework/android.policy.jar:/system/framework/services.jar:/system/framework/apache-xml.jar:/system/framework/mediatek-common.jar:/system/framework/mediatek-framework.jar:/system/framework/secondary-framework.jar:/system/framework/CustomProperties.jar:/system/framework/mediatek-telephony-common.jar:/system/framework/mediatek-op.jar LANG=en_US.UTF-8 ANDROID_BOOTLOGO=1 USER=root PWD=/data/data/berserker.android.apps.sshdroid/home HOME=/data/data/berserker.android.apps.sshdroid/home TMP=/opt/tmp SSH_TTY=/dev/pts/15 TERM=xterm SHELL=/opt/bin/bash SHLVL=1 ANDROID_ROOT=/system TEMP=/opt/tmp ANDROID_DATA=/data TERMINFO=/opt/share/terminfo EXTERNAL_STORAGE=/storage/sdcard0 LOGNAME=root ANDROID_PROPERTYWORKSPACE=8,49664 PATH=/opt/bin:/opt/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin =/opt/bin/env

sh

root@highscreen:/data/data/berserker.android.apps.sshdroid/home # env _=/data/data/berserker.android.apps.sshdroid/home/.bin/env RANDOM=4990 ANDROID_DATA=/data HOSTNAME=highscreen BOOTCLASSPATH=/system/framework/core.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/mms-common.jar:/system/framework/android.policy.jar:/system/framework/services.jar:/system/framework/apache-xml.jar:/system/framework/mediatek-common.jar:/system/framework/mediatek-framework.jar:/system/framework/secondary-framework.jar:/system/framework/CustomProperties.jar:/system/framework/mediatek-telephony-common.jar:/system/framework/mediatek-op.jar PATH=/data/data/berserker.android.apps.sshdroid/dropbear:/data/data/berserker.android.apps.sshdroid/home/.bin:/usr/bin:/usr/sbin:/bin:/sbin:/system/sbin:/system/bin:/system/xbin:/system/xbin/bb:/data/local/bin SSH_TTY=/dev/pts/15 SHELL=/system/bin/sh MKSH=/system/bin/sh USER=root LOGNAME=root ANDROID_PROPERTY_WORKSPACE=8,49664 EXTERNAL_STORAGE=/storage/sdcard0 ANDROID_ASSETS=/system/app TERM=xterm ANDROID_ROOT=/system LD_LIBRARY_PATH=/vendor/lib:/system/lib HOME=/data/data/berserker.android.apps.sshdroid/home ANDROID_BOOTLOGO=1 SSH_CONNECTION=::ffff:192.168.10.230 14421 ::ffff:192.168.10.215 34555


bash new

LC_ALL=en_US.UTF-8 LANG=enUS.UTF-8 PWD=/data/data/berserker.android.apps.sshdroid/home TMP=/opt/tmp SHLVL=1 TEMP=/opt/tmp TERMINFO=/opt/share/terminfo =/opt/bin/env

different

SSH_CONNECTION=::ffff:192.168.10.230 14174 ::ffff:192.168.10.215 34555 PATH=/opt/bin:/opt/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin SHELL=/opt/bin/bash


sh new

RANDOM=4990 HOSTNAME=highscreen MKSH=/system/bin/sh LD_LIBRARY_PATH=/vendor/lib:/system/lib

different

PATH=/data/data/berserker.android.apps.sshdroid/dropbear:/data/data/berserker.android.apps.sshdroid/home/.bin:/usr/bin:/usr/sbin:/bin:/sbin:/system/sbin:/system/bin:/system/xbin:/system/xbin/bb:/data/local/bin SHELL=/system/bin/sh SSH_CONNECTION=::ffff:192.168.10.230 14421 ::ffff:192.168.10.215 34555

benibela commented 4 years ago

LD_LIBRARY_PATH can affect a lot of things. Try to run it with LD_LIBRARY_PATH=/vendor/lib:/system/lib xidel

MegaCron commented 4 years ago

As benibela said:

LD_LIBRARY_PATH can affect a lot of things. Try to run it with LD_LIBRARY_PATH=/vendor/lib:/system/lib

This does not work :( Again the same error :(

-bash-4.4# echo $LD_LIBRARY_PATH

-bash-4.4# /data/sdext1/bin/xidel -se '//title' 'http://www.unit-conversion.info' Error: Internet Error: -4 when talking to: http://www.unit-conversion.info/ -bash-4.4# export LD_LIBRARY_PATH=/vendor/lib:/system/lib -bash-4.4# /data/sdext1/bin/xidel -se '//title' 'http://www.unit-conversion.info' Error: Internet Error: -4 when talking to: http://www.unit-conversion.info/

MegaCron commented 3 years ago

After a long time, I updated (20200201.7173.9d381d1545ec) xidel to 20210620.7908.1f19357dcdf2 and the problem is gone, but there were problems with https requests > _ <