Support for the termux android app

[yifeikong]

Termux does not work well with binary wheels. I havn't find a way to make it work on termux. However, it works on ubuntu inside termux. First, you need to install the AnLinux app, then install a chrooted ubuntu, everything should work out of the box.

[commiyou]

I would like to install revChatGPT on Termux, which depends on curl_cffi. I manually installed curl_cffi with the following modifications:

1. Installed rust and binutils.
2. Modified preprocess/download_so.py, setting LIBDIR = '/data/data/com.termux/files/usr'.
3. Commented out delocate-wheel dist/*.whl in the makefile.
4. Finally, generated dist/curl_cffi-0.5.7-cp37-abi3-linux_aarch64.whl and installed it using pip.

I hope this information is helpful.

[George-Seven]

@commiyou I tried it with old curl_cffi-0.5.7 source but there is same error as https://github.com/yifeikong/curl_cffi/issues/121

[Loamf]

I would like to install revChatGPT on Termux, which depends on curl_cffi. I manually installed curl_cffi with the following modifications:

1. Installed rust and binutils.
2. Modified preprocess/download_so.py, setting LIBDIR = '/data/data/com.termux/files/usr'.
3. Commented out delocate-wheel dist/*.whl in the makefile.
4. Finally, generated dist/curl_cffi-0.5.7-cp37-abi3-linux_aarch64.whl and installed it using pip.

I hope this information is helpful.

@commiyou can you share the curl-cffi whl file? It doesn't work for me.

[George-Seven]

@Almaz-Kabirov you also need to compile the corresponding libcurl-impersonate-chrome.so.4.8.0 shared library via curl-impersonate and place it in Termux's /data/data/com.termux/file/usr/lib folder, then only edit the makefile for curl_cffi and can compile the curl_cffi.whl file.

Check my discussion in install gpt4free in Termux to get started. I can send the build script later when I have time.

[Ivan-Firefly]

@George-Seven >I can send the build script later when I have time.

Faced with the same issues. Kindly remind for building script

[yifeikong]

According to this thread, it seems that the default manylinux wheel if not suitable for termux environment. I will build special wheels and put them in GitHub releases in the future.

[George-Seven]

@Ivan-Firefly this build script is originally meant for creating a single deb file that installs gpt4free for Termux.

It builds all the dependencies like libcurl-impersonate-chrome.so.4.8.0(curl-impersonate-chrome) and curl_cffi.whl, and then bundles them together into a single deb file which can be installed on any same architecture devices using -

pkg install ./path/to/python-g4f.deb

You can install the output deb file which will setup all dependencies itself, or you can modify the script to build as per needs.

You can find all the built artifacts inside the curl-impersonate/output folder, including the built curl_cffi.whl.

To build it nano build_g4f_deb.sh and copy/paste the script and CTRL + X + Y + ENTER to save it and then bash build_g4f_deb.sh

#!/data/data/com.termux/files/usr/bin/env bash

set -e

out_dir="output"
out_dir="$(basename "${out_dir}")"
manifest_json='
{
  "control": {
    "Package": "PLACEHOLDER",
    "Version": "PLACEHOLDER",
    "Architecture": "PLACEHOLDER",
    "Maintainer": "PLACEHOLDER",
    "Pre-Depends": [
      "PLACEHOLDER"
    ],
    "Homepage": "PLACEHOLDER",
    "Description": "PLACEHOLDER"
  },
  "data_files": {
    "PLACEHOLDER": {
      "source": "PLACEHOLDER"
    }
  }
}
'

termux-wake-lock &>/dev/null || true
yes | pkg upgrade -y
pkg update -y
pkg reinstall -y openssl openssl-static
pkg install -y build-essential pkg-config cmake ninja curl autoconf automake libtool golang jq binutils rust git termux-elf-cleaner python-pip python-cryptography libgsasl libpsl termux-create-package
rm -rf curl-impersonate
git clone --depth=1 https://github.com/lwthiker/curl-impersonate 
cd curl-impersonate
mkdir "${out_dir}"
manifest_json="$(jq '.data_files = {"share/python-g4f/bin/curl-impersonate-chrome": { "source": "curl-impersonate-chrome" }}' <<< "$manifest_json")"
for i in $(find chrome -type f -name "curl_*"); do
  cp -rf "${i}" "${out_dir}"
  i="$(basename "${i}")"
  manifest_json="$(jq ".data_files += {\"share/python-g4f/bin/${i}\": { \"source\": \"${i}\" }}" <<< "$manifest_json")"
done
mkdir build
cd build
clear || true
../configure --prefix="$PREFIX"
curl_version="$(sed -En 's/^CURL_VERSION := (.+)/\1/p' Makefile)"
sed -i '/\.\/configure \$\$config_flags LIBS="$$add_libs"/q' Makefile
sed -i '/\.\/configure \$\$config_flags LIBS="\$\$add_libs"/i \\tfind . -maxdepth 1 -type f -exec sed -i "s#/etc/ssl/cert.pem#$(PREFIX)\/etc\/tls\/cert.pem#g" {} +' Makefile
sed -i '/\.\/configure \$\$config_flags LIBS="\$\$add_libs"/i \\tfind . -maxdepth 1 -type f -exec sed -i "s#/etc/ssl/certs#$(PREFIX)/etc/tls#g" {} +' Makefile
sed -i '/\.\/configure \$\$config_flags LIBS="\$\$add_libs"/i \\tfind . -maxdepth 1 -type f -exec sed -i "s/ca-certificates\.crt/cert\.pem/g" {} +' Makefile
sed -i '/\.\/configure \$\$config_flags LIBS="\$\$add_libs"/a \\trm -r "$(PREFIX)\/lib\/"libssl.* "$(PREFIX)\/lib\/"libcrypto.* "$(PREFIX)\/include\/openssl"' Makefile
sed -i '/cd boringssl\//a \\tsed -i '\''s\/-Werror\/\/g'\'' CMakeLists.txt' Makefile
sed -i '/cd boringssl\//a \\tsed -i '\''s\/\&\& \!defined(OPENSSL_ANDROID)\/\/g'\'' ssl\/test\/handshake_util.h' Makefile
set +e
make chrome-build
pkg reinstall -y openssl openssl-static
set -e
cd .."/${out_dir}"
mv -f .."/build/${curl_version}/src/.libs/curl-impersonate-chrome" .
mv -f .."/build/${curl_version}/lib/.libs/libcurl-impersonate-chrome.a" .
mv -f .."/build/${curl_version}/lib/.libs/libcurl-impersonate-chrome.so.4.8.0" .
strip -s * &>/dev/null || true
termux-elf-cleaner * &>/dev/null || true
ln -s libcurl-impersonate-chrome.so.4.8.0 libcurl-impersonate-chrome.so
ln -s libcurl-impersonate-chrome.so.4.8.0 libcurl-impersonate-chrome.so.4
cp -pf curl-impersonate-chrome "$PREFIX/bin"
cp -pf libcurl-impersonate-chrome.so.4.8.0 "$PREFIX/lib"
$(cd "$PREFIX/lib"; ln -sf libcurl-impersonate-chrome.so.4.8.0 libcurl-impersonate-chrome.so; ln -sf libcurl-impersonate-chrome.so.4.8.0 libcurl-impersonate-chrome.so.4)
cd ../build
git clone --depth=1 https://github.com/yifeikong/curl_cffi
cd curl_cffi
# Needs fix, but skipping for now
sed -i -E 's/(.*MAX_RECV_SPEED_LARGE.*)/# \1/' curl_cffi/requests/session.py
# Fix Makefile
sed -i '/delocate-wheel/d' Makefile
sed -i 's/delocate//g' Makefile
sed -i '/urlretrieve(/d' scripts/build.py
set +e
termux-wake-lock &>/dev/null
CARGO_BUILD_TARGET="$(rustc -Vv | grep "host" | awk '{print $2}')" pip install -U maturin || true
retry=1
while ! pip show maturin &>/dev/null && ((retry < 10)); do
    termux-wake-lock &>/dev/null
    sleep 5
    CARGO_BUILD_TARGET="$(rustc -Vv | grep "host" | awk '{print $2}')" pip install -U maturin || true
    ((retry++))
done
set -e
rm -rf "$PREFIX/include/curl"
mv -f .."/${curl_version}/include/curl" "$PREFIX/include"
set +e
make build
rm -rf "$PREFIX/include/curl"
pkg reinstall -y libcurl
set -e
mv -f "dist/"curl_cffi-*.whl ../.."/${out_dir}"
mkdir ../python-g4f
cd ../python-g4f

postinst='#!/data/data/com.termux/files/usr/bin/env bash

set -e

mv -f "$PREFIX/share/python-g4f/bin/"* "$PREFIX/bin"
mv -f "$PREFIX/share/python-g4f/lib/"* "$PREFIX/lib"
ln -sf "$PREFIX/lib/libcurl-impersonate-chrome.so.4.8.0" "$PREFIX/lib/libcurl-impersonate-chrome.so"
ln -sf "$PREFIX/lib/libcurl-impersonate-chrome.so.4.8.0" "$PREFIX/lib/libcurl-impersonate-chrome.so.4"
pip install -U --force-reinstall "$PREFIX/share/python-g4f/"curl_cffi-*.whl || true
retry=1
while ! pip show curl_cffi &>/dev/null && ((retry < 10)); do
    sleep 5
    pip install -U --force-reinstall "$PREFIX/share/python-g4f/"curl_cffi-*.whl || true
    ((retry++))
done

rm -rf "$PREFIX/share/python-g4f" &>/dev/null

pip install -U cython flask

CFLAGS="-Wno-error=incompatible-function-pointer-types" pip install -U lxml

set +e

pip install -U g4f || true
retry=1
while ! pip show g4f &>/dev/null && ((retry < 10)); do
    sleep 5
    pip install -U g4f || true
    ((retry++))
done

if ! pip show g4f &>/dev/null; then
    exit 1
fi

set -e

echo -e "\n  Try testing it -"
echo -e "\n    g4f gui -port 4447"
echo -e "\n  And open the url http://127.0.0.1:4447/chat in the browser"
echo -e ""
'
echo "$postinst" > postinst

curl_cffi_whl="$(basename "$(ls ../.."/${out_dir}/"curl_cffi-*.whl | head -n 1)")"

manifest_json="$(jq '.control.Package = "python-g4f"' <<< "$manifest_json")"
manifest_json="$(jq '.control.Maintainer = "@George-Seven"' <<< "$manifest_json")"
manifest_json="$(jq '.control.["Pre-Depends"] = ["python-pip", "python-numpy", "python-pillow", " binutils", "libxml2", "libxslt", "libgsasl", "libpsl", "rust"]' <<< "$manifest_json")"
manifest_json="$(jq '.control.Homepage = "https://github.com/xtekky/gpt4free"' <<< "$manifest_json")"
manifest_json="$(jq '.control.Description = "Pre-compiled dependencies for installing the g4f (gpt4free) Python PyPi package, which cannot be compiled out-of-the-box in Termux. The g4f PyPi package allows access to various collection of powerful language models, like GPT-4, for free in Python"' <<< "$manifest_json")"
manifest_json="$(jq '.control_files_dir = "."' <<< "$manifest_json")"
manifest_json="$(jq ".deb_dir = \"../../${out_dir}\"" <<< "$manifest_json")"
manifest_json="$(jq ".files_dir = \"../../${out_dir}\"" <<< "$manifest_json")"
manifest_json="$(jq '.data_files += {"share/python-g4f/lib/libcurl-impersonate-chrome.a": { "source": "libcurl-impersonate-chrome.a" }}' <<< "$manifest_json")"
manifest_json="$(jq '.data_files += {"share/python-g4f/lib/libcurl-impersonate-chrome.so.4.8.0": { "source": "libcurl-impersonate-chrome.so.4.8.0" }}' <<< "$manifest_json")"
manifest_json="$(jq ".data_files += {\"share/python-g4f/${curl_cffi_whl}\": { \"source\": \"${curl_cffi_whl}\" }}" <<< "$manifest_json")"
echo "$manifest_json" | jq . > manifest.json
termux-create-package --pkg-arch "$(dpkg --print-architecture)" --pkg-version "$(curl -sL https://pypi.org/pypi/g4f/json | jq -r .info.version)" manifest.json
set +e
shopt -s extglob
$(cd ../..; rm -rf !("${out_dir}") .* &>/dev/null)
echo -e "\nTermux debian package created, you can install it with -\n\n  pkg install -y \"$(cd ../.."/${out_dir}"; echo "$(pwd)/$(basename $(ls python-g4f*.deb | head -n 1))")\"\n\n"

[yifeikong]

Did anyone encountered libpthread not found error?