termux / termux-packages

A package build system for Termux.
https://termux.dev
Other
13.36k stars 3.08k forks source link

Some packages cannot be built 2: Electric boogaloo #21130

Open TomJo2000 opened 3 months ago

TomJo2000 commented 3 months ago

[!IMPORTANT] Build validation should be done without use of the "fast build" -i/-I flags! https://github.com/termux/termux-packages/blob/ed5c640b46197c5b1c36434938bfd68fcdba7536/build-package.sh#L417-L418 e.g.

./scripts/run-docker.sh ./build-package.sh -f [package [, ...]]

We want to verify that every package can be "bootstrapped" without the need for pre-compiled dependencies.

List of packages that ran into build issues

Batch 1

Batch 2 - [x] recutils (#21120) - [x] redir (#21166) - [x] sl (curses.h, can't reproduce) - [x] sord (can't reproduce) - [x] sse2neon - [x] sssnake (curses.h) - [x] ssss (gmp.h) - [x] stag (curses.h) - [x] stfl (ncursesw/ncurses.h and iconv.h) - [x] suitesparse (gmp.h) - [x] sundials (gmp.h) - [x] tar (#21120) - [x] tcc (#21247) - [x] termux-am (probably builds, but did not finish up the build process) - [x] timewarrior (returning, glob.h) - [x] tmux (glob.h) - [x] tokei (license, #21163) - [x] tty-clock (ncurses.h) - [x] tty-solitaire (ncurses.h) - [x] ttyplot (ncurses.h) - [x] typst-lsp (time crates, #21248) - [x] udftools (readline/readline.h) - [x] vbindiff (linker error, panel.h, can't reproduce) - [x] vile - [x] vim - [x] watchexec (rust: time crate, #21277) - [x] wdiff (source dead, e9bfaaaddb479616a6380735249c5c926a39c46c) - [x] weggli (returning, #21231) - [x] z3 #21500 - [x] zig (`error: ld.lld: /data/data/com.termux/files/usr/lib/libncursesw.so is incompatible with elf64-x86-64`) (dont use `build-all.sh`) - [x] zrok (tag rereleased, change sha256 fixed by autoupdate) - [ ] squashfuse (configure: error: Can't find FUSE with pkgconfig) - [x] wavemon (tag rereleased, change sha256, #21164)

Batch 3 - now at commit https://github.com/termux/termux-packages/commit/de399345d9b5f49786c1318809c33e8785b5c3b9 - [x] coreutils (#21120) - [x] cpio (#21120) - [x] dart (#21120) - [x] diffutils (#21120) - [ ] findomain - [x] findutils (#21120) - [x] hexcurse (#21165) - [x] hexer (#21221) - [x] libbsd (returning, https://github.com/termux/termux-packages/commit/344f345fd2eba5e1a2a844dc8cb5acbd4ce1a7c5) - [ ] liblightning - [ ] librocksdb - [x] libtdb (libbsd related) - [x] libtheora - [x] mg (libbsd related) - [x] netcat-openbsd (libbsd related) - [ ] pika - [x] pngquant (license, #21278) - [x] rcs (#21120) - [x] signify (libbsd related) - [x] steghide (returning, #21230) - [ ] tor - [ ] torsocks - [x] typst (rust: time crate, #21249) - [ ] valgrind (returning, symbols) - [ ] erofs-utils (FUSE issue) - [ ] iptables - [ ] apache-orc - [x] crowbook (rust: time crate, #21250) - [x] csh (libbsd related) - [ ] e2fsprogs (fuse, `Please add -D_FILE_OFFSET_BITS=64 to your compile flags!`) - [ ] e2tools (fuse, `Please add -D_FILE_OFFSET_BITS=64 to your compile flags!`) - [x] erlang (OOMs so hard it crashes my whole desktop environment) - [x] fdm (libbsd related) - [x] hollywood (#21120) - [x] lftp (#21120) - [ ] libduckdb (`/bin/sh: 1: /data/data/com.termux/files/usr/bin/ccache: Exec format error`) - [ ] libtd - [ ] libtorrent (symbols) - [ ] lighttpd (symbols) - [ ] luvit (`Aborted (core dumped)`) - [ ] mariadb (#21153) - [x] neovim - [x] nnn (#21120) - [ ] nodejs (abseil related?) - [ ] obfs4proxy (tor related) - [ ] predict (symbols) - [ ] rizin (symbols, and tree-sitter?) - [ ] sqlcipher - [x] xcb-util-image (symbols, #21172) - [x] xorg-mkfontscale (#21120) - [x] alpine (#21120) - [ ] distant (rust: libssh-rs-sys crate) - [x] elixir (Erlang BEAM VM, OOMs so hard it crashes my whole desktop environment) - [ ] fish - [ ] ghostscript (`clang++: error: unsupported option '-mfpu=' for target 'aarch64-linux-android24'`) - [ ] libmediainfo - [ ] libspectre (`clang++: error: unsupported option '-mfpu=' for target 'aarch64-linux-android24'`) - [ ] libxmlrpc (`curl/curl.h`) - [ ] lit (`Aborted (core dumped)`) - [ ] luarocks (`./configure: 437: /data/data/com.termux/files/usr/bin/lua5.4: Exec format error`) - [ ] mediainfo - [x] rsnapshot (#21120) - [ ] rtorrent (symbols) - [ ] sheldon - [x] tectonic (rust: time crate, 9014653415007d45a2076a8f8cb47f448c97fb8d) - [x] termux-tools (#21120) - [x] zsh (#21120) - [x] zsh-completions (#21120) - [ ] recordmydesktop (libtheora related) - [ ] xcb-util-cursor (symbols) - [x] xorg-fonts-100dpi (#21120) - [x] xorg-fonts-75dpi (#21120) - [x] bash (#21120) - [x] bash-completion (#21120) - [x] beanshell (#21120) - [x] c-script (#21120) - [ ] cargo-c (`libz` problems) - [x] fff (#21120) - [x] fzf (#21120) - [x] gdrive-downloader (#21120) - [x] google-drive-upload (#21120) - [x] has (#21120) - [x] hr (#21120) - [x] ldd (#21120) - [x] lesspipe (#21120) - [x] libtool (#21120) - [x] lrzip (#21120) - [x] mailutils (#21120) - [x] neofetch (#21120) - [x] paruz (#21120) - [x] pet (#21120) - [x] pipes.sh (#21120)

Batch 4 - now at commit https://github.com/termux/termux-packages/commit/51516ddb15bf560449ce0ae005b8722fd40dd2d0 + #21120 applied - [ ] tvheadend (`-Wunused-but-set-variable`) - [ ] eltclsh (license) - [ ] jack-example-tools (`Did not find CMake 'cmake'`) - [x] pypy (Related to `bionic-host`, #21197) wasn't this supposed to be fixed in #21026? - [ ] python-scipy (returning, `ERROR: Could not find a version that satisfies the requirement numpy==1.26.5`) - [ ] gdb - [x] gobject-introspection (`Download of gobject-introspection from https://packages-cf.termux.dev/apt/termux-main failed`)

Batch 5 - now at commit https://github.com/termux/termux-packages/commit/9016ef09eae7ac83450a0f8b2927fb4b69a520fa + #21120 applied - [ ] libosmium - [ ] osmium-tool (protobuf version mismatch) - [ ] python-contourpy (`ERROR: Could not find a version that satisfies the requirement numpy==1.26.5`) - [x] ravencoin (returning, #21229) - [ ] smalltalk (symbols) - [x] termux-x11-nightly (*really* slow AWS download) (7455a5ee385d661879e35184d2d6ae586982df5a) - [ ] enchant (`hunspell` dependency is *really* slow to download) - [ ] lgogdownloader - [ ] libarrow-cpp (protobuf or zlib) - [ ] libspatialite (*really* slow source) - [ ] lnav (symbols?) - [ ] mesa (`ERROR: Failed running '/data/data/com.termux/files/usr/bin/llvm-config', binary or interpreter not executable.`) - [x] pypy3 (Related to `bionic-host`) - [ ] python-pyarrow (libarrow-cpp related) - [ ] rust (`failed to execute command: "/data/data/com.termux/files/usr/bin/llvm-config" "--version"; ERROR: Exec format error (os error 8)`) - [x] samba (libbsd related) - [x] silicon (rust: time crate, #21276) - [ ] spatialite-tools (libspatialite related) - [ ] swift (SIMDMaskScalar) - [x] weechat-matrix-rs (#21232) - [ ] frida (wants NDK 25) - [x] far2l (libbsd related) - [ ] gw (can't find xz/lzma) - [ ] picom (xcb-util-image related) - [ ] qt6-qtbase (xcb-util-image related) - [ ] qt6-qtimageformats (xcb-util-image related) - [ ] virglrenderer (mesa related) - [ ] xorg-server-xvfb (mesa related) - [ ] xwayland (mesa related) - [ ] bitlbee - [ ] chrony (#21200) - [ ] crystal (doesn't like our LLVM version) - [ ] cups (strip out systemd stuff, `cp: cannot create regular file '/etc/dbus-1/system.d/#inst.2175092#': Permission denied`) - [ ] cups-pdf (strip out systemd stuff, `cp: cannot create regular file '/etc/dbus-1/system.d/#inst.2175092#': Permission denied`) - [ ] dpkg (probably see `apt`'s source change, `Failed to download https://mirrors.kernel.org/debian/pool/main/d/dpkg/dpkg_1.22.6.tar.xz`) - [ ] emacs - [ ] enscript (cups related) - [ ] gdal (#21147) - [ ] ghc-libs (returning, wants LLVM between 9 and 15) - [ ] gst-plugins-base (libtheora related) - [ ] gst-plugins-ugly (libtheora related) - [ ] gst-python (libtheora related) - [ ] ices (`configure: error: Unable to link with libxml`) - [ ] ldc (returning, #21243) - [ ] lfortran - [ ] libgnustep-base - [ ] libkiwix (libzim related) - [ ] libspice-server - [ ] lilypond (ghostscript related) - [ ] mapserver (gdal related) - [ ] matplotlib (`ERROR: Could not find a version that satisfies the requirement numpy==1.26.5`) - [ ] openjdk-17 (cups related) - [x] plantuml (openjdk-17 related) - [ ] postgis (gdal related) - [ ] procyon-decompiler (cups related) - [ ] qemu-system-x86-64-headless (libspice-server related) - [ ] scala (openjdk-17 related) - [ ] shellcheck (ghc-libs related) - [ ] termplay - [ ] termux-services (dpkg related) - [ ] texlive-bin - [ ] texlive-installer (texlive-bin related) - [ ] tinygo (symbols) - [ ] tizonia (libmediainfo related) - [ ] unar (libgnustep-base related) - [ ] alacritty (license) - [ ] emacs-x (see emacs) - [ ] feathernotes (hunspell-en-us related) - [ ] featherpad (hunspell-en-us related) - [ ] freerdp - [ ] mesa-demos (mesa related) - [x] qt5-qtwebengine (#21368) - [ ] qt6-qtdeclarative - [ ] qt6-qttools - [ ] qt6-qttranslations - [ ] qt6-qtwayland - [ ] qt6ct - [ ] the-powder-toy (symbols) - [ ] tigervnc (mesa related) - [ ] tuxpaint (symbols) - [ ] weston (freerdp related) - [ ] winestable (mesa related) - [ ] wlroots (mesa related) - [ ] xorg-server (mesa related) - [ ] xpdf - [ ] xrdp (mesa related) - [ ] ant (cups related) - [ ] apksigner (cups related) - [ ] apt (dpkg related) - [ ] aptitude (dpkg related) - [ ] bdsup2sub (cups related) - [ ] cabal-install (ghc-libs related) - [ ] d8 (cups related) - [ ] dex2jar (cups related) - [ ] dvdauthor (can't find xz/lzma) - [ ] gradle (openjdk-17 related) - [ ] graphviz (`configure: error: invalid ltdl library directory: '/data/data/com.termux/files/usr/lib'`) - [ ] groovy (openjdk-17 related) - [ ] gst-plugins-bad (libtheora related) - [ ] imagemagick (can't find xz/lzma) - [ ] imlib2 (can't find xz/lzma) - [ ] jython (openjdk-17 related) - [ ] kiwix-tools (libzim related) - [ ] kotlin (openjdk-17 related) - [ ] libapt-pkg-perl (dpkg related) - [ ] libbcprov-java (openjsk-17 related) - [ ] libcaca (can't find xz/lzma) - [ ] libcommons-lang3-java (openjsk-17 related) - [ ] libtsduck (openjdk-17 related) - [ ] lsix (can't find xz/lzma) - [ ] libtsduck (openjdk-17 related) - [ ] maven (openjdk-17 related) - [ ] mdbook-graphviz (graphviz related) - [ ] octave (symbols) - [x] pdftk (openjdk-17 related, #21223) - [x] php-imagick (can't find xz/lzma, #21227) - [ ] python-apt (dpkg related) - [ ] quilt (graphviz related) - [ ] toilet (can't find xz/lzma) - [ ] valac (graphviz related) - [ ] w3m (can't find xz/lzma) - [ ] yosys (can't find xz/lzma) - [ ] zbar (imagemagick related) - [ ] ayatana-ido (can't find xz/lzma) - [ ] bluefish (`hunspell` dependency is *really* slow to download) - [ ] cairo-dock-core (graphviz related) - [ ] chocolate-doom (symbols) - [ ] clutter-gst (`configure: error: Unable to locate required GL library`) - [ ] dosdox-x (SDL symbols) - [ ] fcitx5 (`hunspell` dependency is *really* slow to download) - [ ] fcitx5-hangul (`hunspell` dependency is *really* slow to download) - [ ] fcitx5-qt (`hunspell` dependency is *really* slow to download) - [ ] feh (can't find xz/lzma) - [ ] fluxbox (can't find xz/lzma) - [ ] gcr (can't find xz/lzma) - [ ] gcr4 (can't find xz/lzma) - [x] godot (returning, can't find OpenGL, #21237) - [ ] goffice (ghostscript related) - [ ] gspell (`hunspell` dependency is *really* slow to download) - [ ] gtksourceview3 (libltdl related) - [ ] gtksourceview4 (libltdl related) - [ ] gtkwave (can't find xz/lzma) - [ ] gucharmap (libltdl related) - [ ] gvfs (libltdl related) - [ ] keepassxc (botan3 related) - [ ] kf6-karchive (qt6-qttools related) - [ ] kf6-kcodecs (qt6-qtdeclarative related) - [ ] kf6-kconfig (qt6-qtdeclarative related) - [ ] kf6-kcoreaddons (qt6-qtdeclarative related) - [ ] kf6-kguiaddons (qt6-qtdeclarative related) - [ ] kf6-ki18n (qt6-qtdeclarative related) - [ ] kf6-kitemmodels (qt6-qtdeclarative related) - [ ] kf6-kitemviews (qt6-qtdeclarative related) - [ ] kf6-kwidgetsaddons (qt6-qtdeclarative related) - [ ] kf6-kwindowsystem (qt6-qtdeclarative related) - [ ] layer-shell-qt (qt6-qtdeclarative related) - [ ] libayatana-indicator (valac related) - [ ] libdazzle (graphviz related) - [ ] libdbusmenu (graphviz related) - [ ] libgtksourceviewmm-3.0 (valac related) - [ ] libhandy (graphviz related) - [ ] libhandy-0.0 (graphviz related) - [ ] libportal (graphviz related) - [ ] libvte (graphviz related) - [ ] libxfce4util (graphviz related) - [ ] libxklavier (graphviz related) - [ ] lite-xl (SDL symbols) - [ ] love (SDL symbols) - [ ] lxqt-menu-data (qt6-qtdeclarative related) - [ ] lyx (ghostscript related) - [ ] mate-terminal (libvte related) - [ ] mindforger (hunspell-en-us related) - [ ] mogan (ghostscript related) - [ ] octave-x (symbols) - [ ] openbox (imlib2 reated) - [ ] orca (gst-plugins-base related) - [ ] pavucontrol-qt (qt6-qtdeclarative related) - [ ] pidgin (gstreamer related) - [ ] qemu-system-x86-64 (libtheora related) - [ ] qt-creator (#21244) - [ ] qt5-qtmultimedia (unable to find SDL) - [ ] qt5-qtwebkit (`EGL/eglplatform.h`) - [ ] qt6-qtcharts (qt6-qtdeclarative related) - [ ] qt6-qtmultimedia - [ ] qtermwidget - [ ] quassel (`EGL/eglplatform.h`) - [ ] rhythmbox (`EGL/eglplatform.h`) - [ ] roxterm (valac related) - [ ] schismtracker (SDL symbols) - [ ] scrot (imlib2 related) - [ ] simulide (`EGL/eglplatform.h`) - [ ] sway (wlroots related) - [ ] synaptic (dpkg related) - [ ] texstudio (hunspell related) - [ ] texworks (hunspell related) - [ ] tilda (valac related) - [ ] tint2 (imlib2) - [ ] trojita (qt5-qtmultimedia related) - [ ] tsmuxergui (`EGL/eglplatform.h`) - [x] tumbler (graphviz related) - [ ] wireshark-qt (`EGL/eglplatform.h`) - [ ] wkhtmltopdf (`EGL/eglplatform.h`) - [ ] wmaker (imagemagick related) - [ ] xf86-input-void (mesa related) - [ ] xf86-video-dummy (mesa related) - [ ] xfconf (graphviz related) - [ ] appstream (graphviz related) - [ ] apt-file (dpkg related) - [ ] awesomeshot (imagemagick related) - [ ] babl (graphviz related) - [ ] dmtx-utils (imagemagick related) - [x] ffmpeg - [x] ffmpegthumbnailer - [ ] gegl (graphviz related) - [ ] gexiv2 (libltdl related) - [ ] gpac (libtheora related) - [ ] gst-libav (libtheora related) - [ ] gst-plugins-good (libtheora related) - [ ] imgflo (graphviz related) - [ ] libgee (graphviz related) - [ ] libgmime (graphviz related) - [ ] libsecret (graphviz related) - [ ] libvips (imagemagick related) - [ ] manim (`ERROR: Could not find a version that satisfies the requirement numpy==1.26.5`) - [x] megacmd - [x] minidlna - [x] mplayer - [x] mpv (libtheora related) - [ ] mu (emacs related) - [ ] nala (dpkg related) - [x] navidrome (libtheora related) - [x] notcurses (libtheora related) - [ ] notmuch (graphviz related) - [x] pianobar (libtheora related) - [x] pipewire (libtheora related) - [ ] proton-bridge (graphviz related) - [x] rsgain (libtheora related) - [ ] srt2vobsub (cups related) - [ ] timg (cups related) - [x] unpaper (libtheora related) - [x] vgmstream (libtheora related) - [ ] waypipe - [x] ytui-music (libtheora related) - [ ] zile (graphviz related) - [ ] abiword (hunspell related) - [ ] audacious-plugins - [ ] cherrytree (hunspell related) - [x] deadbeef (`EGL/eglplatform.h`, #21501) - [ ] eog (valac related) - [ ] evince (`EGL/eglplatform.h`) - [ ] eww (valac related) - [ ] ffplay (can't find SDL) - [ ] firefox (skipped for time, I wanna be done) - [ ] geany (libvte related) - [ ] geany-plugins (geany related) - [ ] gimp (SDL symbols) - [ ] gimp-lqr-plugin (gimp related) - [ ] handbrake - [ ] kf6-kauth - [ ] libfm-qt (qt6-qtdeclarative related) - [ ] libqtxdg (qt6-qtdeclarative related) - [ ] libsysstat - [ ] lximage-qt - [ ] lxqt-archiver - [ ] lxqt-qtplugin (qt6-qttools related) - [ ] mgba (SDL symbols) - [ ] mpv-x (mpv related) - [ ] obconf (imlib2 related) - [ ] olivia (can't find xz/lzma) - [ ] opencv - [ ] oshu (SDL symbols) - [ ] otter-browser (qt5-qtwebengine) - [ ] parole (imlib2 related) - [ ] pcmanfm-qt (qt6-qtdeclarative) - [ ] phantomjs (qt5-qtwebengine) - [ ] pulseeffects (imlib2 related) - [ ] gnome-themes-extra (`EGL/eglplatform.h`) - [ ] fluent-gtk-theme (`EGL/eglplatform.h`) - [ ] python-pyqtwebengine (`EGL/eglplatform.h`) - [ ] python-qscintilla (`EGL/eglplatform.h`) - [ ] python-torch (`EGL/eglplatform.h`) - [ ] python-torchaudio (`EGL/eglplatform.h`) - [ ] python-torchvision (`EGL/eglplatform.h`) - [ ] qterminal - [ ] qtkeychain (`EGL/eglplatform.h`) - [ ] qtxdg-tools - [ ] ristretto (`EGL/eglplatform.h`) - [ ] scrcpy (`EGL/eglplatform.h`) - [x] thunderbird (`EGL/eglplatform.h`, #21486, #21511) - [ ] webkit2gtk-4.1 (hunspell-en-us related) - [ ] webkitgtk-6.0 (hunspell-en-us related) - [ ] wxwidgets - [ ] xfce-theme-manager (`EGL/eglplatform.h`) - [ ] xfce4-session (`EGL/eglplatform.h`) - [ ] xfce4-taskmanager (`EGL/eglplatform.h`) - [ ] xfce4-terminal (`EGL/eglplatform.h`) - [ ] xfwm4 (`EGL/eglplatform.h`) - [ ] zenity (`EGL/eglplatform.h`) - [ ] vlc (symbols) - [ ] ardour (`EGL/eglplatform.h`) - [ ] atril (`EGL/eglplatform.h`) - [ ] audacity (`EGL/eglplatform.h`) - [ ] cantata (`EGL/eglplatform.h`) - [ ] codeblock (`EGL/eglplatform.h`) - [ ] epiphany (`EGL/eglplatform.h`) - [ ] exo (`EGL/eglplatform.h`) - [ ] file-roller (`EGL/eglplatform.h`) - [ ] garcon (`EGL/eglplatform.h`) - [ ] gedit (hunspell-en-us related) - [ ] ghex (`EGL/eglplatform.h`) - [ ] gnome-font-viewer (`EGL/eglplatform.h`) - [ ] hugin (can't find OpenGL) - [ ] kid3 (`EGL/eglplatform.h`) - [ ] komorebi (can't find OpenGL) - [ ] lenmus (symbols) - [ ] liblxqt - [ ] lxqt-about - [ ] lxqt-config - [ ] lxqt-globalkeys - [ ] lxqt-notificationd - [ ] lxqt-openssh-askpass - [ ] lxqt-panel - [ ] lxqt-runner - [ ] lxqt-session - [ ] marco (`EGL/eglplatform.h`) - [ ] news-flash-gtk (`EGL/eglplatform.h`) - [ ] nextcloud-client (`EGL/eglplatform.h`) - [ ] obconf-qt (vulkan-headers related) - [ ] spek (can't find OpenGL) - [ ] surf (`EGL/eglplatform.h`) - [ ] vlc-qt (vlc related) - [ ] wxmaxima (can't find OpenGL) - [ ] xfce4-appfinder (`EGL/eglplatform.h`) - [ ] xfce4-panel (`EGL/eglplatform.h`) - [ ] xfce4-panel-profiles (`EGL/eglplatform.h`) - [ ] xfce4-places-plugin (`EGL/eglplatform.h`) - [ ] xfce4-pulseaudio-plugin (`EGL/eglplatform.h`) - [ ] xfce4-screensaver (`EGL/eglplatform.h`) - [ ] xfce4-screenshooter (`EGL/eglplatform.h`) - [ ] xfce4-settings (`EGL/eglplatform.h`) - [ ] xfce4-timer-plugin (`EGL/eglplatform.h`) - [ ] xfce4-wavelan-plugin (`EGL/eglplatform.h`) - [ ] xfce4-whiskermenu-plugin (`EGL/eglplatform.h`) - [ ] thunar (`EGL/eglplatform.h`) - [ ] thunar-archive-plugin (`EGL/eglplatform.h`) - [ ] vala-panel-appmenu (`EGL/eglplatform.h`, probably also valac) - [ ] xfce4-calculator-plugin (`EGL/eglplatform.h`) - [ ] xfce4-clipman-plugin (`EGL/eglplatform.h`) - [ ] xfce4-datetime-plugin (`EGL/eglplatform.h`) - [ ] xfce4-dict (`EGL/eglplatform.h`) - [ ] xfce4-docklike-plugin (`EGL/eglplatform.h`) - [ ] xfce4-eyes-plugin (`EGL/eglplatform.h`) - [ ] xfce4-genmon-plugin (`EGL/eglplatform.h`) - [ ] xfce4-mailwatch-plugin (`EGL/eglplatform.h`) - [ ] xfce4-netload-plugin (`EGL/eglplatform.h`) - [ ] xfce4-notes-plugin (`EGL/eglplatform.h`) - [ ] xfce4-notifyd (`EGL/eglplatform.h`) - [ ] xfdesktop (`EGL/eglplatform.h`) I'll update this as I go.
TomJo2000 commented 3 months ago

Started the second batch, looks like mostly simple linker problems so far.

Maxython commented 3 months ago

glibc-repo (complains about cgct, Maxy's problem)

I know the reason for this error

TomJo2000 commented 3 months ago

I know the reason for this error

Figured it'd be quicker to just leave that one for you to figure out.

TomJo2000 commented 3 months ago

Just finished the second batch, that officially puts us at 51 problems. And 791/2517

TomJo2000 commented 3 months ago

We're up to 80 now. I wanna get to 50%, and then I think I'm gonna call it a night.

TomJo2000 commented 3 months ago

148 now. 1557/2517, 61.86%

I've decided to clean up my build container, rebase to the latest commit, and apply #21120.

I think we get that everything that depends on coreutils is gonna be effected by it, so I'm just gonna save myself the hassle of writing it down another 20 times.

TomJo2000 commented 3 months ago

@Maxython I can't get gobject-introspection to build. You might wanna look into that, seeing as its half the reason for 20513.

Maxython commented 3 months ago

I can't get gobject-introspection to build.

Could you please post the final compilation log of the gobject-introspection package (gobject-introspection.out)?

TomJo2000 commented 3 months ago

Could you please post the final compilation log of the gobject-introspection package (gobject-introspection.out)?

termux - building gobject-introspection for arch aarch64...
Building dependency glib if necessary...
termux - building glib for arch aarch64...
A circular dependency was found on 'gobject-introspection', the old version of the package will be installed to resolve the conflict
Download of gobject-introspection from https://packages-cf.termux.dev/apt/termux-main failed

That's the entire log.

https://packages-cf.termux.dev/apt/termux-main/pool/main/g/gobject-introspection/ exists and I assume that's what it should be downloading. But it doesn't.

TomJo2000 commented 3 months ago

Updated the list with the latest batch. Got a bit delayed due to some glib messiness requiring ~150 extra rebuilds.

TomJo2000 commented 3 months ago

2427/2517.

431 so far. A vast majority are "related" failures. E.g. some dependency is having a problem, so the dependent package can't be built either. Progress has slowed to a crawl, and I need to sleep. I have been at this for 14 hours today.

TomJo2000 commented 3 months ago

In particular:

are a very common cause of related failures. Please try to prioritize these when looking for issues to fix.

licy183 commented 3 months ago

I tested some packages related to #21120 and marked them as resolved.

TomJo2000 commented 3 months ago

Finally done with the whole list. Those last 100 were a real slog to get through. Most common "new" failure from those seems to be

[12/13] Compiling C object src/libepoxy.so.p/meson-generated_.._gl_generated_dispatch.c.o
FAILED: src/libepoxy.so.p/meson-generated_.._gl_generated_dispatch.c.o
aarch64-linux-android-clang -Isrc/libepoxy.so.p -Isrc -I../src/src -Iinclude -I../src/include -Iinclude/epoxy -I/data/data/com.termux/files/usr/include -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=gnu99 -Oz -g -fstack-protector-strong -Oz -fPIC -Wpointer-arith -Wmissing-declarations -Wformat=2 -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Wbad-function-cast -Wold-style-definition -Wdeclaration-after-statement -Wunused -Wuninitialized -Wshadow -Wmissing-noreturn -Wmissing-format-attribute -Wredundant-decls -Werror=implicit -Werror=nonnull -Werror=init-self -Werror=main -Werror=missing-braces -Werror=sequence-point -Werror=return-type -Werror=trigraphs -Werror=array-bounds -Werror=write-strings -Werror=address -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -fno-strict-aliasing -Wno-int-conversion -fvisibility=hidden -MD -MQ src/libepoxy.so.p/meson-generated_.._gl_generated_dispatch.c.o -MF src/libepoxy.so.p/meson-generated_.._gl_generated_dispatch.c.o.d -o src/libepoxy.so.p/meson-generated_.._gl_generated_dispatch.c.o -c src/gl_generated_dispatch.c 
In file included from src/gl_generated_dispatch.c:26:
In file included from ../src/src/dispatch_common.h:59:
In file included from ../src/include/epoxy/egl.h:46:
include/epoxy/egl_generated.h:11:10: fatal error: 'EGL/eglplatform.h' file not found
   11 | #include "EGL/eglplatform.h"
      |          ^~~~~~~~~~~~~~~~~~~

From the opengl package.

TomJo2000 commented 3 months ago

Final count. 513 issues. If I had to guess, a solid 80% of those are because some dependency doesn't build.

fornwall commented 3 months ago

enchant (hunspell dependency is really slow to download) bluefish (hunspell dependency is really slow to download) fcitx5 (hunspell dependency is really slow to download) fcitx5-hangul (hunspell dependency is really slow to download) fcitx5-qt (hunspell dependency is really slow to download) gspell (hunspell dependency is really slow to download)

I think this might have been a temporary github or networking issue - we download from https://github.com/hunspell/hunspell/archive/v1.7.2.tar.gz and the file is less than a megabyte, so I don't see why downloading that file should be slow, and I can't reproduce it either. Do you (or anyone else) still see this?

truboxl commented 2 months ago

Leftovers from the old issue #20966:

truboxl commented 2 months ago

https://github.com/truboxl/termux-packages/commits/393a444cc789cbfc6e7616508d497268f3a8df22/

I haven't clean up and summarised the failed packages yet but can check out the failed GitHub Actions

twaik commented 1 month ago

I am not sure I should create separate issue for this. wine-stable can not be built for x86_64. ./scripts/run-docker.sh ./build-package.sh -I -f wine-stable -a x86_64 fails with error.

x86_64-linux-android-clang -m64 -c -o dlls/dinput/data_formats.o /home/builder/.termux-build/wine-stable/src/dlls/dinput/data_formats.c -Idlls/dinput \
  -I/home/builder/.termux-build/wine-stable/src/dlls/dinput -Iinclude \
  -I/home/builder/.termux-build/wine-stable/src/include \
  -I/home/builder/.termux-build/wine-stable/src/include/msvcrt -D_UCRT -D__WINESRC__ \
  -DDIRECTINPUT_VERSION=0x0700 -Wall -pipe -fcf-protection=none -fvisibility=hidden \
  -fno-stack-protector -fno-strict-aliasing -Wdeclaration-after-statement -Wempty-body \
  -Wignored-qualifiers -Winit-self -Wno-pragma-pack -Wstrict-prototypes -Wtype-limits \
  -Wunused-but-set-parameter -Wvla -Wwrite-strings -Wpointer-arith -fPIC \
  -fasynchronous-unwind-tables -D_WIN32 -fno-builtin -fshort-wchar -Wno-format -mabi=ms \
  -I/data/data/com.termux/files/usr/include 
clang: error: unsupported option '-mabi=' for target 'x86_64-unknown-linux-android24'
make: *** [Makefile:132508: dlls/dinput/data_formats.o] Error 1
make: *** Waiting for unfinished jobs....

It can not be disabled using configure flags because of this. BTW configure does not throw error about this.

  case $HOST_ARCH in
    dnl gcc-4.6+ omits frame pointers by default, breaking some copy protections
    i386) WINE_TRY_CFLAGS([-fno-omit-frame-pointer],[MSVCRTFLAGS="$MSVCRTFLAGS -fno-omit-frame-pointer"]) ;;
    x86_64)
      case $host_os in
        dnl Mingw uses Windows 64-bit types, not Unix ones
        cygwin*|mingw32*) WINE_TRY_CFLAGS([-Wno-format]) ;;
        dnl Default to ms_abi on 64-bit
        *) if test -z "$PE_ARCHS"
           then
               AC_CACHE_CHECK([for working -mabi=ms], ac_cv_mabi_ms,
                   [CFLAGS="$CFLAGS -mabi=ms"
                    AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <stdarg.h>
int a(int b, ...) { __builtin_ms_va_list list; __builtin_ms_va_start(list,b); }]])],
                       [ac_cv_mabi_ms=yes],[ac_cv_mabi_ms=no])
                    CFLAGS=$saved_CFLAGS])
               test $ac_cv_mabi_ms = yes || AC_MSG_ERROR([The compiler doesn't support -mabi=ms. Use gcc instead of clang, or install mingw-w64.])
           fi
           MSVCRTFLAGS="$MSVCRTFLAGS -mabi=ms" ;;
      esac ;;
    arm)
      WINE_TRY_CFLAGS([-Wincompatible-function-pointer-types],[EXTRACFLAGS="$EXTRACFLAGS -Wno-error=incompatible-function-pointer-types"]) ;;
  esac

Full log: winelog.txt Configure script mentions we should install mingw-w64 despite llvm-mingw is already installed by build script. No error thrown during configuring, probably the check for working -mabi=ms is skipped, but in this case I am not sure where -mabi=ms came from.

TomJo2000 commented 1 month ago

I am not sure I should create separate issue for this. wine-stable can not be built for x86_64.

robertkirkman commented 1 month ago
  • [ ] mesa (ERROR: Failed running '/data/data/com.termux/files/usr/bin/llvm-config', binary or interpreter not executable.)

(I also experienced Interpreter: Cannot run the interpreter "/data/data/com.termux/files/usr/bin/python3")

Hello, after testing a series of techniques for a while, this command fully completed successfully for me, bypassing both of the errors above,

scripts/run-docker.sh ./build-package.sh -f mesa

when I did this

--- a/scripts/build/termux_step_override_config_scripts.sh
+++ b/scripts/build/termux_step_override_config_scripts.sh
@@ -3,11 +3,28 @@ termux_step_override_config_scripts() {
        return
    fi

+   # experimental solution to many variants of "cannot execute binary file: Exec format error"
+   # symlink host binaries over incompatible architecture and libc crossbuild rootfs binaries
+   symlink_incompatible_binary() {
+       src="$1"
+       dst="$2"
+       if [ -f $src ] && ! file $(readlink -f $dst) | grep text >/dev/null \
+           && [ "$(readlink -f $src)" != "$(readlink -f $dst)" ]; then
+           echo "symlink_incompatible_binary: linking $dst to $src"
+           ln -sf $src $dst
+       fi
+   }
+   export -f symlink_incompatible_binary
+   find $TERMUX_PREFIX/bin/ -executable -exec bash -c 'symlink_incompatible_binary /usr/bin/$(basename {}) {}' \;
+   unset symlink_incompatible_binary
+   # from one perspective, the above block could be considered an expansion of the same logic
+   # behind the line below this, but for everything in the usr/bin folder instead of only bin/sh
+
    # Make $TERMUX_PREFIX/bin/sh executable on the builder, so that build
    # scripts can assume that it works on both builder and host later on:
    ln -sf /bin/sh "$TERMUX_PREFIX/bin/sh"

-   if [ "$TERMUX_INSTALL_DEPS" = false ] || [ "$TERMUX_PACKAGE_LIBRARY" = "glibc" ]; then
+   if [ "$TERMUX_PACKAGE_LIBRARY" = "glibc" ]; then
        return
    fi

The first error is #20336 . Very curiously, an error I assumed was only relevant to forks, appeared in this issue a few days later, despite the package name in this issue being com.termux. It can be bypassed using the same patch, rebased.

I strongly suspect that the block of code I wrote to bypass the second error could possibly also impact the builds of multiple other packages, especially when using the entrypoint scripts/run-docker.sh ./build-all.sh, as opposed to scripts/run-docker.sh ./build-package.sh. I will run that next while my patch is applied in order to check the result.

As another note, I sort of suspect that termux-play-store/termux-packages might also have its own very different way of bypassing similar errors. However there are possibly different, contrasting side effects for both solutions. I don't really know which is better, but if the termux-play-store/termux-packages will eventually combine into this repository, it might be best to go with that solution when the time comes.

TomJo2000 commented 1 month ago

@robertkirkman it'd be great if you could open a new PR for your patch above so it can be properly reviewed.

twaik commented 1 month ago

Seems like all the packages from the first batch are being built fine (according to all checkboxes marked). 👍

TomJo2000 commented 1 month ago

I'll do a recheck later today.

robertkirkman commented 1 month ago
  • [x] sl (curses.h, can't reproduce)

After applying all the WIP PRs in this thread, I can now consistently reproduce that error by using scripts/run-docker.sh ./build-all.sh and waiting until sl is built, (skipping only one package currently, hilbish, which appears to have a build issue tracked in the repository of one of its dependencies)

Here's what the full context looks like for that error with sl when it's reproduced:

INFO: Identifying files with nproc=32
INFO: Done ... 0s
INFO: Found 1 / 2 files
INFO: Running symbol checks on 1 files with nproc=32
INFO: Done ... 0s
termux - build of  'skate' done
done in 14 sec
Building sl... termux - building  sl for arch aarch64...
Building dependency ncurses if necessary...
ncurses@6.5.20240831-1 built - skipping (rm /data/data/.built-packages/ncurses to force rebuild)
Downloading https://github.com/eyJhb/sl/archive/5.05.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  631k    0  631k    0     0   642k      0 --:--:-- --:--:-- --:--:-- 1280k
Applying patch: makefile.patch
aarch64-linux-android-clang  -fstack-protector-strong -Oz  -isystem/data/data/com.termux/files/usr/include -L/data/data/com.termux/files/usr/lib -Wl,-rpath=/data/data/com.termux/files/usr/lib -Wl,--enable-new-dtags -Wl,--as-needed -Wl,-z,relro,-z,now -o sl sl.c -lncurses
sl.c:51:10: fatal error: 'curses.h' file not found
   51 | #include <curses.h>
      |          ^~~~~~~~~~
1 error generated.
make: *** [Makefile:12: sl] Error 1
ERROR: See /home/builder/.termux-build/_buildall-aarch64/sl.out
tacokoneko@CORSAIR ~/code/termux/electric-boogaloo/termux-packages $ scripts/run-docker.sh 
Running container 'termux-package-builder' from image 'ghcr.io/termux/package-builder'...
builder@78bc2abe14fb:~/termux-packages$ ls /data/data/com.termux/files/usr/include/curses.h
ls: cannot access '/data/data/com.termux/files/usr/include/curses.h': No such file or directory
builder@78bc2abe14fb:~/tmp$ ls /data/data/com.termux/files/usr/lib/libncursesw.so.6.5 
/data/data/com.termux/files/usr/lib/libncursesw.so.6.5
builder@78bc2abe14fb:~/termux-packages$ ls output/ | grep curses
ncurses_6.5.20240831-1_aarch64.deb
ncurses-static_6.5.20240831-1_aarch64.deb
ncurses-ui-libs_6.5.20240831-1_aarch64.deb
ncurses-ui-libs-static_6.5.20240831-1_aarch64.deb
ncurses-utils_6.5.20240831-1_aarch64.deb
pomodoro-curses_2.5_aarch64.deb
builder@78bc2abe14fb:~/termux-packages$ mkdir ~/tmp
builder@78bc2abe14fb:~/termux-packages$ cd ~/tmp
builder@78bc2abe14fb:~/tmp$ ar x ~/termux-packages/output/ncurses_6.5.20240831-1_aarch64.deb 
builder@78bc2abe14fb:~/tmp$ tar xf data.tar.xz 
builder@78bc2abe14fb:~/tmp$ ls data/data/com.termux/files/usr/include/curses.h 
data/data/com.termux/files/usr/include/curses.h
builder@78bc2abe14fb:~/tmp$ ls data/data/com.termux/files/usr/lib/libncursesw.so.6.5 
data/data/com.termux/files/usr/lib/libncursesw.so.6.5
builder@78bc2abe14fb:~/tmp$ 

I can see that ncurses was built and its lib/libncursesw.so.6.5 file was installed, but its include/curses.h file is somehow missing from the prefix right at the moment that sl starts building. I'm quite sure that my PR doesn't cause that since I don't modify or delete any files in the include folder, so I believe this likely has the same root cause as the person who first tested sl in this issue experienced. I'll debug this as much as I can and try to find the root cause.

Root cause found:

https://github.com/termux/termux-packages/blob/4ca64f804aaa24465a3bd4bfd62c6bc338f9c649/packages/simulavr/build.sh#L31-L38

the simulavr package's termux_step_post_make_install() kind of renames the entire $TERMUX_PREFIX/include folder to $TERMUX_PREFIX/include-simulavr . I'll think about that and I guess try to write a modified simulavr package that doesn't do something like that to the $TERMUX_PREFIX.

Done in #21835

TomJo2000 commented 1 month ago

Seems like all the packages from the first batch are being built fine (according to all checkboxes marked). 👍

Still getting a build error with bionic-host locally

```styl Downloading https://storage.googleapis.com/git-repo-downloads/repo % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 47141 100 47141 0 0 192k 0 --:--:-- --:--:-- --:--:-- 192k Downloading Repo source from https://gerrit.googlesource.com/git-repo Traceback (most recent call last): File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 874, in _Main(sys.argv[1:]) File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 850, in _Main result = repo._Run(name, gopts, argv) or 0 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 294, in _Run result = run() ^^^^^ File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 275, in lambda: self._RunLong(name, gopts, argv, git_trace2_event_log) or 0 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 442, in _RunLong execute_command() File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 408, in execute_command execute_command_helper() File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 374, in execute_command_helper result = cmd.Execute(copts, cargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/builder/.termux-build/bionic-host/src/.repo/repo/subcmds/init.py", line 406, in Execute self._ConfigureUser(opt) File "/home/builder/.termux-build/bionic-host/src/.repo/repo/subcmds/init.py", line 217, in _ConfigureUser name = self._Prompt("Your Name", mp.UserName) ^^^^^^^^^^^ File "/home/builder/.termux-build/bionic-host/src/.repo/repo/project.py", line 779, in UserName self._LoadUserIdentity() File "/home/builder/.termux-build/bionic-host/src/.repo/repo/project.py", line 792, in _LoadUserIdentity u = self.bare_git.var("GIT_COMMITTER_IDENT") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/builder/.termux-build/bionic-host/src/.repo/repo/project.py", line 3835, in runner p.Wait() File "/home/builder/.termux-build/bionic-host/src/.repo/repo/git_command.py", line 556, in Wait self.VerifyCommand() File "/home/builder/.termux-build/bionic-host/src/.repo/repo/git_command.py", line 546, in VerifyCommand raise GitCommandError( git_command.GitCommandError: GitCommandError: 'var GIT_COMMITTER_IDENT' on manifests failed stderr: Committer identity unknown *** Please tell me who you are. Run git config --global user.email "you@example.com" git config --global user.name "Your Name" to set your account's default identity. ```

Going into the container and running; git config --global user.email "@termux" git config --global user.name "Termux build container" Makes it proceed to this:

Your identity is: Termux build container <@termux>
If you want to change this, please re-run 'repo init' with --config-name

Testing colorized output (for 'repo diff', 'repo status'):
  black    red      green    yellow   blue     magenta   cyan     white 
  bold     dim      ul       reverse 
Enable color display in this user account (y/N)?

Which is also annoying since it shouldn't pop up anything that requires user input during a build. After that it does successfully fetch the repos though.

I have expressed my dislike for the Google repo utility in #21825 already, and this just reinforces it. But since it's what we have to use here there's no point moaning about it.

twaik commented 1 month ago

Going into the container and running; git config --global user.email "@termux" git config --global user.name "Termux build container" Makes it proceed to this:

It is already fixed in CI since it commits to termux-packages repo.

Makes it proceed to this:

AFAIK it does not pop up in non-interactive shell session (CI case).

TomJo2000 commented 1 month ago

Going into the container and running; git config --global user.email "@termux" git config --global user.name "Termux build container" Makes it proceed to this:

It is already fixed in CI since it commits to termux-packages repo.

Ah must have not fetched that on my test branch yet.

twaik commented 1 month ago

It is weird. In package_updates workflow it is done explicitly

https://github.com/termux/termux-packages/blob/4ca64f804aaa24465a3bd4bfd62c6bc338f9c649/.github/workflows/package_updates.yml#L147-L148

Can not find similar lines in packages workflow, possibly because it does not need to commit changes to repo.

In regular CI it is done by CI itself. Can not find proof but you can check repology-metadata repo, it has commits of Github Action worker. So I assume it handles some configuration.

I am not sure about Docker, bionic-host is built without it because it needs a lot of space and RAM.

TomJo2000 commented 1 month ago

I am not sure about Docker, bionic-host is built without it because it needs a lot of space and RAM.

Found a consistent way to get it to work in the local build container.

diff --git a/packages/bionic-host/build.sh b/packages/bionic-host/build.sh
index 7b87cca311..a3f5286e26 100644
--- a/packages/bionic-host/build.sh
+++ b/packages/bionic-host/build.sh
@@ -60,7 +60,7 @@ termux_step_get_source() {
    cd ${TERMUX_PKG_SRCDIR}

    cp -f ${TERMUX_PKG_BUILDER_DIR}/LICENSE.txt ${TERMUX_PKG_SRCDIR}/LICENSE.txt
-   
+
    for i in libtinfo5 libncurses5 openssh-client; do
        local URL="$(obtain_deb_url $i)"
        wget "$URL" -O ${TERMUX_PKG_CACHEDIR}/${URL##*/}
@@ -72,10 +72,16 @@ termux_step_get_source() {

    termux_download https://storage.googleapis.com/git-repo-downloads/repo ${TERMUX_PKG_CACHEDIR}/repo SKIP_CHECKSUM
    chmod +x ${TERMUX_PKG_CACHEDIR}/repo
+   (
+   # Repo requires us to have a Git user name and email set.
+   # The GitHub workflow does this, but the local build container doesn't
+   [[ "$(git config --get user.name)" != '' ]] || git config --global user.name "Termux Github Actions"
+   [[ "$(git config --get user.email)" != '' ]] || git config --global user.email "contact@termux.dev"
    ${TERMUX_PKG_CACHEDIR}/repo init \
        -u https://android.googlesource.com/platform/manifest \
-       -b main -m ${TERMUX_PKG_BUILDER_DIR}/default.xml
+       -b main -m ${TERMUX_PKG_BUILDER_DIR}/default.xml <<< 'n'
    ${TERMUX_PKG_CACHEDIR}/repo sync -c -j32
+   )

    sed -i '1s|.*|\#!'${TERMUX_PKG_SRCDIR}'/prebuilts/python/linux-x86/2.7.5/bin/python2|' ${TERMUX_PKG_SRCDIR}/bionic/libc/fs_config_generator.py
    sed -i '1s|.*|\#!'${TERMUX_PKG_SRCDIR}'/prebuilts/python/linux-x86/2.7.5/bin/python2|' ${TERMUX_PKG_SRCDIR}/external/clang/clang-version-inc.py

We can just feed an n into the repo init command with a herestring. For the Git User and Email I just checked if they're set and if not set them. The subshell isn't necessary anymore, that's a leftover from an earlier idea. I'll clean it up before I make a PR.

TomJo2000 commented 1 month ago

Okay so gforth doesn't like doing a 32 Bit build in the same environment it just did a 64 Bit build in (or vise versa), but otherwise works just fine.

robertkirkman commented 1 month ago
  • [x] termux-am (probably builds, but did not finish up the build process)

Update: Reproduced with procyon-decompiler + build-all.sh

Update: Bisected to Fixes and improvements to scripts for working with cyclic dependencies #20513

Cloning a clean copy of termux/termux-packages and running its original build-all.sh using a custom buildorder.txt to temporarily bypass the original build-all.sh's subpackage-related errors fully completes, and does not hang.

git clone https://github.com/termux/termux-packages.git
cd termux-packages/
# while i was troubleshooting, fornwall pushed a commit to termux-am,
# so i need to temporarily do this here to retain a consistent test methodology
# see the next comment for all the results of all the same tests on termux-am
# performed on a termux-packages copy AFTER commit a7627acf5a8104ff5634330110ff9acd661c09d8
git checkout 70aa9cf1372220b6bacfc9ac51321f12b11558fa 
scripts/run-docker.sh
mkdir -p ~/.termux-build/_buildall-aarch64/
echo 'termux-am packages/termux-am' > ~/.termux-build/_buildall-aarch64/buildorder.txt 
echo 'procyon-decompiler packages/procyon-decompiler' >> ~/.termux-build/_buildall-aarch64/buildorder.txt
./build-all.sh

^ Result of above: success with no hang

Building termux-am or procyon-decompiler using scripts/run-docker.sh ./build-package.sh -f works, but building it using scripts/run-docker.sh ./build-all.sh always shows "BUILD SUCCESSFUL " then hangs there, while several java processes run indefinitely in the background of the container. This particular problem is not a crossbuild prefix pollution issue like some of the others, because the cause is definitely related to build-all.sh itself specifically, since it happens even when using a clean image of the termux-package-builder container and skipping all packages that are not termux-am.

If you want, it is possible to quickly reproduce the problem by applying Fixes and improvements to scripts for working with cyclic dependencies #20513 and then this temporary patch to use the build-all.sh entrypoint but skip to the part where termux-am fails to proceed normally.

--- a/build-all.sh
+++ b/build-all.sh
@@ -75,6 +75,11 @@ exec &>      >(tee -a "$BUILDALL_DIR"/ALL.out)
 trap 'echo ERROR: See $BUILDALL_DIR/${PKG}.out' ERR

 while read -r PKG PKG_DIR; do
+       # TEMPORARY TEST
+       if [ "$PKG" != "termux-am" ] && [ "$PKG" != "procyon-decompiler" ]; then
+               continue
+       fi
+
        # Check build status (grepping is a bit crude, but it works)
        if [ -e "$BUILDSTATUS_FILE" ] && grep -q "^$PKG\$" "$BUILDSTATUS_FILE"; then
                echo "Skipping $PKG"

This is what the log looks like at the bottom when it hangs.

> Task :app:lintVitalAnalyzeRelease
> Task :app:lintVitalReportRelease
> Task :app:lintVitalRelease
> Task :app:assembleRelease

BUILD SUCCESSFUL in 58s
36 actionable tasks: 36 executed

(Frozen)

> Task :Procyon.Reflection:javadocJar
> Task :Procyon.Reflection:sourcesJar
> Task :Procyon.Reflection:assemble
> Task :Procyon.Reflection:check
> Task :Procyon.Reflection:build

BUILD SUCCESSFUL in 12s
23 actionable tasks: 23 executed

(Frozen)

I will troubleshoot this and try to find any convenient way to bypass this hang, to complete another necessary step in the process of making build-all.sh complete non-interactively.

fornwall commented 1 month ago

@robertkirkman Regarding gradle hanging, check out: https://github.com/gradle/gradle/issues/14961

As it's reported there that the problem no longer exists under gradle 8.9, I've merged a PR bumping the gradle version in termux-am: #21857. Does that fix the issue reliably when you test?

You're welcome to create additional PR:s for other packages, bumping gradle or (if it's not easy to bump gradle for certain packages) piping echo -n | into gradle input, as in:

-   $TERMUX_PKG_TMPDIR/gradle/gradle-$_GRADLE_VERSION/bin/gradle \
+   echo -n | $TERMUX_PKG_TMPDIR/gradle/gradle-$_GRADLE_VERSION/bin/gradle \
robertkirkman commented 1 month ago

@fornwall Thank you! Eventually, i detected that the problem also consistently bisected to #20513 , so I guess it would appear there is a problematic interaction between #20513 and Gradle < 8.9.

Here are my results for the same tests I did, but after your commit a7627acf5a8104ff5634330110ff9acd661c09d8 instead of before.

termux-am

git clone https://github.com/termux/termux-packages.git
cd termux-packages/
scripts/run-docker.sh
mkdir -p ~/.termux-build/_buildall-aarch64/
echo 'termux-am packages/termux-am' > ~/.termux-build/_buildall-aarch64/buildorder.txt
./build-all.sh

✅ Success

git clone https://github.com/termux/termux-packages.git
cd termux-packages/
curl https://patch-diff.githubusercontent.com/raw/termux/termux-packages/pull/20513.diff | git apply -v
patch  --ignore-whitespace << 'EOF'
--- a/build-all.sh
+++ b/build-all.sh
@@ -75,6 +75,9 @@ exec &>       >(tee -a "$BUILDALL_DIR"/ALL.out)
 trap 'echo ERROR: See $BUILDALL_DIR/${PKG}.out' ERR

 while read -r PKG PKG_DIR; do
+       if [ "$PKG" != "termux-am" ]; then
+               continue
+       fi
        # Check build status (grepping is a bit crude, but it works)
        if [ -e "$BUILDSTATUS_FILE" ] && grep -q "^$PKG\$" "$BUILDSTATUS_FILE"; then
                echo "Skipping $PKG"

EOF
scripts/run-docker.sh ./build-all.sh

✅ Success

procyon-decompiler

git clone https://github.com/termux/termux-packages.git
cd termux-packages/
scripts/run-docker.sh
mkdir -p ~/.termux-build/_buildall-aarch64/
echo 'procyon-decompiler packages/procyon-decompiler' > ~/.termux-build/_buildall-aarch64/buildorder.txt
./build-all.sh

✅ Success

[!TIP] For anyone unaware of this subtle detail, when you directly use an entrypoint that has the full dependency graph of termux-packages enabled, the build will take a very long time to get to the actual Gradle part that we are currently troubleshooting because it will compile the entire dependency openjdk-17 from source. For full clarity, just to maintain consistency of testing methodology, in the test below I did not interfere with that phenomenon and I allowed the build to progress fully including openjdk-17 all the way until it reached that point where building procyon-decompiler itself hangs. However if someone else were hypothetically doing this same or a similar process, they could consider temporarily removing the line for dependency on openjdk-17 in order to skip that and use the copy of JDK that already comes with the package builder image. Also, if you are invoking the build-package.sh entrypoint instead of the build-all.sh entrypoint, you could use the argument -I for the same purpose instead. https://github.com/termux/termux-packages/blob/18b8697bdc1bc5800a6b6bf4f09203032bdd6882/packages/procyon-decompiler/build.sh#L9

git clone https://github.com/termux/termux-packages.git
cd termux-packages/
curl https://patch-diff.githubusercontent.com/raw/termux/termux-packages/pull/20513.diff | git apply -v
patch  --ignore-whitespace << 'EOF'
--- a/build-all.sh
+++ b/build-all.sh
@@ -75,6 +75,9 @@ exec &>       >(tee -a "$BUILDALL_DIR"/ALL.out)
 trap 'echo ERROR: See $BUILDALL_DIR/${PKG}.out' ERR

 while read -r PKG PKG_DIR; do
+       if [ "$PKG" != "procyon-decompiler" ]; then
+               continue
+       fi
        # Check build status (grepping is a bit crude, but it works)
        if [ -e "$BUILDSTATUS_FILE" ] && grep -q "^$PKG\$" "$BUILDSTATUS_FILE"; then
                echo "Skipping $PKG"

EOF
scripts/run-docker.sh ./build-all.sh

❌ Hanging

Based on your suggestion and these results, it looks like, to continue using #20513 in its unmodified form, a separate PR should also be opened to update the Gradle versions used by the builds of procyon-decompiler and any other packages affected by a similar situation. I will try to do that now.

plantuml

The issue is also reproducible with this final package dependent on the Gradle build tool because the current release of plantuml , 1.2024.7, pulls in Gradle 8.2. However, I believe it might be easy to cherrypick and use this commit in the upstream development branch of plantuml that bumps its Gradle version to 8.10, very similarly to what you did to fix the equivalent problem in termux-am. I'll most likely try that approach for fixing plantuml.

TomJo2000 commented 1 month ago

For anyone unaware of this subtle detail, when you directly use an entrypoint that has the full dependency graph of termux-packages enabled, the build will take a very long time to get to the actual Gradle part that we are currently troubleshooting because it will compile the entire dependency openjdk-17 from source. For full clarity, just to maintain consistency of testing methodology, in the test below I did not interfere with that phenomenon and I allowed the build to progress fully including openjdk-17 all the way until it reached that point where building procyon-decompiler itself hangs. However if someone else were hypothetically doing this same or a similar process, they could consider temporarily removing the line for dependency on openjdk-17 in order to skip that and use the copy of JDK that already comes with the package builder image.

Yep that is expected behavior with the testing procedure outlined above as we want to verify that all packages can be "bootstrapped" without needing to rely on precompiled dependencies. Which wouldn't be available for forks using a different $PREFIX.

Since we've already validated the dependencies in this instance you can skip that and just build termux-am using:

./scripts/run-docker.sh ./build-package.sh -f -I -a all termux-am

Just be aware that validating "bootstrapability" is the main goal, so only use -I when the dependencies are already verified.

robertkirkman commented 1 month ago
./scripts/run-docker.sh ./build-package.sh -f -I -a all termux-am

Well, I have explained that the problem described in the last few comments is only reproducible specifically when using the PR listed as one of those for testing in this issue, #20513 plus specifically the build-all.sh script entrypoint, so it is not reproducible when using the build-package.sh script entrypoint. Therefore, in order to develop complete patches to solve the issue, it seems to me necessary to repeatedly invoke the build-all.sh script, or whichever specific part of it is triggering the problem, in order to perform checks to determine whether potential solutions have completely solved the issue yet.

For example, you can see that fornwall's recent commit https://github.com/termux/termux-packages/commit/a7627acf5a8104ff5634330110ff9acd661c09d8 does not have an observable impact on the result of the entrypoint "./scripts/run-docker.sh ./build-package.sh -f -I -a all termux-am " . The impact has been observed exclusively on the entrypoint build-all.sh.

git clone https://github.com/termux/termux-packages.git
cd termux-packages
git checkout 70aa9cf1372220b6bacfc9ac51321f12b11558fa
./scripts/run-docker.sh ./build-package.sh -f -I -a all termux-am
git clone https://github.com/termux/termux-packages.git
cd termux-packages
./scripts/run-docker.sh ./build-package.sh -f -I -a all termux-am

The result in both of the examples above is exactly the same in every way other than that in the first case the resulting package built is version termux-am_0.8.0_all.deb and in the second case the resulting package built is termux-am_0.8.0-1_all.deb. So, you can see that the command you sent me is not an effective verification of the utility of fornwall's recent commit, so that is why I used build-all.sh in this case.

It is possible that maybe what you specifically mean is that I should also modify the build-all.sh a little bit more in order to apply the -I argument inside it manually, but I just avoided adding too many additional modifications on top of the build-all.sh for these tests in order to maintain the consistency of the testing methodology compared to the initial tests in the first comment where I described the problem.

robertkirkman commented 1 month ago

I forgot to really mention this anywhere until now that I build it again, but 1 month ago i discovered that the aalib package will not clean compile even as a single package.

docker container stop termux-package-builder 
docker container rm termux-package-builder 
git clone https://github.com/termux/termux-packages.git
cd termux-packages/
scripts/run-docker.sh ./build-package.sh -f aalib

However, fortunately it's very easy to fix using the aalib patch from its Arch Linux package.

I think what I was wondering is, since the aalib package in Termux doesn't need any Termux-specific patches, and all its patches in Arch Linux are not Arch Linux exclusive either they are just updated fixes for the legacy source code of aalib in general, would it be OK for me to make a PR that just deletes all the old aalib patches from termux-packages and replaces them with the more-recent Arch Linux patchset? (EDIT: that's not quite what I meant sorry, there are some Termux-specific patches so in reality I will add more patches rather than deleting all of them and replacing them)

In order to avoid spamming PRs I will wait until after my other open PR goes through the process before I start opening a bunch more.

TomJo2000 commented 1 month ago

I think what I was wondering is, since the aalib package in Termux doesn't need any Termux-specific patches, and all its patches in Arch Linux are not Arch Linux exclusive either they are just updated fixes for the legacy source code of aalib in general, would it be OK for me to make a PR that just deletes all the old aalib patches from termux-packages and replaces them with the more-recent Arch Linux patchset?

In order to avoid spamming PRs I will wait until after my other open PR goes through the process before I start opening a bunch more.

That sounds reasonable to me. And don't worry about "spamming PRs", we've got something like 500 packages to validate here. One extra PR isn't gonna break the camel's back.

robertkirkman commented 1 month ago

I am not sure if I should be opening separate issues for every "independent root cause error" (errors in packages that are unlikely to share that exact root cause with too many other packages due to the uniqueness of the error as it relates to that package's code) or continuing to use this issue to describe all package build errors generally, but since the docker builder image was updated to Ubuntu 24.04 which comes with glibc 2.39 https://github.com/termux/termux-packages/commit/ce35b3a1265b8e2eaab5ff686e0e8817be2c0b21, the hostbuild-step of the termux-packages fte package now fails to complete successfully due to the presence of glibc >= 2.38 in the build environment:

https://github.com/termux/termux-packages/blob/3bf8927ead6130cd5aab0a9075d5ae00c6b5821a/packages/fte/build.sh#L28-L32

Result:

g++ -c s_string.cpp   -Wall -Wpointer-arith -Wconversion -Wwrite-strings -Winline -g    -I/usr/include/qt3 -I/usr/lib64/qt-3.3/include  -I/usr/include/slang -DUNIX -DLINUX
In file included from s_string.cpp:3:
/usr/include/string.h:506:15: error: conflicting declaration of ‘size_t strlcpy(char*, const char*, size_t)’ with ‘C’ linkage
  506 | extern size_t strlcpy (char *__restrict __dest,
      |               ^~~~~~~
In file included from s_string.cpp:1:
s_string.h:10:8: note: previous declaration with ‘C++’ linkage
   10 | size_t strlcpy(char *dst, const char *src, size_t size);
      |        ^~~~~~~
/usr/include/string.h:506:15: error: declaration of ‘size_t strlcpy(char*, const char*, size_t) noexcept’ has a different exception specifier
  506 | extern size_t strlcpy (char *__restrict __dest,
      |               ^~~~~~~
s_string.h:10:8: note: from previous declaration ‘size_t strlcpy(char*, const char*, size_t)’
   10 | size_t strlcpy(char *dst, const char *src, size_t size);
      |        ^~~~~~~
/usr/include/string.h:512:15: error: conflicting declaration of ‘size_t strlcat(char*, const char*, size_t)’ with ‘C’ linkage
  512 | extern size_t strlcat (char *__restrict __dest,
      |               ^~~~~~~
s_string.h:14:8: note: previous declaration with ‘C++’ linkage
   14 | size_t strlcat(char *dst, const char *src, size_t size);
      |        ^~~~~~~
/usr/include/string.h:512:15: error: declaration of ‘size_t strlcat(char*, const char*, size_t) noexcept’ has a different exception specifier
  512 | extern size_t strlcat (char *__restrict __dest,
      |               ^~~~~~~
s_string.h:14:8: note: from previous declaration ‘size_t strlcat(char*, const char*, size_t)’
   14 | size_t strlcat(char *dst, const char *src, size_t size);
      |        ^~~~~~~
make[2]: *** [fte-unix.mak:159: s_string.o] Error 1

Very fortunately, there is actually an existing package for fte in the Ubuntu 24.04 official repository, so because the toolchain and build environment that termux-packages hostbuild-steps use is uncannily similar to the build environment that Ubuntu 24.04 official packages use, I can see the exact solution they have used for the same error by looking at the source code created by Ubuntu or Debian for the fte packaged in Ubuntu 24.04 and newer. https://git.launchpad.net/ubuntu/+source/fte/tree/debian/patches/strlcpy.patch

Using that patch seems to successfully solve the problem and allow the hostbuild-step and all the other steps of the fte package to successfully complete, so I have opened a draft PR to potentially solve the problem, if the package remains enabled and is not removed from this repository.

TomJo2000 commented 1 month ago

This round of testing was opened before we moved the build container to 24.04, so it's probably better to move this to a new issue.

robertkirkman commented 1 week ago

I have identified all packages that currently cannot be built that have no solution yet, meaning that using a clean docker container does not work around the problem.

All packages that cannot be built in a clean docker container - `aubio` - `ModuleNotFoundError: No module named 'imp'` - `jack2` - `ModuleNotFoundError: No module named 'imp'` - `qt5-qtwebengine` - `ModuleNotFoundError: No module named 'imp'` - `enchant` - `404` - `ircd-irc2` - `404` - `ghc-libs` - `error: Warning: Couldn't figure out LLVM version! Make sure you have installed LLVM between [9 and 15)` - `lenmus` - `lomse_font_freetype.cpp:203:31: error: assigning to 'char *' from 'unsigned char *' converts between pointers` - `inkscape` - `ld.lld: error: unable to find library -lGraphicsMagick++` - `cairo-dock-core` - `ld.lld: error: undefined reference due to --no-allow-shlib-undefined: atan2 >>> referenced by gldit/libgldi.so` - `crypto-monitor` - `error: member access into incomplete type 'WINDOW' (aka '_win_st')` - `frida` - `Unsupported NDK version 27. Please install NDK r25.` - `iptables` - `../include/linux/netfilter/nfnetlink.h:6:6: error: redefinition of 'nfnetlink_groups'` - `iwyu` - `iwyu_path_util.h:94:16: error: no member named 'equals' in 'llvm::StringRef'` - `ldc` - `Signals.h:119:8: error: variable or field ‘CleanupOnSignal’ declared void` - `lfortran` - `asr_to_llvm.cpp:36:10: fatal error: 'llvm/Transforms/Vectorize.h' file not found` - `mapserver` - `agg_font_freetype.cpp:177:35: error: assigning to 'char *' from 'unsigned char *' converts between pointers to integer types where one is of the unique plain 'char' type and the other is not ` - Bug report for the same package in another distro: https://bugs.gentoo.org/939022 - `openfoam` - `CGAL/IO/io.h:280:22: error: no member named 'optional' in namespace 'std'` - `openscad` - `FreetypeRenderer.h:127:37: error: no template named 'unary_function' in namespace 'std'; did you mean '__unary_function'?` - `poac` - `Error: tbb >= 2021.5.0, tbb < 2022.0.0 not found` - `postgis` - `pg_iovec.h:54:10: error: call to undeclared function 'preadv'` - `pypy3` - `proot error: 'uname' not found` - `qt-creator` - `clangformatbaseindenter.cpp:76:17: error: no member named 'startswith' in 'llvm::StringRef'; did you mean 'starts_with'?` - `swift` - `ld.lld: error: unable to find library -lswiftCore` - `thunderbird` - `error: the lock file /home/builder/.termux-build/thunderbird/src/comm/rust/Cargo.lock needs to be updated but --frozen was passed to prevent this` - `tint2` - `ld.lld: error: undefined symbol: log10` - `tvheadend` - `src/service.c:1168:7: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]` - `valgrind` - `ERROR: ./libexec/valgrind/vgpreload_drd-arm64-linux.so contains undefined symbols` - `vulkan-validation-layers` - `layer_chassis_dispatch.cpp:6309:97: error: too few arguments to function call, expected 4, have 2`

I have also found many packages that only fail to build if one or more other specific packages are already installed in the same prefix before building. I have listed them below the ones above in this WIP branch that I used to find all the errors, where I will continue looking for solutions to these errors. I have not solved any of the listed errors yet, but the packages listed above are the ones that have errors that can be reproduced by building them from the unmodified repository with a clean docker container.