clearlinux / clr-installer

Clear Linux* OS Installer
GNU General Public License v3.0
94 stars 43 forks source link

Requires internet connection for base install #817

Open GabeAl opened 1 month ago

GabeAl commented 1 month ago

This was supposedly fixed in 2019.

But with absolutely nothing configured or installed, no bundles selected, here's what the live installation CD does.

PXL_20241011_182514245

When it fails, I get kicked back to the installer and cannot progress.

Kernel 6.10.12 (very recent, build 42440) live desktop.

Clearly a bug

GabeAl commented 1 month ago

I was able to use my phone for hotspot and confirmed that 4GB of data was pulled. This is problematic, as I literally just downloaded an ISO of equal size that was completely up to date and contained the same packages, so what is going on here?

bwarden commented 1 month ago

Can you capture /root/clr-installer.log and add it here, please? This should contain more detailed messages about attempting offline installation.

K1ngfish3r commented 1 month ago
2024/10/19 07:33:17 [DBG] { "type" : "progress", "currentStep" : 2, "totalSteps" : 9, "stepCompletion" : 0, "stepDescription" : "download_packs" },
2024/10/19 07:33:17 [DBG] Target OS: : Setting progress for task download_packs
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : "Downloading packs (3969.87 MB) for:" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - os-core-plus" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - os-core-webproxy" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - btrfs-progs" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - openssh-client" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - iptables" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - wayland-server" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - python3-tcl" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - PyYAML" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - cloud-api" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - Babel" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - Sphinx" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - pypi-requests" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - pypi-cython" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - pypi-six" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - openblas" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - pypi-numpy" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - pandas" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - python-extras" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - file-roller" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - graphviz" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libva-utils" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - seahorse" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - baobab" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - connections" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - eog" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - evince" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - evolution" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - geary" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-calculator" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-color-manager" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-disk-utility" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-font-viewer" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-logs" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-music" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-photos" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-screenshot" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-system-monitor" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-text-editor" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - Endeavour" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-todo" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-weather" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - snapshot" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - totem" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - emacs-x11" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - firefox" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gvim" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - desktop-apps" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - desktop-assets" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - desktop-locales" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - openssl" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - fuse" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - pygobject" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - alsa-utils" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - tcl-basic" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - NetworkManager-extras" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - polkit" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - vte-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gphoto" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - mpfr-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - poppler" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - hardware-printing" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - xterm" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - x11vnc" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - xscreensaver" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - x11-tools" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - ghostscript" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnupg" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - flatpak" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libXpm-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gegl" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - lib-poppler" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gimp" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gjs" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-characters" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - lib-samba" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - webkitgtk" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - audio-pipewire" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libssh-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - dav1d-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - not-ffmpeg-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - desktop-gnomelibs" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - fontconfig" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - fonts-basic" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - qt-core" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - qt-basic" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gnome-base-libs" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libarchive" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - nautilus" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - thermal_daemon" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - desktop" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - git-lfs" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libpsl-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - linux-firmware-wifi" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - wpa_supplicant" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - NetworkManager-tui" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - iproute2" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - sudo" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - which" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - acpica-unix2" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - accel-config" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - diffutils" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - file" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - glibc-locale" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gzip" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - htop" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - iperf" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - kbd" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - kernel-install" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - less" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - lsof" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - man-pages" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - minicom" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - parallel" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - patch" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - powertop" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - procps-ng" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - socat" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libunwind-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - strace" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libevent-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - tmux" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - tree" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - unzip" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - lz4" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - zstd" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libpcap-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - pcre-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - clr-hardware-files" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - protobuf-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - aspell-es" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - aspell-fr" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - aspell-de" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - aspell" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - perl-basic" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - moreutils" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - bootloader-extras" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gobject-introspection" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - trurl" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - curl" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - lib-opengl" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - pulseaudio" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gstreamer" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - lib-openssl" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - openldap" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libX11client" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - icu4c-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - harfbuzz-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gdk-pixbuf" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - LibRaw-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - lib-imageformat" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - gmp-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - nettle-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - findutils" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - p11-kit" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - ncurses-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - sqlite" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - python3-basic" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libxml2" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libnl" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - hardware-uefi" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - syslinux" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - popt-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - bootloader" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - linux-firmware" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - linux-firmware-extras" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libstdcpp" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - systemd" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - yaml" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - dbus-lib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - libglib" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - os-core" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - numactl" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - qemu-guest-additions" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - os-core-update" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - openssh-server" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - desktop-autostart" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - git" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - vim" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - NetworkManager" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - sysadmin-basic" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - tzdata" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - boot-encrypted" },
2024/10/19 07:34:15 [DBG] { "type" : "info", "msg" : " - kernel-native" },

installing from 42150 full log: https://gist.github.com/K1ngfish3r/4020a29c19c785d2ab68f624f73341e4

GabeAl commented 1 month ago

Mine looks very similar.

But still exists with latest ISO (guaranteed fully up to date). Completely ridiculous to deploy an entirely broken installer that ignores the 4gb download you just did, to download another 4gb from the net, with packages EXACTLY IDENTICAL to what's on the ISO.

I literally can't believe it's possible for a bug this ridiculous to exist.

You don't need a log to reproduce this. Just try it. This isn't some one off thing.

It also persists in the text mode installer. I change telemetry, set a user، change time zone, select destructive install on a disk, and literally nothing else. No packages selected.

bwarden commented 1 month ago

I literally can't believe it's possible for a bug this ridiculous to exist.

Because it's not a bug. I spent a while in the source code investigating this. That's just not what was implemented.

The --offline option is supported when performing a scripted (YAML) installation. Offline mode is not supported from the interactive installation. You can save the installation configuration to a YAML file and feed that right back to the installer on the command line, and that should work with --offline. Note that this would work just as well from the live server image, which is half the size.

GabeAl commented 1 month ago

Oh. Well, that works. But it's far from ideal, nor is it best practice to require a workaround for what should be a basic functionality (and is, in most other OS's).

It's definitely still an issue. It's never mentioned during the install, doesn't really adhere to common sense, negates the use of delta files and hammers the network system twice for every install (costing tremendous bandwidth waste for the default installation for users and you alike), and is generally not how any other distribution works. Since efficiency is king for clear Linux in many other ways, something so wildly antipathic to that ethos sticks out like a sore thumb.

How can one save the yaml file and get it to persist through a reboot into the command line installer mode? Alternatively, how can one call up the command line when already booted and point to the file?

What's implemented seems like a hack, and doesn't match the spirit of what was announced years ago, and doesn't make intuitive or UI design sense.

Rather than have users deeply research how to go through this runaround, why can't the installer simply proceed with an offline installation exactly like the command line one would? More fundamentally, what's the point of even having your own installer if it is so incomplete (yes, I use the superlative "so" because this is a gaping oversight in a graphical installer and violates most best principles) compared to other installers already out there?

I think this is still a bug. The desire to acknowledge it as a coding technicality or simply a (substantial) UI and usability misstep notwithstanding, it clearly needs to be fixed/improved to meet user expectations of basic functionality.

K1ngfish3r commented 1 month ago

I need a checkbox that adds the --offline flag to the install Would be nice to have.

Unless the GUI installer and the CLI installer are different entities and operate in completely different fashions.

bwarden commented 1 month ago

How can one save the yaml file and get it to persist through a reboot into the command line installer mode? Alternatively, how can one call up the command line when already booted and point to the file?

I need a checkbox that adds the --offline flag to the install Would be nice to have.

I agree, that makes sense. clr-installer is a single binary that implements GUI/TUI/CLI. The big difference is whether Wayland and the GUI libs are included in the image. I'll see what I can do, but I want to make sure this gets a lot more testing before it gets exposed as an easy checkbox. I'm not sure how many people have actually executed that code in the real world.

I do want to point out that offline installation in Clear Linux works completely differently than in other distributions. For an RPM-based distro, for example, they just need to include repodata and a pile of RPMs in the image, and ask dnf to go install them. swupd is designed to fetch the latest manifests from our CDN, then go download only exactly the pieces it needs to update (which for installation, means the complete contents of files, not deltas). Offline installation is instead a shortcut that copies files from the live filesystem within the image. In theory this should work, but we can't rely on the decades of experience from other distros' approaches to reveal nor address the hidden pitfalls. Other distros are shipping redundant content in their images, so for a given size, we're shipping way more functional software in the live filesystem than they are. One option is for us to try to weed that waaaaay down to make an installer-only image. That means breaking bundles down a lot more -- the live-server image only includes these:

But it's still about 2.5 GB.