Calamares unable to load modules on slim iso install on vmware

Nathanwoodburn commented 1 year ago

[x] I have searched open and closed issues for duplicates
[x] I am submitting a bug report for existing functionality that does not work as intended

Bug description

The installer in the slim iso is giving an error about Calamares unable to load modules. This is happening on the latest vmware workstation.

Full error

BlackArch Linux can not be installed. Calamares was unable to load all of the configured modules. This is a problem with the way Calamares is being used
by the distribution.
The following modules could not be loaded:

• mount@mount
• unpackfs@rootfs
• unpackfs@vmlinuz
• unpackfs@ucode
• fstab@fstab
• localecfg@localecfg
• networkcfg@networkcfg
• hwclock@hwclock
• services-systemd@services-systemd
• packages@choice
• displaymanager@displaymanager
• initcpiocfg@initcpiocfg
• grubcfg@grubcfg
• bootloader@bootloader

Steps to reproduce

Download latest slim iso
Setup using workstation and type Other Linux 5.x kernel 64-bit
After boot run the executable on the live desktop

Actual result: This doesn't allow installing on VMware machine.

Expected result: Should continue to installation process.

Screenshots

Info for developers

Device: Windows 11 Pro laptop with Hyper-V & Windows Sandbox enabled. GNU/Linux distribution: Just the iso from the downloads page

Link to debug log

None available

apt-ghetto commented 1 year ago

Starting the installer with sudo calamares -d shows

2023-04-07 - 12:52:52 [6]: virtual void Calamares::ViewModule::loadSelf()
    ViewModule "summary@summary" loading complete. 
2023-04-07 - 12:52:52 [6]: void Calamares::ModuleManager::loadModules()
    Module "partition@partition" already loaded. 
2023-04-07 - 12:52:52 [6]: Calamares::SlideshowQML::SlideshowQML(QWidget*)
    QML import paths: 
     "/usr/share/calamares/qml" 
     "/usr/bin" 
     "qrc:/qt-project.org/imports" 
     "/usr/lib/qt/qml" 
2023-04-07 - 12:52:52 [6]: Calamares::SlideshowQML::SlideshowQML(QWidget*)
    QML load on startup, API 2. 
2023-04-07 - 12:52:52 [2]: QThread::setPriority: Cannot set priority, thread is not running
2023-04-07 - 12:52:52 [1]: void Calamares::ModuleManager::loadModules()
    ERROR: Module "mount@mount" not found in module search paths. 
     "/usr/lib/calamares/modules" 
2023-04-07 - 12:52:52 [1]: void Calamares::ModuleManager::loadModules()
    ERROR: Module "unpackfs@rootfs" not found in module search paths. 
     "/usr/lib/calamares/modules" 
2023-04-07 - 12:52:52 [1]: void Calamares::ModuleManager::loadModules()
    ERROR: Module "unpackfs@vmlinuz" not found in module search paths. 
     "/usr/lib/calamares/modules"

Then I copied the files for the mount module from https://github.com/calamares/calamares/tree/v3.2.61/src/modules/mount into /usr/lib/calamares/modules/mount and started Calamares again. This time I see:

2023-04-07 - 14:08:02 [6]: void Calamares::ModuleManager::loadModules()
    Module "partition@partition" already loaded. 
2023-04-07 - 14:08:02 [6]: Calamares::SlideshowQML::SlideshowQML(QWidget*)
    QML import paths: 
     "/usr/share/calamares/qml" 
     "/usr/bin" 
     "qrc:/qt-project.org/imports" 
     "/usr/lib/qt/qml" 
2023-04-07 - 14:08:02 [6]: Calamares::SlideshowQML::SlideshowQML(QWidget*)
    QML load on startup, API 2. 
2023-04-07 - 14:08:02 [2]: QThread::setPriority: Cannot set priority, thread is not running
2023-04-07 - 14:08:02 [1]: Calamares::Module* Calamares::moduleFromDescriptor(const ModuleSystem::Descriptor&, const QString&, const QString&, const QString&)
    ERROR: Python modules are not supported in this version of Calamares. 
2023-04-07 - 14:08:02 [1]: Calamares::Module* Calamares::moduleFromDescriptor(const ModuleSystem::Descriptor&, const QString&, const QString&, const QString&)
    ERROR: Bad module type ( "job" ) or interface string ( "python" ) for module  "mount" 
2023-04-07 - 14:08:02 [1]: void Calamares::ModuleManager::loadModules()
    ERROR: Module "mount@mount" cannot be created from descriptor "mount.conf"

The problems may be related to cmake and/or Boost

anunna commented 1 year ago

I have checked, and boost 1.81 is installed. Calamares requires at least 1.72 to not have the issues listed. Issue is somewhere else in the iso. since I have redone calamares config from scratch and added all the missing modules that became required after recent calamares updates. Still investigating the problem.

apt-ghetto commented 1 year ago

I was playing around with PKGBUILD from https://raw.githubusercontent.com/BlackArch/blackarch/master/packages/calamares/PKGBUILD

sha256sum PKGBUILD
9290faa788ad440ac78021413003925280619ba025bd5ca93b0e20a0cfb647d4  PKGBUILD

and have locally built the calamares package after manually installing the missing dependencies with:

LC_ALL=C makepkg --log

and got the binary with

sha256sum calamares-3:3.2.61-1-x86_64.pkg.tar.zst
2b80e979c5fa4fe2b4659ddb520297a8f2f6d645bdefa6f58ea7fbce9e7356a3  calamares-3:3.2.61-1-x86_64.pkg.tar.zst

When I compare the output of

sudo pacman -S calamares
pacman -Ql calamares | grep modules

with the output of

sudo pacman -U ./calamares-3:3.2.61-1-x86_64.pkg.tar.zst
pacman -Ql calamares | grep modules

I obtain the following differences:

--- ./modules_official.txt  2023-04-19 18:40:27.471065454 +0200
+++ ./modules_local.txt 2023-04-19 18:39:47.004731423 +0200
@@ -16,9 +16,18 @@
 calamares /usr/include/libcalamares/modulesystem/RequirementsModel.h
 calamares /usr/include/libcalamares/modulesystem/ViewModule.h
 calamares /usr/lib/calamares/modules/
+calamares /usr/lib/calamares/modules/bootloader/
+calamares /usr/lib/calamares/modules/bootloader/bootloader.schema.yaml
+calamares /usr/lib/calamares/modules/bootloader/main.py
+calamares /usr/lib/calamares/modules/bootloader/module.desc
+calamares /usr/lib/calamares/modules/bootloader/test.yaml
 calamares /usr/lib/calamares/modules/contextualprocess/
 calamares /usr/lib/calamares/modules/contextualprocess/libcalamares_job_contextualprocess.so
 calamares /usr/lib/calamares/modules/contextualprocess/module.desc
+calamares /usr/lib/calamares/modules/displaymanager/
+calamares /usr/lib/calamares/modules/displaymanager/displaymanager.schema.yaml
+calamares /usr/lib/calamares/modules/displaymanager/main.py
+calamares /usr/lib/calamares/modules/displaymanager/module.desc
 calamares /usr/lib/calamares/modules/finished/
 calamares /usr/lib/calamares/modules/finished/libcalamares_viewmodule_finished.so
 calamares /usr/lib/calamares/modules/finished/module.desc
@@ -28,12 +37,28 @@
 calamares /usr/lib/calamares/modules/fsresizer/
 calamares /usr/lib/calamares/modules/fsresizer/libcalamares_job_fsresizer.so
 calamares /usr/lib/calamares/modules/fsresizer/module.desc
+calamares /usr/lib/calamares/modules/fstab/
+calamares /usr/lib/calamares/modules/fstab/fstab.schema.yaml
+calamares /usr/lib/calamares/modules/fstab/main.py
+calamares /usr/lib/calamares/modules/fstab/module.desc
+calamares /usr/lib/calamares/modules/fstab/test.yaml
+calamares /usr/lib/calamares/modules/fstab/test2.yaml
+calamares /usr/lib/calamares/modules/grubcfg/
+calamares /usr/lib/calamares/modules/grubcfg/grubcfg.schema.yaml
+calamares /usr/lib/calamares/modules/grubcfg/main.py
+calamares /usr/lib/calamares/modules/grubcfg/module.desc
 calamares /usr/lib/calamares/modules/hostinfo/
 calamares /usr/lib/calamares/modules/hostinfo/libcalamares_job_hostinfo.so
 calamares /usr/lib/calamares/modules/hostinfo/module.desc
+calamares /usr/lib/calamares/modules/hwclock/
+calamares /usr/lib/calamares/modules/hwclock/main.py
+calamares /usr/lib/calamares/modules/hwclock/module.desc
 calamares /usr/lib/calamares/modules/initcpio/
 calamares /usr/lib/calamares/modules/initcpio/libcalamares_job_initcpio.so
 calamares /usr/lib/calamares/modules/initcpio/module.desc
+calamares /usr/lib/calamares/modules/initcpiocfg/
+calamares /usr/lib/calamares/modules/initcpiocfg/main.py
+calamares /usr/lib/calamares/modules/initcpiocfg/module.desc
 calamares /usr/lib/calamares/modules/keyboard/
 calamares /usr/lib/calamares/modules/keyboard/libcalamares_viewmodule_keyboard.so
 calamares /usr/lib/calamares/modules/keyboard/module.desc
@@ -43,6 +68,9 @@
 calamares /usr/lib/calamares/modules/locale/
 calamares /usr/lib/calamares/modules/locale/libcalamares_viewmodule_locale.so
 calamares /usr/lib/calamares/modules/locale/module.desc
+calamares /usr/lib/calamares/modules/localecfg/
+calamares /usr/lib/calamares/modules/localecfg/main.py
+calamares /usr/lib/calamares/modules/localecfg/module.desc
 calamares /usr/lib/calamares/modules/luksbootkeyfile/
 calamares /usr/lib/calamares/modules/luksbootkeyfile/libcalamares_job_luksbootkeyfile.so
 calamares /usr/lib/calamares/modules/luksbootkeyfile/module.desc
@@ -52,33 +80,61 @@
 calamares /usr/lib/calamares/modules/machineid/
 calamares /usr/lib/calamares/modules/machineid/libcalamares_job_machineid.so
 calamares /usr/lib/calamares/modules/machineid/module.desc
+calamares /usr/lib/calamares/modules/mkinitfs/
+calamares /usr/lib/calamares/modules/mkinitfs/main.py
+calamares /usr/lib/calamares/modules/mkinitfs/module.desc
+calamares /usr/lib/calamares/modules/mount/
+calamares /usr/lib/calamares/modules/mount/main.py
+calamares /usr/lib/calamares/modules/mount/module.desc
+calamares /usr/lib/calamares/modules/mount/mount.schema.yaml
 calamares /usr/lib/calamares/modules/netinstall/
 calamares /usr/lib/calamares/modules/netinstall/libcalamares_viewmodule_netinstall.so
 calamares /usr/lib/calamares/modules/netinstall/module.desc
+calamares /usr/lib/calamares/modules/networkcfg/
+calamares /usr/lib/calamares/modules/networkcfg/main.py
+calamares /usr/lib/calamares/modules/networkcfg/module.desc
 calamares /usr/lib/calamares/modules/notesqml/
 calamares /usr/lib/calamares/modules/notesqml/libcalamares_viewmodule_notesqml.so
 calamares /usr/lib/calamares/modules/notesqml/module.desc
 calamares /usr/lib/calamares/modules/oemid/
 calamares /usr/lib/calamares/modules/oemid/libcalamares_viewmodule_oemid.so
 calamares /usr/lib/calamares/modules/oemid/module.desc
+calamares /usr/lib/calamares/modules/openrcdmcryptcfg/
+calamares /usr/lib/calamares/modules/openrcdmcryptcfg/main.py
+calamares /usr/lib/calamares/modules/openrcdmcryptcfg/module.desc
 calamares /usr/lib/calamares/modules/packagechooser/
 calamares /usr/lib/calamares/modules/packagechooser/libcalamares_viewmodule_packagechooser.so
 calamares /usr/lib/calamares/modules/packagechooser/module.desc
 calamares /usr/lib/calamares/modules/packagechooserq/
 calamares /usr/lib/calamares/modules/packagechooserq/libcalamares_viewmodule_packagechooserq.so
 calamares /usr/lib/calamares/modules/packagechooserq/module.desc
+calamares /usr/lib/calamares/modules/packages/
+calamares /usr/lib/calamares/modules/packages/main.py
+calamares /usr/lib/calamares/modules/packages/module.desc
+calamares /usr/lib/calamares/modules/packages/packages.schema.yaml
 calamares /usr/lib/calamares/modules/partition/
 calamares /usr/lib/calamares/modules/partition/libcalamares_viewmodule_partition.so
 calamares /usr/lib/calamares/modules/partition/module.desc
 calamares /usr/lib/calamares/modules/plasmalnf/
 calamares /usr/lib/calamares/modules/plasmalnf/libcalamares_viewmodule_plasmalnf.so
 calamares /usr/lib/calamares/modules/plasmalnf/module.desc
+calamares /usr/lib/calamares/modules/plymouthcfg/
+calamares /usr/lib/calamares/modules/plymouthcfg/main.py
+calamares /usr/lib/calamares/modules/plymouthcfg/module.desc
+calamares /usr/lib/calamares/modules/plymouthcfg/plymouthcfg.schema.yaml
 calamares /usr/lib/calamares/modules/preservefiles/
 calamares /usr/lib/calamares/modules/preservefiles/libcalamares_job_preservefiles.so
 calamares /usr/lib/calamares/modules/preservefiles/module.desc
+calamares /usr/lib/calamares/modules/rawfs/
+calamares /usr/lib/calamares/modules/rawfs/main.py
+calamares /usr/lib/calamares/modules/rawfs/module.desc
 calamares /usr/lib/calamares/modules/removeuser/
 calamares /usr/lib/calamares/modules/removeuser/libcalamares_job_removeuser.so
 calamares /usr/lib/calamares/modules/removeuser/module.desc
+calamares /usr/lib/calamares/modules/services-systemd/
+calamares /usr/lib/calamares/modules/services-systemd/main.py
+calamares /usr/lib/calamares/modules/services-systemd/module.desc
+calamares /usr/lib/calamares/modules/services-systemd/services-systemd.schema.yaml
 calamares /usr/lib/calamares/modules/shellprocess/
 calamares /usr/lib/calamares/modules/shellprocess/libcalamares_job_shellprocess.so
 calamares /usr/lib/calamares/modules/shellprocess/module.desc
@@ -91,6 +147,11 @@
 calamares /usr/lib/calamares/modules/umount/
 calamares /usr/lib/calamares/modules/umount/libcalamares_job_umount.so
 calamares /usr/lib/calamares/modules/umount/module.desc
+calamares /usr/lib/calamares/modules/unpackfs/
+calamares /usr/lib/calamares/modules/unpackfs/main.py
+calamares /usr/lib/calamares/modules/unpackfs/module.desc
+calamares /usr/lib/calamares/modules/unpackfs/runtests.sh
+calamares /usr/lib/calamares/modules/unpackfs/unpackfs.schema.yaml
 calamares /usr/lib/calamares/modules/users/
 calamares /usr/lib/calamares/modules/users/libcalamares_viewmodule_users.so
 calamares /usr/lib/calamares/modules/users/module.desc
@@ -104,27 +165,38 @@
 calamares /usr/lib/calamares/modules/zfs/libcalamares_job_zfs.so
 calamares /usr/lib/calamares/modules/zfs/module.desc
 calamares /usr/share/calamares/modules/
+calamares /usr/share/calamares/modules/bootloader.conf
 calamares /usr/share/calamares/modules/contextualprocess.conf
+calamares /usr/share/calamares/modules/displaymanager.conf
 calamares /usr/share/calamares/modules/finished.conf
 calamares /usr/share/calamares/modules/finishedq.conf
 calamares /usr/share/calamares/modules/fsresizer.conf
+calamares /usr/share/calamares/modules/fstab.conf
+calamares /usr/share/calamares/modules/grubcfg.conf
 calamares /usr/share/calamares/modules/initcpio.conf
 calamares /usr/share/calamares/modules/keyboard.conf
 calamares /usr/share/calamares/modules/license.conf
 calamares /usr/share/calamares/modules/locale.conf
 calamares /usr/share/calamares/modules/luksopenswaphookcfg.conf
 calamares /usr/share/calamares/modules/machineid.conf
+calamares /usr/share/calamares/modules/mount.conf
 calamares /usr/share/calamares/modules/netinstall.conf
 calamares /usr/share/calamares/modules/notesqml.conf
 calamares /usr/share/calamares/modules/oemid.conf
+calamares /usr/share/calamares/modules/openrcdmcryptcfg.conf
 calamares /usr/share/calamares/modules/packagechooser.conf
 calamares /usr/share/calamares/modules/packagechooserq.conf
+calamares /usr/share/calamares/modules/packages.conf
 calamares /usr/share/calamares/modules/partition.conf
 calamares /usr/share/calamares/modules/plasmalnf.conf
+calamares /usr/share/calamares/modules/plymouthcfg.conf
 calamares /usr/share/calamares/modules/preservefiles.conf
+calamares /usr/share/calamares/modules/rawfs.conf
 calamares /usr/share/calamares/modules/removeuser.conf
+calamares /usr/share/calamares/modules/services-systemd.conf
 calamares /usr/share/calamares/modules/shellprocess.conf
 calamares /usr/share/calamares/modules/umount.conf
+calamares /usr/share/calamares/modules/unpackfs.conf
 calamares /usr/share/calamares/modules/users.conf
 calamares /usr/share/calamares/modules/usersq.conf
 calamares /usr/share/calamares/modules/welcome.conf

Maybe I am doing it wrong, but shouldn't makepkg produce a package with the same files in it as the package from the repo?

meyer-pidiache commented 1 year ago

I have the same issue when trying to install natively.

anunna commented 1 year ago

We have made changes thats confirmed working and are currently building/testing. We will rerelease slim iso with working installer.

anunna commented 1 year ago

We have released a working version of slim iso tested on multiple machines - blackarch-linux-slim-2023.05.01.

apt-ghetto commented 1 year ago

@anunna Thank you. Just out of curiosity: What was the problem?

anunna commented 1 year ago

Over the past three years calamares has changed a bit and our configs did not follow even though we continued to update the package, this was redone from scratch, you can check the last commits of the last two days in the commit history of blacakrch-config-calamares repo and the PKGBUILD got additional dep changes and some changes to the install process

BlackArch / blackarch-iso