OSInside / kiwi

KIWI - Appliance Builder Next Generation
https://osinside.github.io/kiwi
GNU General Public License v3.0
300 stars 152 forks source link

Testsuite fails in Debian after updating to 10.0.x branch #2548

Closed glaubitz closed 4 months ago

glaubitz commented 4 months ago

I am currently trying to update the Debian package from 9.25.22 to 10.0.15 and so far have been unsuccessful to build the updated version.

So far, I made the following changes to the packaging source:

diff --git a/debian/control b/debian/control
index 9d5d69b..a4e7073 100644
--- a/debian/control
+++ b/debian/control
@@ -6,11 +6,13 @@ Uploaders: Marcus Schäfer <marcus.schaefer@gmail.com>
 Build-Depends: debhelper-compat (= 13),
                dh-python,
                passwd,
+               pybuild-plugin-pyproject,
                python3-all,
                python3-anymarkup,
                python3-docopt,
                python3-lxml,
                python3-mock,
+               python3-poetry,
                python3-pytest,
                python3-pyxattr,
                python3-requests,
diff --git a/debian/rules b/debian/rules
index 437e959..dbe59d7 100755
--- a/debian/rules
+++ b/debian/rules
@@ -20,7 +20,7 @@ ifeq ($(filter nocheck nobench, $(DEB_BUILD_OPTIONS)),)
 endif

 override_dh_auto_build:
-       make tools
+#      make tools
        dh_auto_build
        make -C doc man

The build fails with:

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tasks/base_test.py::TestCliTask::test_load_xml_description - kiwi.exce...
FAILED tasks/base_test.py::TestCliTask::test_load_xml_description_buildservice
ERROR cli_test.py::TestCli::test_show_and_exit_on_help_request - kiwi.excepti...
ERROR cli_test.py::TestCli::test_get_servicename_system - kiwi.exceptions.Kiw...
ERROR cli_test.py::TestCli::test_warning_on_use_of_legacy_disk_type - kiwi.ex...
ERROR cli_test.py::TestCli::test_set_target_arch - kiwi.exceptions.KiwiComman...
ERROR cli_test.py::TestCli::test_get_servicename_image - kiwi.exceptions.Kiwi...
ERROR cli_test.py::TestCli::test_get_servicename_result - kiwi.exceptions.Kiw...
ERROR cli_test.py::TestCli::test_get_command - kiwi.exceptions.KiwiCommandNot...
ERROR cli_test.py::TestCli::test_get_command_args - kiwi.exceptions.KiwiComma...
ERROR cli_test.py::TestCli::test_load_command - kiwi.exceptions.KiwiCommandNo...
ERROR cli_test.py::TestCli::test_load_command_unknown - kiwi.exceptions.KiwiC...
ERROR cli_test.py::TestCli::test_load_command_undefined - kiwi.exceptions.Kiw...
ERROR cli_test.py::TestCli::test_get_command_args_not_loaded - kiwi.exception...
ERROR cli_test.py::TestCli::test_get_servicename_unknown - kiwi.exceptions.Ki...
ERROR tasks/image_info_test.py::TestImageInfoTask::test_process_image_info - ...
ERROR tasks/image_info_test.py::TestImageInfoTask::test_process_image_info_color_output
ERROR tasks/image_info_test.py::TestImageInfoTask::test_process_image_info_resolve_package_list
ERROR tasks/image_info_test.py::TestImageInfoTask::test_process_image_info_add_repo
ERROR tasks/image_info_test.py::TestImageInfoTask::test_process_image_info_delete_repos
ERROR tasks/image_info_test.py::TestImageInfoTask::test_process_image_info_help
ERROR tasks/image_info_test.py::TestImageInfoTask::test_process_image_info_print_xml
ERROR tasks/image_info_test.py::TestImageInfoTask::test_process_image_info_print_yaml
ERROR tasks/image_info_test.py::TestImageInfoTask::test_process_image_info_print_toml
ERROR tasks/image_resize_test.py::TestImageResizeTask::test_process_no_root_directory_specified
ERROR tasks/image_resize_test.py::TestImageResizeTask::test_process_no_raw_disk_found
ERROR tasks/image_resize_test.py::TestImageResizeTask::test_process_unsupported_size_format
ERROR tasks/image_resize_test.py::TestImageResizeTask::test_process_image_resize_gb
ERROR tasks/image_resize_test.py::TestImageResizeTask::test_process_image_resize_mb
ERROR tasks/image_resize_test.py::TestImageResizeTask::test_process_image_resize_bytes
ERROR tasks/image_resize_test.py::TestImageResizeTask::test_process_image_resize_not_needed
ERROR tasks/image_resize_test.py::TestImageResizeTask::test_process_image_resize_help
ERROR tasks/result_bundle_test.py::TestResultBundleTask::test_process_invalid_bundle_directory
ERROR tasks/result_bundle_test.py::TestResultBundleTask::test_process_result_bundle
ERROR tasks/result_bundle_test.py::TestResultBundleTask::test_process_result_bundle_as_rpm
ERROR tasks/result_bundle_test.py::TestResultBundleTask::test_process_result_bundle_with_bundle_format
ERROR tasks/result_bundle_test.py::TestResultBundleTask::test_process_result_bundle_with_bundle_format_from_commandline
ERROR tasks/result_bundle_test.py::TestResultBundleTask::test_process_result_bundle_name_includes_version
ERROR tasks/result_bundle_test.py::TestResultBundleTask::test_process_result_bundle_zsyncmake_missing
ERROR tasks/result_bundle_test.py::TestResultBundleTask::test_process_result_bundle_help
ERROR tasks/result_list_test.py::TestResultListTask::test_process_result_list
ERROR tasks/result_list_test.py::TestResultListTask::test_process_result_list_help
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_build
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_build_add_package
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_update_delete_package
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_build_prepare_stage_set_container_tag
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_build_add_container_label
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_build_add_container_label_invalid_format
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_build_prepare_stage_set_derived_from_uri
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_build_prepare_stage_set_repo
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_build_prepare_stage_add_repo
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_build_help
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_prepare_ignore_repos
ERROR tasks/system_build_test.py::TestSystemBuildTask::test_process_system_prepare_ignore_repos_used_for_build
ERROR tasks/system_create_test.py::TestSystemCreateTask::test_process_system_create
ERROR tasks/system_create_test.py::TestSystemCreateTask::test_process_system_create_help
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_run_debootstrap_only_once
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_add_package
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_delete_package
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_set_container_tag
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_add_container_label
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_add_container_label_invalid_format
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_set_derived_from_uri
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_set_repo
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_add_repo
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_help
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_delete_repos
ERROR tasks/system_prepare_test.py::TestSystemPrepareTask::test_process_system_prepare_delete_repos_used_for_build
ERROR tasks/system_update_test.py::TestSystemUpdateTask::test_process_system_update
ERROR tasks/system_update_test.py::TestSystemUpdateTask::test_process_system_update_add_package
ERROR tasks/system_update_test.py::TestSystemUpdateTask::test_process_system_update_delete_package
ERROR tasks/system_update_test.py::TestSystemUpdateTask::test_process_system_update_help
======= 2 failed, 1625 passed, 1 warning, 71 errors in 442.62s (0:07:22) =======

I assume I'm missing build dependencies. However, it's not very obvious which are missing.

Would be great to get some advise, so I can speedily update the package and fix the outstanding FTBFS bug [1].

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1069389

glaubitz commented 4 months ago

The newly released update to 10.0.16 fixed some the issues I was seeing, but I'm afraid the build still fails.

See the attached full build log.

kiwi_10.0.15-1_amd64.log

glaubitz commented 4 months ago

I attached the wrong build log yesterday, apologies. Attaching the correct one now.

kiwi_10.0.16-1_amd64.log

glaubitz commented 4 months ago

Quick guide on how to reproduce the issue in a Debian VM.

Prepare sbuild setup (as root):

# apt install sbuild git-buildpackage dh-python sphinx-common
# mkdir /srv/chroot
# sbuild-createchroot unstable /srv/chroot/unstable-amd64 --alias=sid http://deb.debian.org/debian
# sbuild-adduser $MYUSER

Continue as normal user from here.

Fetch and prepare kiwi source:

$ wget https://files.pythonhosted.org/packages/07/ab/488ec118e6572c0eb807de65f9548f4a8ae3122a257d24fefd43ab2a4330/kiwi-10.0.16.tar.gz -O kiwi_10.0.16.orig.tar.gz
$ git clone https://salsa.debian.org/debian/kiwi.git
$ cd kiwi
$ git checkout upstream && git checkout pristine-tar && git checkout master
$ gbp import-orig --pristine-tar ../kiwi_10.0.16.orig.tar.gz
$ export EMAIL=user@mail.com
$ gbp dch --auto

Apply this diff:

diff --git a/debian/control b/debian/control
index 9d5d69b..a4e7073 100644
--- a/debian/control
+++ b/debian/control
@@ -6,11 +6,13 @@ Uploaders: Marcus Schäfer <marcus.schaefer@gmail.com>
 Build-Depends: debhelper-compat (= 13),
                dh-python,
                passwd,
+               pybuild-plugin-pyproject,
                python3-all,
                python3-anymarkup,
                python3-docopt,
                python3-lxml,
                python3-mock,
+               python3-poetry,
                python3-pytest,
                python3-pyxattr,
                python3-requests,
diff --git a/debian/rules b/debian/rules
index 437e959..7e537c7 100755
--- a/debian/rules
+++ b/debian/rules
@@ -20,7 +20,6 @@ ifeq ($(filter nocheck nobench, $(DEB_BUILD_OPTIONS)),)
 endif

 override_dh_auto_build:
-       make tools
        dh_auto_build
        make -C doc man

Then build with:

$ gbp buildpackage --git-ignore-new --git-builder="sbuild -d unstable \
--arch=amd64 --arch-all --source --source-only-changes"