search

osbuild / osbuild-composer

An HTTP service for building bootable OS images.
https://www.osbuild.org
Apache License 2.0
161 stars 107 forks source link

Ostree image definitions are now requiring a commit to be downloaded and that breaks manifest-db. #3195

Open lavocatt opened 1 year ago

lavocatt commented 1 year ago

See https://gitlab.com/redhat/services/products/image-builder/ci/manifest-db/-/jobs/3484838129

rhel_87-aarch64-edge_installer-boot Building only: FAIL: osbuild failed: Traceback (most recent call last):
  File "/usr/bin/osbuild", line 11, in <module>
    load_entry_point('osbuild==73', 'console_scripts', 'osbuild')()
  File "/usr/lib/python3.6/site-packages/osbuild/main_cli.py", line 157, in osbuild_cli
    manifest.download(object_store, monitor, args.libdir)
  File "/usr/lib/python3.6/site-packages/osbuild/pipeline.py", line 404, in download
    source.download(mgr, store, libdir)
  File "/usr/lib/python3.6/site-packages/osbuild/sources.py", line 41, in download
    reply = client.call_with_fds("download", args, fds)
  File "/usr/lib/python3.6/site-packages/osbuild/host.py", line 384, in call_with_fds
    raise error
osbuild.host.RemoteError: RuntimeError: rhsm consumer key and cert not found
   File "/usr/lib/python3.6/site-packages/osbuild/host.py", line 268, in serve
    reply, reply_fds = self._handle_message(msg, fds)
  File "/usr/lib/python3.6/site-packages/osbuild/host.py", line 301, in _handle_message
    ret, fds = self.dispatch(name, args, fds)
  File "/usr/lib/python3.6/site-packages/osbuild/sources.py", line 106, in dispatch
    self.download(SourceService.load_items(fds))
  File "/usr/lib/python3.6/site-packages/osbuild/sources.py", line 82, in download
    for _ in executor.map(self.fetch_one, *zip(*transformed)):
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 586, in result_iterator
    yield fs.pop().result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib64/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/osbuild/sources/org.osbuild.ostree", line 118, in fetch_one
    secrets = Subscriptions.get_consumer_secrets()
  File "/usr/lib/python3.6/site-packages/osbuild/util/rhsm.py", line 48, in get_consumer_secrets
    raise RuntimeError("rhsm consumer key and cert not found")
rhel_87-aarch64-edge_installer_with_users-boot Building only: FAIL: osbuild failed: Traceback (most recent call last):
  File "/usr/bin/osbuild", line 11, in <module>
    load_entry_point('osbuild==73', 'console_scripts', 'osbuild')()
  File "/usr/lib/python3.6/site-packages/osbuild/main_cli.py", line 157, in osbuild_cli
    manifest.download(object_store, monitor, args.libdir)
  File "/usr/lib/python3.6/site-packages/osbuild/pipeline.py", line 404, in download
    source.download(mgr, store, libdir)
  File "/usr/lib/python3.6/site-packages/osbuild/sources.py", line 41, in download
    reply = client.call_with_fds("download", args, fds)
  File "/usr/lib/python3.6/site-packages/osbuild/host.py", line 384, in call_with_fds
    raise error

We have several solutions possible for this:

This also brings the question of how we test the changes in the image definitions. I think we should trigger a build on manifest-db's CI every thime we change the image definition. And link the success of the DB generation as a prerequisite of the image definition's change approval. Maybe something automated, for a more manual start I've got a PR going on where I added a mechanism to test custom branches of composer on manifest-db see https://github.com/osbuild/manifest-db/pull/67.

I will temporarily do the third option in manifest-db while waiting for a better solution in 2023.

achilleas-k commented 1 year ago

These image types (edge installers and edge raw image) always needed a commit to be downloaded. What changed is that now these manifests are "valid" (in the strict sense) so they're not skipped by the validation check: https://github.com/osbuild/manifest-db/blob/4ce71bf6a1a2f07423e356ba58cdf8a726b77ef4/tools/osbuild-image-test#L208-L213

The relevant change is here: b9098c64c42531f8e101a91166ff3b4a7705eeb7

The parent value in the manifest needs to be a valid commit ID but I incorrectly added a ref when I initially created these manifests. In real scenarios, osbuild-composer resolves refs and includes only the ID/hash in the manifest. It wasn't an issue because we were never building them and the manifest-db check was added to skip them if they're detected as invalid.

I think it's better now that the manifests at least pass the osbuild validity check, even though they can't be built. We should have a different way of skipping the build and make validation errors fail the test. In other words, I think manifest-db's CI should produce errors if it finds a manifest that fails the osbuild validity checkers and we should probably add the same check to osbuild-composer's CI as well.

Moving forward, it would be nice to have a way to build these as well. We could have one sample commit for each distro version that we can reference in these manifests, as you mentioned. As a bonus, osbuild now supports building ostree native containers (https://github.com/osbuild/osbuild/pull/1091) which means we can encapsulate commits in containers and host them in a container registry so we wont need to serve it locally (which is a nice convenience).

lavocatt commented 1 year ago

I think it's better now that the manifests at least pass the osbuild validity check, even though they can't be built. We should have a different way of skipping the build and make validation errors fail the test. In other words, I think manifest-db's CI should produce errors if it finds a manifest that fails the osbuild validity checkers and we should probably add the same check to osbuild-composer's CI as well.

I agree, it's better. Let's work on something about this on the Manifest-db side.

Moving forward, it would be nice to have a way to build these as well. We could have one sample commit for each distro version that we can reference in these manifests, as you mentioned. As a bonus, osbuild now supports building ostree native containers (osbuild/osbuild#1091) which means we can encapsulate commits in containers and host them in a container registry so we wont need to serve it locally (which is a nice convenience).

That could be a nice addition too!