Closed hlin closed 1 month ago
With this patch, createrepo can recognize modules.yaml.gz and mdtype field (and modifyrepo) is not used, @lubomir can you take a look?
With this patch, createrepo can recognize modules.yaml.gz and mdtype field (and modifyrepo) is not used, @lubomir can you take a look?
That's a good point. Createrepo will match on the file name. Realistically the URL will point to some existing repodata, where the filename will match what createrepo expects. The mdtype in the lockfile is indeed not necessary, so cachi2 could drop it.
I confirm that this approach which is consistent with the rest of the lockfile based logic and behaviour works. I tested this with the following minimalistic lockfile and Fedora 38 (39+ dropped modularity):
---
lockfileVersion: 1
lockfileVendor: redhat
arches:
- arch: x86_64
packages:
- url: https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/38/Modular/x86_64/os/Packages/s/subversion-libs-1.14.2-5.module_f38+15272+edcc0b97.x86_64.rpm
repoid: fedora-modular
size: 1583542
checksum: sha256:b52f95619aa4ee70190fce7837985504550938fe0c0ebfba0f2ccd94b9a0a897
name: subversion-libs
- url: https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/38/Everything/x86_64/os/Packages/a/apr-1.7.2-2.fc38.x86_64.rpm
repoid: fedora
name: apr
size: 130140
checksum: sha256:ca4680178a8d7d5562dfbd361bc7caa13f8fc9a2756d337cf9ded77687e763ee
- url: https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/38/Everything/x86_64/os/Packages/a/apr-util-1.6.3-2.fc38.x86_64.rpm
name: apr-util
repoid: fedora
size: 97926
checksum: sha256:ecad94eb8493689ac063df1d4e141ea992996d1a1a73f7a6b77724d93b4063c0
- url: https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/38/Everything/x86_64/os/Packages/l/libserf-1.3.9-27.fc38.x86_64.rpm
name: libserf
repoid: updates
size: 57965
checksum: sha256:d04afa98de14d4a30ad8f4c335f0fa458ec89596a5bf6848718989ff03755363
- url: https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/38/Everything/x86_64/os/Packages/u/utf8proc-2.7.0-4.fc38.x86_64.rpm
name: utf8proc
repoid: fedora
size: 82281
checksum: sha256:5b4cdd58179acda1cbeedfbacba872d9e43f39c629486860420cf578b30c9a2c
metadata:
- url: https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/38/Modular/x86_64/os/repodata/a5b951a984ab6b7edd2c08d68162014d9f6ccfdbc4c721d318edca0a21697cb1-modules.yaml.gz
repoid: fedora-modular
mdtype: modules
size: 31546
checksum: sha256:a5b951a984ab6b7edd2c08d68162014d9f6ccfdbc4c721d318edca0a21697cb1
Then, after having run fetch --dev-package-mangers rpm
and inject-files cachi2-output --for-output-dir /tmp/
$ podman run -it --rm --network=none -v cachi2-output/deps/rpm/x86_64/repos.d/:/etc/yum.repos.d/:rw,Z -v cachi2-output/deps:/tmp/deps:rw,Z fedora:38 /bin/bash
# dnf module enable -y subversion:1.14
==========================================================================================================
Package Architecture Version Repository Size
==========================================================================================================
Enabling module streams:
subversion 1.14
Transaction Summary
==========================================================================================================
Complete!
# dnf install subversion-libs -y
Repository 'fedora-modular' is missing name in configuration, using id.
Repository 'fedora' is missing name in configuration, using id.
Repository 'updates' is missing name in configuration, using id.
Last metadata expiration check: 0:00:06 ago on Wed Aug 28 11:28:29 2024.
Dependencies resolved.
==========================================================================================================
Package Architecture Version Repository Size
==========================================================================================================
Installing:
subversion-libs x86_64 1.14.2-5.module_f38+15272+edcc0b97 fedora-modular 1.5 M
Installing dependencies:
apr x86_64 1.7.2-2.fc38 fedora 127 k
apr-util x86_64 1.6.3-2.fc38 fedora 96 k
libserf x86_64 1.3.9-27.fc38 updates 57 k
utf8proc x86_64 2.7.0-4.fc38 fedora 80 k
Transaction Summary
==========================================================================================================
Install 5 Packages
Total size: 1.9 M
Installed size: 5.6 M
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : apr-1.7.2-2.fc38.x86_64 1/5
Installing : apr-util-1.6.3-2.fc38.x86_64 2/5
Installing : libserf-1.3.9-27.fc38.x86_64 3/5
Installing : utf8proc-2.7.0-4.fc38.x86_64 4/5
Installing : subversion-libs-1.14.2-5.module_f38+15272+edcc0b97.x86_64 5/5
Running scriptlet: subversion-libs-1.14.2-5.module_f38+15272+edcc0b97.x86_64 5/5
Verifying : subversion-libs-1.14.2-5.module_f38+15272+edcc0b97.x86_64 1/5
Verifying : apr-1.7.2-2.fc38.x86_64 2/5
Verifying : apr-util-1.6.3-2.fc38.x86_64 3/5
Verifying : utf8proc-2.7.0-4.fc38.x86_64 4/5
Verifying : libserf-1.3.9-27.fc38.x86_64 5/5
Installed:
apr-1.7.2-2.fc38.x86_64 apr-util-1.6.3-2.fc38.x86_64
libserf-1.3.9-27.fc38.x86_64 subversion-libs-1.14.2-5.module_f38+15272+edcc0b97.x86_64
utf8proc-2.7.0-4.fc38.x86_64
Complete!
Can someone help to trigger the workflows?
Are there any more blockers to merge this PR?
We have a target end date of Sep 11 for this feature. Can this PR be merged then?
We have a target end date of Sep 11 for this feature. Can this PR be merged then?
Release is planned around 17th-18th, we'll get it in by then. @brunoapimentel or @taylormadore can we get one more ACK here?
/ok-to-test
DNF requires the module metadata in repodata to install modular packages, so that if there's any modular packages in the lock file, the module metadata file should be included too.
https://github.com/konflux-ci/rpm-lockfile-prototype/pull/33 generates the lock file with a new module_metadata field and then cachi2 should download the metadata files together with RPMs and
createrepo_c
can recognize the metadata files when generating repository.Maintainers will complete the following section
Note: if the contribution is external (not from an organization member), the CI pipeline will not run automatically. After verifying that the CI is safe to run:
/ok-to-test
(as is the standard for Pipelines as Code)