search

archlinuxcn / repo

Arch Linux CN Repository
1.48k stars 282 forks source link

[RFC][WIP] archlinuxcn-testing #976

Open yan12125 opened 5 years ago

yan12125 commented 5 years ago

问题类型 / Type of issues

受影响的软件包 / Affected packages

Make [archlinuxcn] friendly for [testing] users

Note: In this issue [testing] refers to all official [*testing] repos, including [testing], [community-testing] and [multilib-testing].

From time to time, when a library releases a new version ABI-incompatible with its previous versions, dependent packages should be rebuilt. Boost, ICU and Python are regular examples. In many cases, official rebuilt packages stay in [testing], [community-testing] and [multilib-testing] for a long time (e.g., a few days). During that time, users with [archlinuxcn] and [testing] repos enabled have their system in an inconsisten state - official packages are built against the new version of a library, while [archlinuxcn] packages are built against the previous version. I'd like to propose changes to [archlinuxcn] build infrastructure, including nvchecker, pre-commit, devtools-cn, lilac, archrepo2 and KISS2U/KISS2UI, so that a new repository [archlinuxcn-testing] can be established to play a similar role like official [testing] repos - storing packages built against the new version of ABI-incompatible libraries.

There are 3 different levels towards this goal:

  1. Build packages against [testing] manually and host built packages somewhere else
  2. Build packages against [testing] and upload them to a directory for [archlinuxcn-testing] on repo.archlinuxcn.org
  3. Check packages in [testing] and rebuild automatically

In the first level, packagers need to issue commands to trigger a build against [testing]. A possible design is using multiple build_prefix values in lilac.yaml. During 8-hourly normal builds, lilac use the first build_prefix, and other values can be run manually. For example, with the following lilac.yaml:

build_prefix:
  - extra-x86_64
  - testing-x86_64

Running ./lilac --build_prefix testing-x86_64 foobar will result in a package built using testing-x86_64. The packager can upload packages manually.

In the second level, lilac will accept a new directory layout similar to official packages:

┗ foobar
  ┗ archlinuxcn
  ┗ archlinuxcn-testing

And the build_prefix item will be a mapping from the build_prefix to the target repo:

build_prefix:
  - extra-x86_64: archlinuxcn
  - testing-x86_64: archlinuxcn-testing

With such a configuration file lilac will run extra-x86_64-build and testing-x86_64-build and put built packages to archlinuxcn and archlinuxcn-testing, respectively. pre_build and post_build function will accept an additional parameter build_prefix to do different action for different build prefixes (e.g., bump pkgrel once more for archlinuxcn-testing).

When the package in [testing] is moved to [core]/[extra]/[community], dependent packages in [archlinuxcn-testing] should be moved to [archlinuxcn]. Note that we should not rebuild packages in [archlinuxcn], or there will be signature checking failures for users with a cached version from [archlinuxcn-testing]. I have a script with a similar purpose (and similar usage with official db-scripts :D) at https://gitlab.com/yan12125/aur/blob/master/scripts/db-move.py.

(To be continued...)

ghost commented 5 years ago

Arch CN 源有无足够多的用户，来主动使用 testing 并提 bug？

yan12125 commented 5 years ago

@tobiichiamane 不太確定我們心中所想的[archlinuxcn-testing]是否相同。我把一部份的想法寫在第一個comment裡了。（全部內容太長了還沒整理完，先寫個2/3）

lilydjwg commented 5 years ago

What do we want out of this?

  1. a testing compatible repo
  2. handle ABI rebuilds smoothly (this won't work well because failed packages cannot block official testing move-outs.)

For the first one, we can use an overlay on current repo to provide the needed compatibility, provided that we have a mechanism to check if an archlinuxcn-testing package should be built (maybe through dependency analysis).

Work that is needed to make this happen:

yan12125 commented 5 years ago

The first one meets my goal. And thanks for listing some necessary changes. I'll finish my ideas and a rough solution soon.

to utilize current mirrors, we will have to put the testing repo inside the current repo, affecting some tools

How about puting the archlinuxcn-testing repo in another place and let mirrors decide whether to pull it or not?

lilydjwg commented 5 years ago

How about puting the archlinuxcn-testing repo in another place and let mirrors decide whether to pull it or not?

This is OK since we can expect testing users be more powerful.