Blacklisting

[cromerc]

I think that we need a blacklist of packages that should not be imported directly from arch repos. Specifically packages that use things like configure --with-systemd

I just did an import from arch linux on a package "lib32-dbus", which is modified by us downstream to strip out systemd. Luckily I noticed that the import undid all our changes and I fixed it before I pushed, but other people might not be so lucky and accidentally bring systemd depends and files back in.

My idea would be that when artools sees a package that is in our blacklist it warns the user. Maybe it could create a new temp sub folder called "archpackage" which has arch's changes, we could then compare those files to our files, and then when done remove the archpackage folder and push our update.

[udeved]

I thought about a blacklist, but I figured, lists are bad to maintain, so I removed any auto batch/list importing. Having multiple uncommited packages gives headache if you need to pull before push. The problem with lists and bash is, that we can only sequentially access the array elements.

[cromerc]

I am open to other suggestions, but this could be a problem as we grow. Not everyone knows which packages need to be stripped of systemd and stay stripped of it.

[udeved]

I was more thinking along the lines of eventually creating more patches like here... https://github.com/artix-linux/artools/blob/devel/lib/util-pkg.sh#L30

The function runs on importing a pkg.

downside, we can't add all packages there we need to modify to get rid of systemd. some stuff is difficult to remove in automated manner, eg depends and such.

[artixnous]

How about a "universal" parser that seds things like "--with-systemd" into "--with-systemd=no" in configure, "libsystemd" into "libsystemd-dummy" in depends() etc? It should be trained to detect the real dependencies of specific package of course (e.g. elogind or eudev instead of systemd), but those could be placed on top and it would pay off in the long run.

[udeved]

@archnous

I have also had this thought. This is very difficult to implement, because the pkgbuilds change, they are not static. Say we had a set of patches that patched the imports, these patches likely would only work if the pkgbuild does not change anything in the functions.

Look at these few pkgs, that are patched at import... https://github.com/artix-linux/artools/blob/master/lib/util-pkg.sh#L30

I fear we'd be more busy updating and developing patches than maintaining the builds.

@cromerc

If you could create blacklists, one per git repo?

I've changed some import code to add a check if the package exists in arch tree. Could be enhances to also filter blacklisted package.

As for my working with unpatched arch builds, I use qgit to simply view the change as patch.

[udeved]

Another idea, we could also make trunk build a package, and if it leaves trunk towards a repo on build success, it only triggers deploypkg. A and M in trunk would buildpkg.

The pipeline code would perhaps change to something like

stage('Build') {
            environment {
                BUILDBOT_GPGP = credentials('BUILDBOT_GPGP')
            }
            when {
                expression { return  IS_BUILD == 'true' }
            }
            steps {
                dir("${PKG_PATH}") {
                    sh "buildpkg -r ${ADD_REPO}"
                }
            }
        }
        stage('Add') {
            when {
                AnyOf {
                    expression { return  IS_ADD == 'true' }
                    expression { return  currentBuild.buildStatus ==  'success' }
               }
            }
            steps {
                dir("${PKG_TRUNK}") {
                    sh "deploypkg -a -d ${ADD_REPO} -s"
                }
            }
        }

Currently, buildpkg is triggered on git file status A and M of a PKGBUILD in repos. A and M in trunk only do nothing, but A & M in repos do. trunk is filtered off.

[udeved]

Related to blacklisting and importing. We could also make it possible to import from AUR in a galaxy pkg trunk. It would tie buildtree to yaourt though.

[konimex]

Instead of tying buildtree to yaourt (which is an AUR wrapper), IMO it's better to clone the git repo, source the PKGBUILD and compare it to what we have.

[udeved]

@konimex

buildtree already import from arch git, it would be a yaourt -G $pkg call to get it in our git and use commitpkg like for other imports/updates

[konimex]

yaourt -G $pkg is a wrapper for git clone https://aur.archlinux.org/$pkg.git (in case of AUR) and https://git.archlinux.org/svntogit/$repo.git/snapshot/packages/$pkg.tar.gz (in case of ABS/Arch repos). I think it's not worth it to depend on a whole AUR wrapper just to fetch a PKGBUILD to our git repo.

[udeved]

@konimex

True, but we eventually also need a AUR search function. AUR is not really something I was gonna support with artools, but we may could do sort of AUR addon, that make some artools scripts AUR capable.

What I think we could really use, a dependency resolver for PKGBUILD. Afaik yaourt uses package-query for that. If we could determine the import/push order, we could automate the thing with cron job or so, minus eventual blacklisted packages.