emerge-gitclone: always checks out HEAD because of 'kinvolk' git clone url vs. 'flatcar-linux' in release.xml

[rck]

Description

In a developer container emerge-gitclone always checks out HEAD because there is a mismatch between the repo name emerge-gitlclone generates and repo names in the online release.xml

Impact

As at always checks out HEAD it makes module building impossible/cumbersome as host kernel and kernel headers mismatch while the headers that get installed in the container obviously should match the flatcar release information.

Environment and steps to reproduce

• get current developer container (wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_developer_container.bin.bz2 -O flatcar_developer_container_current.bin.bz2)
• extract and and systemd-nspawn (bunzip2 -k flatcar_developer_container_current.bin.bz2 && sudo systemd-nspawn --image=flatcar_developer_container_current.bin)
• The source of the problem is /usr/lib/python-exec/python3.6/emerge-gitclone, diff with debug prints:

flatcardevelopercontainercurrent ~ # diff -Nur /usr/lib/python-exec/python3.6/emerge-gitclone{.orig,}
--- /usr/lib/python-exec/python3.6/emerge-gitclone.orig 2021-12-16 14:09:33.000000000 +0100
+++ /usr/lib/python-exec/python3.6/emerge-gitclone      2021-12-16 14:14:47.000000000 +0100
@@ -18,6 +18,7 @@
 commits = {}
 if release:
        manifest = "https://raw.githubusercontent.com/kinvolk/manifest/v%s/release.xml" % release
+       print("\nDEBUG: manifest-url:", manifest)
        try:
                from xml.dom.minidom import parseString as pxs
                from urllib import request as req
@@ -31,7 +32,11 @@
 for repo in portage.db[eroot]['vartree'].settings.repositories:
        if repo.sync_type != 'git':
                continue
-       commit = commits.get(repo.sync_uri.replace('//', '').replace('.git', '').split('/', 1)[-1])
+       repo_want = repo.sync_uri.replace('//', '').replace('.git', '').split('/', 1)
+       commit = commits.get(repo_want[-1])
+       print("\nDEBUG: repo want:", repo_want)
+       print("\nDEBUG: commits:", commits)
+       print("\nDEBUG: commit:", repo_want)

        # If commit points to a branch name starting with "refs/heads",
        # "refs/heads/flatcar-build-x", then we cannot simply get its

• emerge-gitclone rusults in:

DEBUG: manifest-url: https://raw.githubusercontent.com/kinvolk/manifest/v3033.2.0/release.xml

DEBUG: repo want: ['https:github.com', 'kinvolk/portage-stable']

DEBUG: commits: {'appc/acbuild': 'refs/heads/master', 'appc/spec': 'refs/heads/master', 'flatcar-linux/afterburn': 'refs/heads/mas
ter', 'flatcar-linux/baselayout': 'refs/heads/master', 'flatcar-linux/bootengine': 'refs/heads/master', 'flatcar-linux/coreos-clou
dinit': 'refs/heads/master', 'flatcar-linux/coreos-overlay': 'refs/tags/stable-3033.2.0', 'flatcar-linux/flatcar-dev-util': 'refs/
heads/master', 'kinvolk/docker': 'refs/heads/master', 'flatcar-linux/fero': 'refs/heads/master', 'flatcar-linux/grub': 'refs/heads
/master', 'flatcar-linux/ignition': 'refs/heads/master', 'flatcar-linux/init': 'refs/heads/master', 'flatcar-linux/locksmith': 're
fs/heads/master', 'flatcar-linux/mantle': 'refs/heads/master', 'flatcar-linux/mayday': 'refs/heads/master', 'flatcar-linux/nss-alt
files': 'refs/heads/master', 'flatcar-linux/portage-stable': 'refs/tags/stable-3033.2.0', 'flatcar-linux/scripts': 'refs/tags/stab
le-3033.2.0', 'flatcar-linux/sdnotify-proxy': 'refs/heads/master', 'flatcar-linux/seismograph': 'refs/heads/master', 'flatcar-linu
x/shim': 'refs/heads/master', 'flatcar-linux/toolbox': 'refs/heads/master', 'flatcar-linux/torcx': 'refs/heads/master', 'flatcar-l
inux/update-ssh-keys': 'refs/heads/master', 'flatcar-linux/update_engine': 'refs/heads/master', 'flatcar-linux/updateservicectl':
'refs/heads/master', 'rkt/rkt': 'refs/heads/master'}

DEBUG: commit: ['https:github.com', 'kinvolk/portage-stable']
Warning: No revision found for https://github.com/kinvolk/portage-stable.git
>>> Cloning repository 'portage-stable' from 'https://github.com/kinvolk/portage-stable.git'...
>>> Starting git clone in /var/lib/portage/portage-stable
Cloning into '/var/lib/portage/portage-stable'...
remote: Enumerating objects: 104525, done.
remote: Counting objects: 100% (96/96), done.
remote: Compressing objects: 100% (40/40), done.
remote: Total 104525 (delta 58), reused 87 (delta 56), pack-reused 104429
Receiving objects: 100% (104525/104525), 56.59 MiB | 21.79 MiB/s, done.
Resolving deltas: 100% (57576/57576), done.
>>> Git clone in /var/lib/portage/portage-stable successful

So it tries to find **kinvolk**/portage-stable, but the release.xml has **flatcar-linux**/portage-stable which can be seen in the debug output of commits, as well as in:

curl -s https://raw.githubusercontent.com/kinvolk/manifest/v3033.2.0/release.xml | grep portage-stable | tail -n1
  <project groups="minilayout" name="**flatcar-linux**/portage-stable" path="src/third_party/portage-stable" revision="refs/tags/stable-3033.2.0" upstream="refs/tags/stable-3033.2.0"/>

Expected behavior

emerge-gitclone checkout matches source of FLATCAR_RELEASE_VERSION.

Additional information

My assumption is that happened when switching github repos/orginisations between "kinvolk" and "flatcar-linux". I'm sure you know best if you want to change the clone url or change the manifest.xml.

[rck]

Is there some progress on that? It is pretty obvious that this is a bug, isn't it, do you need further information? It is just not obvious where to fix it, and if you want to fix the "flatcar-linux" part or the "kinvolk" part, that is something the project needs to decide, I can not make that decision and prepare a MR.

If somebody runs into the same issue, a hackish fix for now is:

commit = repo.sync_uri.replace('//', '').replace('.git', '').split('/', 1)[-1]
commit = commits.get(commit.replace('kinvolk','flatcar-linux'))

[krnowak]

Hi. Thanks for reporting this issue. This looks like a fallout from moving the repos from kinvolk to flatcar-linux org, and it looks to me that redirects that github installs are rather a mixed blessing here.

As a stop-gap measure, could you edit /etc/portage/repos.conf/coreos.conf inside the dev-container and replace kinvolk with flatcar-linux in sync-uri keys for both coreos and portage-stable sections? Would this change make things work again?

[rck]

As a stop-gap measure, could you edit /etc/portage/repos.conf/coreos.conf inside the dev-container and replace kinvolk with flatcar-linux in sync-uri keys for both coreos and portage-stable sections? Would this change make things work again?

I'd assume it from what I saw in the code, but also gave it a real test run which results in what I would expect.

>>> Git clone in /var/lib/portage/coreos-overlay successful
Note: switching to 'refs/tags/stable-3033.2.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 7f631ba50 Merge pull request #1488 from flatcar-linux/linux-5.10.84-flatcar-3033
>>> Release checkout refs/tags/stable-3033.2.0 in /var/lib/portage/coreos-overlay successful

[krnowak]

Thanks for confirming, this should be fixed in next release.