search

ceph / shaman

source of truth for the state of repositories on Chacra nodes
8 stars 19 forks source link

Need way to get most recent ARM build reliably #138

Open BlaineEXE opened 1 year ago

BlaineEXE commented 1 year ago

I am trying to facilitate upstream quay.io/ceph/daemon-base and quay.io/ceph/ceph container builds that have regular ARM updates.

This PR (https://github.com/ceph/ceph-container/pull/2102) will do so, but requires some modifications to Shaman to reliably retrieve the latest ARM-compatible repo. To the best of my knowledge, we don't require bleeding edge ARM builds. From what I have seen looking into Shaman, the ARM builds happen with some regularity, and that should be sufficient for development of Rook on ARM systems (and I think cephadm as well).

Generally, I think curl -L https://shaman.ceph.com/api/repos/nfs-ganesha/next/latest/centos/8/flavors/ceph_main/repo should return the bleeding edge. However, if I specify ?arch=aarch64, I explicitly want the latest ARM build, and I should get a result. For ceph-container it should be okay if the result isn't the same as what would be returned for the equivalent x86 build.

As an example, this is what I saw on March 6, 2023: 

[
  {
    "status": "ready",
    "sha1": "91dd6865b71bbe99ad828c9c8bae1827922cd2a6",
    "extra": {
      "build_url": "https://jenkins.ceph.com/job/nfs-ganesha/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge/3427/",
      "root_build_cause": "TIMERTRIGGER",
      "version": "5-dev.3",
      "node_name": "172.21.2.6+braggi06",
      "job_name": "nfs-ganesha/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge",
      "package_manager_version": "54.34.3-"
    },
    "url": "https://2.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/",
    "distro_codename": null,
    "modified": "2023-03-04 01:05:01.711522",
    "distro_version": "8",
    "project": "nfs-ganesha",
    "flavor": "ceph_main",
    "ref": "next",
    "chacra_url": "https://2.chacra.ceph.com/repos/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/",
    "archs": [
      "source",
      "x86_64"
    ],
    "distro": "centos"
  },
  {
    "status": "ready",
    "sha1": "91dd6865b71bbe99ad828c9c8bae1827922cd2a6",
    "extra": {
      "build_url": "https://jenkins.ceph.com/job/nfs-ganesha/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge/3425/",
      "root_build_cause": "SCMTRIGGER",
      "version": "5-dev.3",
      "node_name": "172.21.2.6+braggi06",
      "job_name": "nfs-ganesha/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge",
      "package_manager_version": "54.34.3-"
    },
    "url": "https://3.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/",
    "distro_codename": null,
    "modified": "2023-03-03 23:21:03.852603",
    "distro_version": "8",
    "project": "nfs-ganesha",
    "flavor": "ceph_main",
    "ref": "next",
    "chacra_url": "https://3.chacra.ceph.com/repos/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/",
    "archs": [
      "x86_64",
      "source"
    ],
    "distro": "centos"
  }
]

Searching for arm returns no result

$ curl -s -L "https://shaman.ceph.com/api/repos/nfs-ganesha/next/latest/centos/8/flavors/ceph_main/repo?arch=aarch64"
<html>
 <head>
  <title>504 Gateway Timeout</title>
 </head>
 <body>
  <h1>504 Gateway Timeout</h1>
  The gateway has timed out.<br /><br />
no repository is ready for: nfs-ganesha/next

 </body>
</html>

Whereas searching for x86_64 does return a result:

curl -s -L "https://shaman.ceph.com/api/repos/nfs-ganesha/next/latest/centos/8/flavors/ceph_main/repo?arch=x86_64" 
[nfs-ganesha]
name=nfs-ganesha packages for $basearch
baseurl=https://2.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/$basearch
enabled=1
gpgcheck=0
type=rpm-md

[nfs-ganesha-noarch]
name=nfs-ganesha noarch packages
baseurl=https://2.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/noarch
enabled=1
gpgcheck=0
type=rpm-md

[nfs-ganesha-source]
name=nfs-ganesha source packages
baseurl=https://2.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/SRPMS
enabled=1
gpgcheck=0
type=rpm-md
andrewschoen commented 1 year ago

I talked with @BlaineEXE about this in chat. I think we'd need a patch in this controller here: https://github.com/ceph/shaman/blob/main/shaman/controllers/api/repos/sha1s.py#L9

I can work on a fix for this.

andrewschoen commented 1 year ago

@BlaineEXE have you tried the api/search/ endpoint? It has a way to specify the arch in the ?distros flag.

This is returning a repo for me: https://shaman.ceph.com/api/search/?project=nfs-ganesha&ref=next&sha1=latest&distros=centos/8/arm64&flavor=ceph_main

Would this help you out at all?

BlaineEXE commented 1 year ago

It has taken me way too long to get back to this!

The command used by ceph-container returns text that is put straight into a repo config file I guess I'd call it, like this:

[nfs-ganesha]
name=nfs-ganesha packages for $basearch
baseurl=https://2.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/$basearch
enabled=1
gpgcheck=0
type=rpm-md

[nfs-ganesha-noarch]
name=nfs-ganesha noarch packages
baseurl=https://2.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/noarch
enabled=1
gpgcheck=0
type=rpm-md

[nfs-ganesha-source]
name=nfs-ganesha source packages
baseurl=https://2.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/SRPMS
enabled=1
gpgcheck=0
type=rpm-md

From the query, I see something different:

❯ curl "https://shaman.ceph.com/api/search/?project=nfs-ganesha&ref=next&sha1=latest&distros=centos/8/arm64&flavor=ceph_main"
[{"status": "ready", "sha1": "9a84f015b86c5424ff7321a13754e5be4b77a6ac", "extra": {"build_url": "https://jenkins.ceph.com/job/nfs-ganesha/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge/3604/", "root_build_cause": "TIMERTRIGGER", "version": "5.2", "node_name": "172.21.4.66+confusa04", "job_name": "nfs-ganesha/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge", "package_manager_version": "5.25.05.0-"}, "url": "https://2.chacra.ceph.com/r/nfs-ganesha/next/9a84f015b86c5424ff7321a13754e5be4b77a6ac/centos/8/flavors/ceph_main/", "distro_codename": null, "modified": "2023-05-30 13:13:04.009631", "distro_version": "8", "project": "nfs-ganesha", "flavor": "ceph_main", "ref": "next", "chacra_url": "https://2.chacra.ceph.com/repos/nfs-ganesha/next/9a84f015b86c5424ff7321a13754e5be4b77a6ac/centos/8/flavors/ceph_main/", "archs": ["arm64", "source", "x86_64"], "distro": "centos"}]

The url field there is the repo index (https://2.chacra.ceph.com/r/nfs-ganesha/next/9a84f015b86c5424ff7321a13754e5be4b77a6ac/centos/8/flavors/ceph_main/) which has the RPMs but isn't the repo config file.

It looks like we could use that as the baseurl field in a repo config file though. @andrewschoen would you recommend that route? We can use jq to extract that.

BlaineEXE commented 1 year ago

I played around with using the url as the baseurl in a manually-created repo definition file, and it's working. This could be a good quality of life improvement but isn't necessary now that you helped figure out a better means of querying the Shaman API.