canonical / test_observer

Flutter based dashboard for visualising SRU regression test results
2 stars 2 forks source link

Artefacts conflating snap revisions from multiple channels based on version strings #100

Open Saviq opened 6 months ago

Saviq commented 6 months ago

Steps:

Expected:

Current:

Snap version strings are only meant to be human readable (contrary to debs). Test observer should not consider revisions not released to certain channels as part of the same artefact solely based on their version string. The channel on which the revision was published needs to be considered as well.

Saviq commented 6 months ago

One way to group snaps, as employed by snapcraft.io release pages, are builds ids:

$ yq '."image-info"' /snap/ubuntu-frame/current/snap/manifest.yaml
{
  "build-request-id": "lp-85100538",
  "build-request-timestamp": "2024-01-05T20:15:48Z",
  "build_url": "https://launchpad.net/~mir-team/+snap/ubuntu-frame-22-edge/+build/2346943"
}

If present, these could take precedence over version strings.

omar-selo commented 6 months ago

This is actually how Test Observer is currently designed. A snap artefact is unique based on the combination of name, version and track. Under a snap artefact there is one artefact build per architecture. This build is the latest revision seen for the specific architecture for this artefact (name, version and track combination). If I understand you correctly, you're suggesting snap artefacts to be uniquely identified using name, track and build-request-id instead of version? @jocave what do you think about this?

Saviq commented 6 months ago

based on the combination of name, version and track

Yeah but that's the thing - it merges runs for edge and beta, off of snap name and version string alone. In the example above r2 was never on beta, and yet that's the artefact it gets hooked up to.

Using build-request-id (if available, falling back to version) is somewhat orthogonal, has my vote - but I'll let you decide.

xnox commented 6 months ago

snap artefact unique identifier is "snap-name + snap revision id". In kernel team, "version" is never unique (as it is human readable version of the kernel portion of the snap only), and the same "version" is rebuild multiple times to stage updated initrd, updated firmware, updated packaging. Thus kernel team for all kernel snaps requires results by snap revision id, not version.

Especially when we do security resnap of existing kernel versions, with updated security releases of firmware files.