Unable to locate package zeek-nightly" and "Package 'zeek' has no installation candidate"

[awelzel]

I've attempted to use the github-ci package feature from zeek/package-template for the AF_Packet plugin.

The zeek-nightly and zeek-feature builds fail during setup with the messages from the subject. Link to job: https://github.com/J-Gras/zeek-af_packet-plugin/runs/8244777211

I haven't looked closely, but it seems the implementation of this action is using the binary images from OBS installed into a Debian container. Could that maybe use the zeekurity/zeek images directly instead? Minimally it's a more direct supply chain with at least one third-party service removed :-)

---

Run zeek/action-zkg-install@v1
/usr/bin/docker run --name a9b10ce8b500e1642299868320f73449f61_1b8a4b --label 786a9b --workdir /github/workspace --rm -e "INPUT_ZEEK_VERSION" -e "INPUT_PKG" -e "INPUT_PKG_VERSION" -e "INPUT_PKG_SYSDEPS" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/zeek-af_packet-plugin/zeek-af_packet-plugin":"/github/workspace" 786a9b:10ce8b500e1642299868320f73449f61  "--pkg" "." "--pkg-version" "" "--pkg-sysdeps" "" "--zeek-version" "zeek"
Reading package lists...
Building dependency tree...
Reading state information...
Package zeek is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'zeek' has no installation candidate

Run zeek/action-zkg-install@v1
/usr/bin/docker run --name a9b885072b5de8f495ba5bae7d7ee9217fd_e2291f --label 786a9b --workdir /github/workspace --rm -e "INPUT_ZEEK_VERSION" -e "INPUT_PKG" -e "INPUT_PKG_VERSION" -e "INPUT_PKG_SYSDEPS" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/zeek-af_packet-plugin/zeek-af_packet-plugin":"/github/workspace" 786a9b:885072b5de8f495ba5bae7d7ee9217fd  "--pkg" "." "--pkg-version" "" "--pkg-sysdeps" "" "--zeek-version" "zeek-nightly"
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package zeek-nightly

[awelzel]

Digging a bit with @bbannier - he suspects that maybe the action could use a tag?

The job output output from the link are showing the Debian_10 repo to build the action's container, while the action's Dockerfile has moved on to Debian_11 now - so there's probably some unwanted caching going on.

...
  Step 7/12 : RUN echo 'deb http://download.opensuse.org/repositories/security:/zeek/Debian_10/ /'     | tee /etc/apt/sources.list.d/security:zeek.list
   ---> Running in 8b44fc94db4e
  deb http://download.opensuse.org/repositories/security:/zeek/Debian_10/ /
  Removing intermediate container 8b44fc94db4e
   ---> ac1db4fbb9ef
  Step 8/12 : RUN curl -fsSL -m 15 https://download.opensuse.org/repositories/security:zeek/Debian_10/Release.key     | gpg --dearmor     | tee /etc/apt/trusted.gpg.d/security_zeek.gpg > /dev/null
   ---> Running in e729f5a72286
  Removing intermediate container e729f5a72286
   ---> 0184f9f62f1f

https://github.com/zeek/action-zkg-install/blob/ada9cf7bd27faf98ef66afeba1d0ec451ce41115/Dockerfile#L15-L21

[awelzel]

Okay, I think I've understood now. It's not quite caching, rather than that an out-dated version of the action was used.

The v1 tag in this repository is pointing at a Dockerfile that's still using Debian_10. The newer v1.2 tag is pointing at one with Debian_11.

The action templates in the zeek/package-template repo use v1: https://github.com/zeek/package-template/blob/master/features/github-ci/.github/workflows/zeek-matrix.yml and therefore and outdated action is used.

From the action docs, it reads like the v1 tag should be moved forward to whatever is the current release (otherwise all repos using this action will need to update their tag to v1.2). I believe doing this in the repo should mostly fix the issue (?).

https://docs.github.com/en/actions/creating-actions/about-custom-actions#using-tags-for-release-management

Move the major version tag (such as v1, v2) to point to the Git ref of the current release. For more information, see "Git basics - tagging."

Moving the v1 tag like that implies that the version of the github action does not related to the Zeek version in any way. That's probably okay.

[ckreibich]

The action templates in the zeek/package-template repo use v1: https://github.com/zeek/package-template/blob/master/features/github-ci/.github/workflows/zeek-matrix.yml and therefore and outdated action is used.

Yeah, I simply forgot to reflect the action update in the template. My sense was that one generally has to stay on top of version updates in the actions one uses. For example, I recently noticed that I was using an outdated version of cibuildwheel over in the zeek-client CI. Dependapot looks handy for this.

Are you suggesting moving the v1 tag because it should align with v1.2? That wasn't my intended use, but perhaps it should have been v1.0 then to make that clear.

[bbannier]

Are you suggesting moving the v1 tag because it should align with v1.2? That wasn't my intended use, but perhaps it should have been v1.0 then to make that clear.

I think an issue here is partially that the Zeek repo doesn't provide stable versions, i.e., if one stays on a fixed minor version of the action eventually the action might break like it did for @awelzel here (that is: unless a complete image is build once and cached, but then a version like zeek-lts would forever be tied to whatever version LTS was when the image was built).

I think having the action provide zeek and zeek-lts is nice, but if I'd use this I'd much rather would like to specify versions like zeek-4, zeek-5.0 or zeek-5.0.1 instead (zeek-nightly is a different story).

[ckreibich]

Ah, I think I understand — much of this is about versioning of the Github action vs the Zeek version this pulls in, right? I fully agree that it shouldn't be required to bump the action version just to keep the Zeek install functional, and I disliked that I had to do that.

Fwiw, I don't think the action should still use the binary packages. I only did this because our Docker images weren't yet available. If we move this to using those images, we also automatically gain the freedom to pick a specific version, lts, latest, etc. Would that address your concerns?

[bbannier]

Fwiw, I don't think the action should still use the binary packages. I only did this because our Docker images weren't yet available. If we move this to using those images, we also automatically gain the freedom to pick a specific version, lts, latest, etc. Would that address your concerns?

Yes, I think that would work much better.

[awelzel]

Are you suggesting moving the v1 tag because it should align with v1.2? That wasn't my intended use, but perhaps it should have been v1.0 then to make that clear.

Ah, I think I understand — much of this is about versioning of the Github action vs the Zeek version this pulls in, right? I fully agree that it shouldn't be required to bump the action version just to keep the Zeek install functional, and I disliked that I had to do that.

Yes, so moving the tag instead of changing individual consumers was my thought. Given how v1 can't be used, v1.2 is semantically still v1 (very hand-wavy due to underlying Zeek versions changing) and with the official docs suggesting that too:

Move the major version tag (such as v1, v2) to point to the Git ref of the current release. For more information, see "Git basics - tagging."

I think having the action provide zeek and zeek-lts is nice, but if I'd use this I'd much rather would like to specify versions like zeek-4, zeek-5.0 or zeek-5.0.1 instead (zeek-nightly is a different story).

The nice thing about the zeek/zeek-lts is that as a package author you don't need to do anything to test against newer versions. If you explicitly provide the Zeek version, I would not be surprised to see stale repos over time that don't maintain this (which is also why I'm thorny around the v1.2 bump being on package authors).

Using the container images for a simpler/direct supply, yes, please. Then call this v2.0 ? Then decide whether the v2 tag would be moved up to v2.1 or v2.0.1 when we switch to use containers provided by ECR, for example. But in writing this, creating a v2.0.1 should certainly not result in users needing to update their repos to get v2.0.1. Which to me implies we would move minor and major tags v2.0 and v2 tags to v2.0.1.

[bbannier]

The nice thing about the zeek/zeek-lts is that as a package author you don't need to do anything to test against newer versions. If you explicitly provide the Zeek version, I would not be surprised to see stale repos over time that don't maintain this (which is also why I'm thorny around the v1.2 bump being on package authors).

We publish images for latest and lts which would directly map to the existing zeek and zeek-lts flavors, so I think translating this to image tags could support your use case as well. zeek-nightly would either a different approach I think though, or we need to publish that image as well.

[J-Gras]

Fully agreed that having zeek/zeek-lts automatically pointing to the latest version makes most sense.

I think having the action provide zeek and zeek-lts is nice, but if I'd use this I'd much rather would like to specify versions like zeek-4, zeek-5.0 or zeek-5.0.1 instead (zeek-nightly is a different story).

+1: I think the ability to provide specific versions in addition would be nice. This way package maintainers could easily verify compatibility for packages that should support multiple Zeek versions (e.g., zeek-community-id).