Feature request: Update flag

[modem7]

Would it be possible to have an -update flag which either:

1. Checks latest version and if there is a new version, grabs from github automatically.
2. Failing 1, state that there is a new version, a clickable URL to download from github?
3. A Docker image which can be run with latest/versioned tag (basically, official version of https://hub.docker.com/r/markcaudill/runitor).

This is more so that we can keep up to date with things, especially if we've missed a release notification.

[bdd]

I don't expect to find time to implement this any time soon. If you're interested in contributing, here's what I have in mind.

I'd like this to be implemented in Go with no external dependencies. Other popular CLI tools that update themselves rely on hitting a well known URL to execute some installer script (deno, flyctl, etc). It's arduous to build something that allow the vendor to make live changes but retain artifact transparency. That's why I prefer the self update implementation to be embedded, so it's signed and whatever I ship is retained forever by proxy.golang.org's checksum tree.

It needs to implement this cautiously with download and atomic name swaps, on all supported platforms. For the first version, maybe we can leave Windows out, but it should run on all Unixes with proper tmpdir handling.

Interested in working on a PR?

[modem7]

Heya,

Unfortunately my skillset doesn't extend to writing programming languages, I'm an infrastructure/backend admin as a day job, so I wouldn't be the best choice to even start looking at how to implement the requirements.

But I certainly see where you're coming from in terms of your requirements for the task.

Would it be worthwhile keeping this open in case someone with the relevant skillsets wishes to take it on?

[bdd]

Would it be worthwhile keeping this open in case someone with the relevant skillsets wishes to take it on?

Of course. I'm not closing this until such feature gets added.

[bdd]

I found some time to work on building multi-architecture images with podman and pushed it to Docker Hub.

The images are currently based on Alpine only. I'm guessing some people would prefer to have at least Debian there.

As can be seen at https://hub.docker.com/r/runitor/runitor/tags, latest and 1.2.0 tags are manifests that point to the same amd64, arm64/v8, arm/v7, arm/v6.

I could really use some testers with access to various Linux or ARM hardware. I tested these images with binfmt and QEMU on my build machine. They source the correct base image and the runitor binary, and pass the smoke test of sending a test ping. ...But nothing like running it on the real hardware. So if you have access to a Raspberry Pi with Docker or Podman, please report back.

[bdd]

Basic check that image exists on Docker Hub.

% uname -sm
Linux x86_64
% podman run --rm -t docker.io/runitor/runitor:latest runitor -version
Trying to pull docker.io/runitor/runitor:latest...
Getting image source signatures
Copying blob 14ccca701fe5 skipped: already exists
Copying blob 63b65145d645 skipped: already exists
Copying config 66ffb492e6 done
Writing manifest to image destination
Storing signatures
runitor v1.2.0

Check it's a multi-architecture manifest

% podman manifest inspect docker.io/runitor/runitor:latest \
  | jq -r '
    .manifests[].platform
    | (if has("variant") then "/\(.variant)" else "" end) as $v
    |  "\(.os)/\(.architecture)\($v)"'
linux/arm64/v8
linux/arm/v7
linux/arm/v6
linux/amd64

Test a non-native image

% podman run --platform=linux/arm64/v8 --rm -t docker.io/runitor/runitor sh -c 'arch; apk -q add file; file /usr/bin/runitor; runitor -version'
aarch64
/usr/bin/runitor: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=AP67UzeWLESqZJOxbAc5/fyoSI7LOXeg1-zWd5GRP/BXtcZPhuBkuJ4paT63Gk/fCntjfLS0wzTgVbb1x4L, stripped
runitor v1.2.0

[modem7]

Can confirm it at least runs w/ Docker on x64.

If using Alpine, I'd recommend installing bash as a lot of scripts will probably not be using an ash shell.

[bdd]

Can confirm it at least runs w/ Docker on x64.

I tested the arm64 image on AWS's Graviton 2 and Gravition 3 machines. Works as intended. Though I doubt there are any runitor users there burning with the desire to have container image to run on their Arm EC2 boxes.

If using Alpine, I'd recommend installing bash as a lot of scripts will probably not be using an ash shell.

I feel like I should leave all that to the end user, because these images are simply intended to be a base image for whatever workload image people want to build. I'm guessing they are going to invoke some apk add one way or the other. Bash can go in there.

[cuu508]

I'm not sure if it is Docker Hub issue, or something else, but https://hub.docker.com/r/runitor/runitor/tags does not list any tags right now. And I get error messages about unknown manifest on the command line:

$ docker run --rm -t docker.io/runitor/runitor:latest runitor -version
Unable to find image 'runitor/runitor:latest' locally
docker: Error response from daemon: manifest for runitor/runitor:latest not found: manifest unknown: manifest unknown.

$ podman run --rm -t docker.io/runitor/runitor:latest runitor -version
Trying to pull docker.io/runitor/runitor:latest...
Error: initializing source docker://runitor/runitor:latest: reading manifest latest in docker.io/runitor/runitor: manifest unknown: manifest unknown

[bdd]

I'm not sure if it is Docker Hub issue, or something else, but https://hub.docker.com/r/runitor/runitor/tags does not list any tags right now.

It was a joint effort between Docker Hub and yours truly.

As I was iterating on my build script to make and push OCI images + manifests; I eventually hit Docker Hub's 100 pulls in 6 hours limit. I decided to call it a day and go to sleep.

It's all pushed now. https://hub.docker.com/repository/docker/runitor/runitor/tags?page=1&ordering=last_updated

[bdd]

Made some changes:

• Now building alpine, debian, and ubuntu based images from their latest tag.
  • One can track the latest runitor with just the base image name as a tag, e.g runitor:debian
  • runitor follows semver. So we can easily provide a MAJOR.MINOR tag as well for people who feel uneasy about not being able to properly bolt down their dependencies.
• runitor's latest points to the ubuntu:latest based image. I made decision entirely non-scientifically, based on the number of pulls from Docker Hub. Ubuntu seems to be the most popular image belonging to a distribution.
• I realized I misunderstood ENTRYPOINT in the base image is inherited (turns out it's not), so I added one to runitor image for pure convenience of smoke testing.

[cuu508]

I tested on Raspberry Pi 400 with its factory image. Both "latest" and "debian" tags give me certificate errors:

user@raspberrypi:~ $ uname -a
Linux raspberrypi 5.15.61-v7l+ #1579 SMP Fri Aug 26 11:13:03 BST 2022 armv7l GNU/Linux

user@raspberrypi:~ $ docker run --rm -t docker.io/runitor/runitor:latest -version
runitor v1.2.0

user@raspberrypi:~ $ docker run --rm -t docker.io/runitor/runitor:latest -uuid 7c175a71-e59c-4413-afeb-dd64d8b94ce7 -- uname -a
2023/03/29 16:34:36 Ping(start): Post "https://hc-ping.com/7c175a71-e59c-4413-afeb-dd64d8b94ce7/start?rid=547a04c2-3a90-4af2-98d5-dd79c600c923": x509: certificate signed by unknown authority
Linux 758ba248eccf 5.15.61-v7l+ #1579 SMP Fri Aug 26 11:13:03 BST 2022 armv7l armv7l armv7l GNU/Linux
2023/03/29 16:34:36 Ping(success): Post "https://hc-ping.com/7c175a71-e59c-4413-afeb-dd64d8b94ce7?rid=547a04c2-3a90-4af2-98d5-dd79c600c923": x509: certificate signed by unknown authority

user@raspberrypi:~ $ podman run --rm -t docker.io/runitor/runitor:latest -version
Trying to pull docker.io/runitor/runitor:latest...
Getting image source signatures
Copying blob aea1895b7fd0 done  
Copying blob 627e3c12b106 done  
Copying config be32a61e62 done  
Writing manifest to image destination
Storing signatures
runitor v1.2.0

user@raspberrypi:~ $ podman run --rm -t docker.io/runitor/runitor:latest -uuid 7c175a71-e59c-4413-afeb-dd64d8b94ce7 -- uname -a
2023/03/29 16:37:51 Ping(start): Post "https://hc-ping.com/7c175a71-e59c-4413-afeb-dd64d8b94ce7/start?rid=30f89f3e-0763-4ddc-8498-4ef970e331d4": x509: certificate signed by unknown authority
Linux be8733ef8de2 5.15.61-v7l+ #1579 SMP Fri Aug 26 11:13:03 BST 2022 armv7l armv7l armv7l GNU/Linux
2023/03/29 16:37:51 Ping(success): Post "https://hc-ping.com/7c175a71-e59c-4413-afeb-dd64d8b94ce7?rid=30f89f3e-0763-4ddc-8498-4ef970e331d4": x509: certificate signed by unknown authority

The alpine tag works:

user@raspberrypi:~ $ podman run --rm -t docker.io/runitor/runitor:alpine -uuid 7c175a71-e59c-4413-afeb-dd64d8b94ce7 -- uname -a
Trying to pull docker.io/runitor/runitor:alpine...
Getting image source signatures
Copying blob 6fb81ff47bd6 done  
Copying blob 8d87fffbdde4 done  
Copying config a79e694630 done  
Writing manifest to image destination
Storing signatures
Linux 0ae616352ccd 5.15.61-v7l+ #1579 SMP Fri Aug 26 11:13:03 BST 2022 armv7l Linux

[bdd]

I tested on Raspberry Pi 400 with its factory image. Both "latest" and "debian" tags give me certificate errors:

Bummer. For reasons I'm not certain about, debian and ubuntu base images doesn't ship with ca-certificates package. An issue still open since 2017 https://github.com/debuerreotype/docker-debian-artifacts/issues/15

I was happy about my single Containerfile for 3 distributions and 3 architectures that didn't require binfmt QEMU emulation. Now even if I add detect debian || ubuntu and run apt update; apt install -y ca-certificates, it will require emulation.

Let me see if there's a more elegant way out.

[bdd]

@cuu508 OK. I pushed new debian and ubuntu based manifests. I need to do qemu backed binfmt emulation while building these images now.

% for dist in debian ubuntu; do \
  podman run --network host -it docker.io/runitor/runitor:$dist -uuid <...> sh -c "echo hello from $dist" \
done
hello from debian
hello from ubuntu

[cuu508]

Awesome, all three variants work now:

user@raspberrypi:~ $ podman run --rm -t docker.io/runitor/runitor:latest -uuid 7c175a71-e59c-4413-afeb-dd64d8b94ce7 -- uname -a
Linux ab799a33ba52 6.1.19-v7l+ #1637 SMP Tue Mar 14 11:07:55 GMT 2023 armv7l armv7l armv7l GNU/Linux

user@raspberrypi:~ $ podman run --rm -t docker.io/runitor/runitor:debian -uuid 7c175a71-e59c-4413-afeb-dd64d8b94ce7 -- uname -a
Linux 8b27bffc1e6b 6.1.19-v7l+ #1637 SMP Tue Mar 14 11:07:55 GMT 2023 armv7l GNU/Linux

user@raspberrypi:~ $ podman run --rm -t docker.io/runitor/runitor:alpine -uuid 7c175a71-e59c-4413-afeb-dd64d8b94ce7 -- uname -a
Linux 504b173d4dd0 6.1.19-v7l+ #1637 SMP Tue Mar 14 11:07:55 GMT 2023 armv7l Linux

Also tested on 64-bit ARM system (Radxa Zero):

rock@radxa-zero:~$ podman run --rm -t docker.io/runitor/runitor:latest -uuid 7c175a71-e59c-4413-afeb-dd64d8b94ce7 -- uname -a
Linux 9195bab02744 5.10.69-13-amlogic-g104342c59952 #amlogic SMP PREEMPT Thu Sep 15 13:06:12 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

rock@radxa-zero:~$ podman run --rm -t docker.io/runitor/runitor:debian -uuid 7c175a71-e59c-4413-afeb-dd64d8b94ce7 -- uname -a
Linux 4e36e540c3dd 5.10.69-13-amlogic-g104342c59952 #amlogic SMP PREEMPT Thu Sep 15 13:06:12 UTC 2022 aarch64 GNU/Linux

rock@radxa-zero:~$ podman run --rm -t docker.io/runitor/runitor:alpine -uuid 7c175a71-e59c-4413-afeb-dd64d8b94ce7 -- uname -a
Linux 1d5d98b87592 5.10.69-13-amlogic-g104342c59952 #amlogic SMP PREEMPT Thu Sep 15 13:06:12 UTC 2022 aarch64 Linux

[bdd]

@modem7 https://github.com/bdd/runitor/commit/7935b789fb4546d7b31124f1ada681d87a866b69 merged and initial multi-arch image on Docker Hub, I'd like to close this issue, but continue keeping option 2 first and later option 1 in my possibly-to-do list.

Regarding option 1: I thought about it and wrote a straw man implementation for self updates; but I wasn't content with the dependency tree I was dragging in (go/x/crypto/ssh and ssh signature verification from sigstore along with its dep chain).

Implementing option 2 isn't a big deal, but instead of tying that feature to GitHub's mercy of not randomly taking down my account, I'd like to build it to either talk to Go Module Proxy or version check endpoint on a domain I control.

[modem7]

That sounds great!

Aye, for option two, a URL you control + a gpg key would certainly be a good solution!