[arm64] Discourage or prevent installation of Rockons that are not available on the machines architecture

[mcbridematt]

The good news is that Rock-ons work happily on arm64 as long as the containers used have multi-arch manifests. If not, the user may not find this out until they attempt to install a Rockon. If the published Docker image only has a v1 (non-multiarch) manifest then Docker will pull the image regardless and the user will be greeted by an "exec-format-error". (Edit to further clarify: Docker will pull the correct image for the machines architecture, if one exists. So you can do docker run -i -t opensuse/leap and get the correct container for your system. The problem is that if the Docker registry only has a v1 (non-multiarch) image then it will go ahead, pull and attempt to run it)

(Sidenote: If a user really wants to run an x86 container on their arm64 system, or vice versa, they can use qemu-binfmt emulation)

Some possible ways this could be tackled:

• Housekeeping: Go through all the Rockons and figure out if an alternative image is available (I'll file an issue in rockon-registry)

For example, LinuxServer.io now publishes multi-arch images and would be an alternative for some applications where the official images do not support arm64.

• Flag supported architectures ahead of time in the Rockon registry (could this be automated?)
• Hide non-multiarch rockons by default if not on x86? (Users may wonder why a Rockon is missing)

I wrote up a script to check which Rockons will work on arm64: https://gist.github.com/mcbridematt/54d51ed813e6910b9244ce3c7d149e51

(This doesn't have to be run on an arm64 system - it merely checks the manifests to see if an arm64 container is published)

Rockon	Container Name	Arm64 supported?
Bitcoin	kylemanna/bitcoind	No
Bitwarden-rs	bitwardenrs/server:alpine	No
booksonic	linuxserver/booksonic	Yes
Cardigann	linuxserver/cardigann	No
collabora-online	collabora/code:latest	No
COPS	linuxserver/cops	Yes
CouchPotato	linuxserver/couchpotato	Yes
crashplan	jrcs/crashplan	No
Deluge	linuxserver/deluge	Yes
dropbox	janeczku/dropbox	No
DuckDNS	linuxserver/duckdns	Yes
Duplicati - by Linuxserver.io	linuxserver/duplicati:latest	Yes
ecoDMS 18.09	ecodms/allinone-18.09	No
Emby server	emby/embyserver	No
Folding@home Linuxserver.io	linuxserver/foldingathome:latest	No
FreshRSS	linuxserver/freshrss	Yes
Ghost	ghost	Yes
GitLab CE	gitlab/gitlab-ce	No
Gogs	gogs/gogs	No
Gollum	adtac/gollum-alpine	No
haproxy-letsencrypt	illi/docker-haproxy-letsencrypt:latest	No
HandBrake	jlesage/handbrake	No
Headphones	linuxserver/headphones	Yes
Home Assistant	homeassistant/home-assistant	Yes
HTTP to HTTPS redirect	geldim/https-redirect	No
Jackett	linuxserver/jackett	Yes
Jdownloader 2	jlesage/jdownloader-2	No
JenkinsCI	jenkins/jenkins:lts	No
Koel	binhex/arch-koel	No
LazyLibrarian	linuxserver/lazylibrarian	Yes
Medusa	medusa	Yes
Logitech Squeezebox	tdeckers/logitechmediaserver	No
MariaDB	linuxserver/mariadb	Yes
medusa	linuxserver/medusa	Yes
Minecraft	itzg/minecraft-server	No
Muximux	linuxserver/muximux	Yes
Mylar	linuxserver/mylar	Yes
Netdata	netdata/netdata	Yes
Nextcloud-Official	nextcloud	Yes
Nginx-proxy-manager	jlesage/nginx-proxy-manager	No
nginx	nginx:latest	Yes
Node-RED by nodered.org	nodered/node-red:latest	Yes
NZBGet	linuxserver/nzbget	Yes
NZBHydra	linuxserver/hydra	Yes
Ombi	linuxserver/ombi	Yes
OpenVPN	kylemanna/openvpn	No
OpenVPN	busybox	Yes
OwnCloud	pschmitt/owncloud:8.2.1	No
OwnCloud	postgres:9.5	Yes
OwnCloud-Official	owncloud	Yes
owncloudHTTPS	owncloud	Yes
Pi-Hole	pihole/pihole	Yes
Plex	linuxserver/plex	Yes
Plexpy	linuxserver/plexpy	No
PocketMine	magicalyak/pocketmine	No
PostgreSQL 9.5	postgres:9.5.21	Yes
PostgreSQL 10.6	postgres:10.6	Yes
Radarr	linuxserver/radarr	Yes
Resilio Sync	resilio/sync	No
Rocket.Chat	mongo	Yes
Rocket.Chat	rocketchat/rocket.chat	No
sabnzb	linuxserver/sabnzbd	Yes
Seafile	seafileltd/seafile	No
Sickbeard	linuxserver/sickbeard	No
Sickrage	linuxserver/sickrage	Yes
SmokePing	linuxserver/smokeping	Yes
Sonarr	linuxserver/sonarr	Yes
Subsonic	hurricane/subsonic	No
Syncthing	linuxserver/syncthing	Yes
Teamspeak3	teamspeak	No
TFTP server	pghalliday/tftp	No
Transmission - OpenVPN	haugene/transmission-openvpn	No
Transmission	dperson/transmission	Yes
Ubiquiti Unifi linuxserver.io	linuxserver/unifi-controller	Yes
Ubiquiti Unifi	jacobalberty/unifi	No
utorrent	dbarton/utorrent	No
Watchtower	v2tec/watchtower	No
Xeoma Video Surveillance	coppit/xeoma	No
YouTrack official	jetbrains/youtrack:2019.2.58831	No
Zabbix-XXL	busybox	Yes
Zabbix-XXL	monitoringartist/zabbix-db-mariadb	No
Zabbix-XXL	monitoringartist/zabbix-xxl	No
ZeroNet	nofish/zeronet	No
ZoneMinder	aptalca/zoneminder-1.29	No
ZoneMinder-1.30	magicalyak/docker-zoneminder	No
ZoneMinder-latest	dlandon/zoneminder	No

[phillxnet]

@mcbridematt This is a most welcome addition and excellent preparatory work.

Also way too many No entries there !! But enough Yes entries for sure.

Re:

• Flag supported architectures ahead of time in the Rockon registry (could this be automated?)

Yes I think that's the way to go, along with some automation if we can. And add to our Rock-on pull request template that we favour multi arch images and to add the new proposed element (see below):

https://github.com/rockstor/rockon-registry/pull/196

• Hide non-multiarch rockons by default if not on x86? (Users may wonder why a Rockon is missing)

I think we are agreed on this, it's going to be confusing if folks see Rock-ons available in one instance and not in another, different arch, instance. I'm sort of favouring a Rock-on definition extension myself. I know it's hard coded and potentially a maintenance wack-a-mole, but if we used something like your script to 'monitor' the JSON element re architecture with current state of the image referenced we might be able to have periodic updates of this automated, or just flagged as in need of update. But back to the point, I think it would be good to have the install button greyed out for non compatible options and some text indicating it's "Not (arch-here) compatible" type thing when the JSON element doesn't contain the instance machine architecture. That way a user still has all the info and links to report this incompatibility/unavailable nature to the docker authors say.

And if we make the new JSON element optional it eases this features introduction, ie we don't have to back port to all definitions. But given your preparatory work here it may not be all that bad, but still, it would make for a more robust solution.

@FroggyFlox as your the main Rock-ons management code contributor, this design decision would be in your ball park as it goes.

Personally I'd favour a retro fit of a new arch element that lists compatible architectures and we add code that can handle this element not being found that defaults to 'finding' x86_64. That way we play safe and reduce the transition work by only having o update say aarch64 compatible Rock-ons first.

Nice script by the way, thanks for sharing it. I could add the like of this to our backend management systems to flag changes in compatibility re the proposed element drifting from reality, We might even pop this in as a GitHub Action. Bit of a scatty arrangement all in but would work and would be fairly quick to setup, which is important in this case as arm64 is important to our immediate future and Rock-ons are a key element of Rockstor's bag.

Comments on how we best proceed are most welcome. But lets keep things as simple as possible as it needs to be 'doable' with a limited human resource. Hence the simple additional JSON element idea as likely once an image is arm64 available it's likely to stay that way in virtually all cases.

[FroggyFlox]

Thanks a lot, indeed, @mcbridematt !

@phillxnet , thanks for all your details as well... I'll try to bring some additional input on that.

I agree with you that hiding an incompatible rock-on would lead to more problems than anything. Similarly, we need to keep compatibility in rock-on definition current rock-on definition files.

I do concur with the idea of providing a feedback on the UI itself:

• greying out the install button (with explicit tooltip explaining why) is a requirement in my opinion as in the end the installation of an incompatible rock-on shouldn't be attempted.
• we could also consider adding some sort of icons showing platform compatibility (or even github-style badges). That might be in a later time, though, and maybe part of the re-work of our rock-ons catalog display we discussed a few times (https://github.com/rockstor/rockstor-core/issues/2015 and https://github.com/rockstor/rockon-registry/issues/31)

Regarding how to do it:

• we could, indeed, add logic based on @mcbridematt 's script to our current rock-ons update process to feed platform compatibility into the database and use that to direct the UI elements discussed above. This might even remove the need to request platform information for each json definition file, actually, as that would be automated.
• a difficulty I might see with using the script within Rockstor is that it seems to be python3 dependent. We could switch to using our run_command() wrapper, but I don't think we are geared to specifying custom env for the process in question so that might cause a problem as we need that to activate the docker experimental features needed to use docker manifest.
• in this context, your approach of using a json element that defaults to x86_64 seems the easiest and most compatible approach.

[mcbridematt]

• a difficulty I might see with using the script within Rockstor is that it seems to be python3 dependent. We could switch to using our run_command() wrapper, but I don't think we are geared to specifying custom env for the process in question so that might cause a problem as we need that to activate the docker experimental features needed to use docker manifest.

I'm not sure if it is just me, but 'docker manifest' does seem to take quite a while to grab the manifest information for each container image (about 5-10 seconds according to my stopwatch, on two different systems), so it may not be practical to run the architecture discovery on the 'client' (Rockstor install) side.

There might be a faster way to do it (given there is no such delay when pulling arm versions of multi-arch images) but I'm not familiar enough with the Docker ecosystem to know.

I agree indicating architecture availability in the JSON would be the best way. A year or two ago I would have suggested allowing a per-architecture container image override (allowing someone to recompile and rehost ARM versions OR cases where arm64 images are distributed as :arm64 tags) but I think there is enough momentum with multi-arch that the gap will narrow organically.

[phillxnet]

@mcbridematt & @FroggyFlox Re

... so it may not be practical to run the architecture discovery on the 'client' (Rockstor install) side.

So we seem agreed that this would be pushing our luck, plus we have an outstanding 'sloppiness' where we pull docker info from our own repo way too often, apparently on each page refresh in some place. So we should really track that one down first and I doubt docker hub will like each and every one of our clients polling all our Rock-ons info and as stated, it just doesn't look practical from the observed timings (or polite really). So if we are, apparently, agreed on a Rock-on JSON extension that is optional, so we can piecemeal add it retrospectively to our growing list of Rock-ons, how fancies taking this one on. I'm afraid I wont be able to do it myself for a bit however as I've got a verging on unhealthy backlog to sort our re-release out.

So if anyone fancies having a go at this, with the UI as indicated by @FroggyFlox with the basic function of blocking Rock-ons install by architecture, where that arch can be extended, but starting with allowing all on x86_64 and checking first if the JSON definition has AArch64 (our only other architecture planned currently) but with the capability to extend to other architectures, then that would be dandy.

The most obvious additional JSON element in the Rock-on definition I'm thinking would be an Array so we can interpret in Python as a list and do a simple platform.machine() member check. But as this is a do-ocracy (within reason) the implementation is down to the doer :). What ever works, and fits nicely into the existing format, and is robust really. But that would seem to be the most basic approach and getting a working minimum implementation based on an extendable format looks like a plan. Once the format is sorted I can do some 'leg work' on retro fitting if need be.

And if we are in the future able to retrieve reliable, and politely (with negligible overhead) the image definition, then we can always deprecate the JSON element and go a new way.

[phillxnet]

@mcbridematt @FroggyFlox As of https://github.com/rockstor/rockon-registry/pull/266 "Add new Netdata (official) rock-on" our replacement Netdata Rock-on is now Aarch64 comaptible. I'll edit the original post accordingly.

[phillxnet]

@mcbridematt @FroggyFlox & @Hooverdan96

Noting for context here that as from @Hooverdan96 work here:

pr: '309 align architecture tags' https://github.com/rockstor/rockon-registry/pull/310 for issue: 'Standardise architecture compatibility reports' https://github.com/rockstor/rockon-registry/issues/309

All existing and future Rock-ons have their at-time-of-publish architecture compatibility within their description. Not ideal but a step up form our prior situation of completely unknown architecture compatibility, and scatty inconsistent descriptions of the same.

I'll try and update the above table shortly, as we now have around 80 Rock-ons with the vast majority arm64 as well as amd64 compatible.

[phillxnet]

Just wanted to raise the profile of this issue again. @Hooverdan96 clarification in descriptions is a step in the right direction, along with asserting this expectation in our README.md, but we have a little more to do on this front. Likely explicitly identifying via the Rock-on definition is my thinking. Allowing then for insertion in description (or better) and enabling an auto filter to show only compatible Rock-ons.

[Hooverdan96]

would that be best done by introducing a separate tag in the json (with the subsequent database changes, views)? If yes, we should probably bundle that with discussions like exposing the image version tag (I unfortunately have a hard time finding where I/we logged this discussion some time ago) to reduce e.g. having explicit postgresql (or other db Rock-ons) for each version, but the user being able to pick up a version during install. Not sure there are other tags that were requested for other reasons ...

I would assume in general, the effort would be less adding multiple (agreed upon) at one time vs. separating them all out? Though, also considering the one PR for one issue ... well, in any case, wanted to throw that out there, since we're so capacity constrained as it is.

[phillxnet]

@Hooverdan96 Re:

done by introducing a separate tag in the json (with the subsequent database changes, views)?

Yes, that was my thinking. But I think introducing more changes that are just in-the-area are an over-stretch/scope creep really. Easier to manage smaller chunck was my thinking. And in this case we just have to transfer our descriptions to json tags. However @mcbridematt python script is curious: but again this could be used off-line as it were to just update this repo. Maybe by way off a GitHub task: once we have the new tag in place that is.

The related but tangential question of latest/etc is I think best approached seperately as it's mostly unrelated, but not entirely of course as some docker tag may have arm64 and another not: but currently we have nothing to inform the Web-UI what to address other than the excellent description additions you did more recently. So just moving them to an official tag and is complex enough given backward compatibility: but you have now normalized them all so we could just do a stip of description if an arch tag is provided: to avoid duplication and enable older Rockstors to benefit from your prior description work.