[OLD docker] untergeek/curator `missing signature key` on server

sastorsl commented 1 year ago

Expected Behavior

That docker run untergeek/curator should start curator.

Actual Behavior

On RedHat Enterprise Linux 7 (RHEL7) getting the curator docker image fails:

Command:

docker run --rm untergeek/curator:8.0.3

output:

Unable to find image 'untergeek/curator:8.0.3' locally
Trying to pull repository untergeek/curator ... 
/usr/bin/docker-current: missing signature key.

Running the same on i.e. Ubuntu 22.04 LTS works as expected, so I expect that it is an issue with the version of RHEL7 docker.

Steps to Reproduce the Problem

Configure docker on RHEL7
Try and start undergeek/docker

Specifications

Operating system: Red Hat Enterprise Linux Server release 7.9 (Maipo)

Docker version, is the bundled docker version for RHEL7

Client:
 Version:         1.13.1
 API version:     1.26
 Package version: docker-1.13.1-209.git7d71120.el7_9.x86_64
 Go version:      go1.10.3
 Git commit:      7d71120/1.13.1
 Built:           Fri Jan  7 13:15:46 2022
 OS/Arch:         linux/amd64

Server:
 Version:         1.13.1
 API version:     1.26 (minimum version 1.12)
 Package version: docker-1.13.1-209.git7d71120.el7_9.x86_64
 Go version:      go1.10.3
 Git commit:      7d71120/1.13.1
 Built:           Fri Jan  7 13:15:46 2022
 OS/Arch:         linux/amd64
 Experimental:    false

Context (Environment)

One of our elasticsearch / kibana / logstash (ELK) platforms runs on RHEL7, and we use (used) curator to manage indices. Since the upgrade to ELK-8 curator has not worked with the old version of curator.

Detailed Description

It would be great if the docker build could support all major versions of the underlying docker client / server versions.

untergeek commented 1 year ago

I can't do that. It's not that I'm being dogmatic about it so much as the requirement to make and release both amd64 and arm64 packages in the same manifest requires me to use much newer versions. My Docker build process won't even build on 20.10.23 (see #1672) leave alone 1.13.1 or I would recommend doing the Docker build yourself and pushing to your own local repository.

I'd be happy to coach you through a few edits to the Dockerfile so you can build your own using 1.13.1 if you like. Even that process is tuned for multi-architecture builds and will not work out of the box with 1.13.1.

sastorsl commented 1 year ago

I understand.

With this issue others can at least find the reason for it, and the decision behind it.

I'll be AFK for a couple of weeks now but I'm very interested in building since we won't be sunsetting that solution anytime soon.

Should we follow up in this issue?

untergeek commented 1 year ago

Indeed. Any solution we can provide here will be of use to anyone else in the same situation.

sastorsl commented 1 year ago

Linking the Dockerfile here: https://github.com/elastic/curator/blob/master/Dockerfile

Would one need to fork your repo with the appropriate changes to Dockerfile, or can one take your image and "re-shape" it in some way?

untergeek commented 1 year ago

I will likely create a gist that you can use as a drop in replacement to start.

sastorsl commented 1 year ago

Did you get around to doing that gist?

untergeek commented 1 year ago

I have been out of office for family matters for most of the time we've been discussing this. I haven't had a chance yet.

untergeek commented 1 year ago

https://gist.github.com/untergeek/00dcb3f64cb0afa2f8fd194e9a5856d2

I can build and run Curator as a Docker image now in RHEL 7.9. I tested against an instance in Elastic Cloud and it works. That said, it is not a painless process, nor a quick fix.

Caveats include:

A lot of extra work to make a build possible (OpenSSL 1.1.1, Python, etc.). Fully explained in the gist.
Much larger image size as the concept of builder is not available in 1.13.1 (450MB vs 16.25MB). What this really means is that I have to build Curator and run it from the same Docker image, complete with all of the necessary Python files and dependencies, rather than just the executable and necessary libraries/dependencies. It's not a small difference.
Requires the use of the --privileged flag to read configuration files
Could be other things I haven't encountered yet.

untergeek commented 1 year ago

Note that the addition of OpenSSL 1.1.1t will not impact the rest of the system as I chose to install it at /opt/openssl.

wfhu commented 5 months ago

I have encountered the same problem, and I solved it by remove the old version Docker provided by CentOS 7's Default Repository and installed a new version of Docker

[root@ip-xxx]# cat /etc/redhat-release CentOS Linux release 7.9.2009 (Core) [root@ip-xxx]# uname -a Linux ip-xxx.ec2.internal 3.10.0-1160.119.1.el7.x86_64 #1 SMP Tue Jun 4 14:43:51 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

# docker version

Client: Version: 1.13.1 API version: 1.26 Package version: docker-1.13.1-210.git7d71120.el7.centos.x86_64 Go version: go1.10.3 Git commit: 7d71120/1.13.1 Built: Wed Mar 20 16:04:34 2024 OS/Arch: linux/amd64

Server: Version: 1.13.1 API version: 1.26 (minimum version 1.12) Package version: docker-1.13.1-210.git7d71120.el7.centos.x86_64 Go version: go1.10.3 Git commit: 7d71120/1.13.1 Built: Wed Mar 20 16:04:34 2024 OS/Arch: linux/amd64 Experimental: false

# docker build -t hello-world-nginx .

Sending build context to Docker daemon 3.072 kB Step 1/2 : FROM nginx:alpine Trying to pull repository docker.io/library/nginx ... missing signature key

# yum remove docker \
                              docker-client \
                              docker-client-latest \
                              docker-common \
                              docker-latest \
                              docker-latest-logrotate \
                              docker-logrotate \
                              docker-engine

# yum install -y yum-utils

# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

# yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# systemctl restart docker.service

# docker version

Client: Docker Engine - Community Version: 26.1.4 API version: 1.45 Go version: go1.21.11 Git commit: 5650f9b Built: Wed Jun 5 11:32:04 2024 OS/Arch: linux/amd64 Context: default

Server: Docker Engine - Community Engine: Version: 26.1.4 API version: 1.45 (minimum version 1.24) Go version: go1.21.11 Git commit: de5c9cf Built: Wed Jun 5 11:31:02 2024 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.33 GitCommit: d2d58213f83a351ca8f528a95fbd145f5654e957 runc: Version: 1.1.12 GitCommit: v1.1.12-0-g51d5e94 docker-init: Version: 0.19.0 GitCommit: de40ad0

Then docker pull works!

# docker build -t hello-world-nginx .

[+] Building 2.2s (7/7) FINISHED docker:default => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 94B 0.0s => [internal] load metadata for docker.io/library/nginx:alpine 0.3s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build context 0.0s => => transferring context: 182B 0.0s => [1/2] FROM docker.io/library/nginx:alpine@sha256:69f8c2c72671490607f52122be2af27d4fc09657ff57e4 1.4s => => resolve docker.io/library/nginx:alpine@sha256:69f8c2c72671490607f52122be2af27d4fc09657ff57e4 0.0s => => sha256:69f8c2c72671490607f52122be2af27d4fc09657ff57e42045801aa93d2090f7 9.07kB / 9.07kB 0.0s => => sha256:059cdcc5de66cd4e588b5f416b98a7af82e75413e2bf275f1e673c5d7d4b1afa 2.50kB / 2.50kB 0.0s => => sha256:70ea0d8cc5300acde42073a2fbc0d28964ddb6e3c31263d92589c2320c3ccba4 10.99kB / 10.99kB 0.0s => => sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8 3.41MB / 3.41MB 0.2s => => sha256:b1e69ebc7f924a03f4e1d3906db5423920d8b40d8f315db72445e6a7041c6237 3.99MB / 3.99MB 0.2s => => sha256:628158b45bceaf19d9e86fbfb08c925d75e1e2ab888cd9b97b7c8a8181232be4 628B / 628B 0.1s => => sha256:346e52e95fa0a52e495913d9d99e4766d1164631ddbf3a79b1b7860c44a4582a 957B / 957B 0.2s => => extracting sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8 0.1s => => sha256:e3227d68030df2f1c6db2654cf30f1e42d5700dc7b5c73eb1a4585bbd588affa 1.40kB / 1.40kB 0.2s => => sha256:8c50e1264d11b6f97944fb962f743063fbe75e06535780bb4919d491cf9ccde4 13.04MB / 13.04MB 0.5s => => sha256:dc3800d1d0f27990204f4c7f60ef0a8fdbf41a3199d38467475aba551823ccd4 1.21kB / 1.21kB 0.2s => => sha256:8c57fb1cd6448c27acb916942fed2522079e5256bc92466c1351f1b6d5f201e9 396B / 396B 0.3s => => extracting sha256:b1e69ebc7f924a03f4e1d3906db5423920d8b40d8f315db72445e6a7041c6237 0.3s => => extracting sha256:628158b45bceaf19d9e86fbfb08c925d75e1e2ab888cd9b97b7c8a8181232be4 0.0s => => extracting sha256:346e52e95fa0a52e495913d9d99e4766d1164631ddbf3a79b1b7860c44a4582a 0.0s => => extracting sha256:8c57fb1cd6448c27acb916942fed2522079e5256bc92466c1351f1b6d5f201e9 0.0s => => extracting sha256:dc3800d1d0f27990204f4c7f60ef0a8fdbf41a3199d38467475aba551823ccd4 0.0s => => extracting sha256:e3227d68030df2f1c6db2654cf30f1e42d5700dc7b5c73eb1a4585bbd588affa 0.0s => => extracting sha256:8c50e1264d11b6f97944fb962f743063fbe75e06535780bb4919d491cf9ccde4 0.4s => [2/2] COPY index.html /usr/share/nginx/html/ 0.4s => exporting to image 0.0s => => exporting layers 0.0s => => writing image sha256:eaae43e9fb19d7088bbb36b05613a3fa2f944aa0e9189c0d787c3d27817dd4a4 0.0s => => naming to docker.io/library/hello-world-nginx 0.0s

elastic / curator