search

NCAR / container-dtc-nwp

End-to-end NWP systems in containers.
https://dtcenter.org/community-code/numerical-weather-prediction-nwp-containers
28 stars 13 forks source link

Fix keyserver error in base images #50

Closed mkavulich closed 2 years ago

mkavulich commented 2 years ago

The keyserver pool.sks-keyservers.net, which I had been using to verify the download of the "gosu" tool in the base images, is no longer active. So when trying to build the base image, this step fails. This is summarized in this issue:

This is mentioned in several places across the internet, including https://www.reddit.com/r/crypto/comments/o7oh4w/skskeyserversnet_pool_dns_records_disabled/ . Even though this keyserver has been deprecated for a while, there's a lot of old stale documentation pointing to it, including that which I originally based by entrypoint script on.

The fix is simply to point to an active, more reliable keyserver, hkps://keys.openpgp.org. This will fix the build issue, although it apparently remains to be seen what the future holds for key signing in this way.

Still unsure if this is related to #46 or not....but hopefully when this is fixed we will see more reliable behavior?

mkavulich commented 2 years ago

In addition to local testing on my Mac (MacOS Catalina 10.15.7, Docker Desktop 3.5.2.18, Docker version 20.10.7) Dockerhub tests for the base image and simple base image were also successful.

fossell commented 2 years ago

I will note that I tried one attempt of building base_image manually with Mike's changes in this PR, but it was unsuccessful. I did not dig to see what I may have done wrong. Using the Dockerhub latest was successful for building wps_wrf image.

fossell commented 2 years ago

I will note that I tried one attempt of building base_image manually with Mike's changes in this PR, but it was unsuccessful. I did not dig to see what I may have done wrong. Using the Dockerhub latest was successful for building wps_wrf image.

` => ERROR [2/4] RUN gpg --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 && curl -o /usr/local/bin/ 2.3s

[2/4] RUN gpg --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 && curl -o /usr/local/bin/gosu -SL "https://github.com/tianon/gosu/releases/download/1.13/gosu-amd64" && curl -o /usr/local/bin/gosu.asc -SL "https://github.com/tianon/gosu/releases/download/1.13/gosu-am" && gpg --verify /usr/local/bin/gosu.asc && rm /usr/local/bin/gosu.asc && rm -r /root/.gnupg/ && chmod +x /usr/local/bin/gosu:

5 0.294 gpg: directory `/root/.gnupg' created

5 0.296 gpg: new configuration file `/root/.gnupg/gpg.conf' created

5 0.296 gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run

5 0.296 gpg: keyring `/root/.gnupg/secring.gpg' created

5 0.296 gpg: keyring `/root/.gnupg/pubring.gpg' created

5 0.297 gpg: requesting key BF357DD4 from hkps server keys.openpgp.org

5 1.288 gpg: /root/.gnupg/trustdb.gpg: trustdb created

5 1.288 gpg: key BF357DD4: public key "Tianon Gravi tianon@tianon.xyz" imported

5 1.288 gpg: Total number processed: 1

5 1.288 gpg: imported: 1 (RSA: 1)

5 1.294 % Total % Received % Xferd Average Speed Time Time Time Current

5 1.294 Dload Upload Total Spent Left Speed

100 616 100 616 0 0 1923 0 --:--:-- --:--:-- --:--:-- 1925 100 2228k 100 2228k 0 0 3803k 0 --:--:-- --:--:-- --:--:-- 34.1M

5 1.886 % Total % Received % Xferd Average Speed Time Time Time Current

5 1.886 Dload Upload Total Spent Left Speed

100 9 100 9 0 0 32 0 --:--:-- --:--:-- --:--:-- 32

5 2.164 gpg: no valid OpenPGP data found.

5 2.164 gpg: the signature could not be verified.

5 2.164 Please remember that the signature file (.sig or .asc)

5 2.164 should be the first file given on the command line.

executor failed running [/bin/sh -c gpg --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 && curl -o /usr/local/bin/gosu -SL "https://github.com/tianon/gosu/releases/download/1.13/gosu-amd64" && curl -o /usr/local/bin/gosu.asc -SL "https://github.com/tianon/gosu/releases/download/1.13/gosu-am" && gpg --verify /usr/local/bin/gosu.asc && rm /usr/local/bin/gosu.asc && rm -r /root/.gnupg/ && chmod +x /usr/local/bin/gosu]: exit code: 2`

mkavulich commented 2 years ago

When this PR is merged, we will need to change the Dockerhub auto-build rules to point back to main.

fossell commented 2 years ago

I tried a clean manual build of the base image with this branch and was successful. docker system prune --all cd components/base docker build -t base_image .

fossell commented 2 years ago

I was able to build the wps_wrf image when pointing to my manual local build of the base_image. So given that, I'm fairly confident these mods and PR can be merged. I did have another error with building the branch using the dtcenter/base_image:latest on Dockerhub, which presumably points to Mike's fork with the fix now. But the errors are not reproducible in a meaningful way yet, so I don't think it's related to changes in this PR. Given my success with manual build, and others success as well, I think this can be merged.