search

rkt / rkt

[Project ended] rkt is a pod-native container engine for Linux. It is composable, secure, and built on standards.
Apache License 2.0
8.83k stars 887 forks source link

Download failure: coreos.com/rkt/stage1-coreos #3345

Open TerraTech opened 7 years ago

TerraTech commented 7 years ago

Environment bare metal deployment

rkt Version: 1.14.0
appc Version: 0.8.7
Go Version: go1.7.1
Go OS/Arch: linux/amd64
Features: -TPM +SDJOURNAL
--
Linux 4.7.3-coreos-r2 x86_64
--
NAME=CoreOS
ID=coreos
VERSION=1185.3.0
VERSION_ID=1185.3.0
BUILD_ID=2016-11-01-0605
PRETTY_NAME="CoreOS 1185.3.0 (MoreOS)"
ANSI_COLOR="1;32"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"
--
systemd 231
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS -ACL +XZ -LZ4 +SECCOMP +BLKID -ELFUTILS +KMOD -IDN

What did you do? I'm running rktnetes with the following kubelet options:

[..snip..]
Environment=KUBELET_VERSION=v1.4.5_coreos.0
Environment=KUBELET_ACI=quay.io/coreos/hyperkube
[..snip..]
--container-runtime=rkt \
--rkt-stage1-image=coreos.com/rkt/stage1-coreos \
[..snip..]

as recommended via: https://coreos.com/rkt/docs/latest/using-rkt-with-kubernetes.html

While trying to see why my rktnetes pods were stuck with status: ContainerCreating

20s       20s       1         tiller-deploy-488961084-olkca   Pod       spec.containers{tiller}   Warning   Failed    {kubelet 10.42.11.111}   Failed to create rkt container with error: failed to run [prepare --quiet --pod-manifest /tmp/manifest-tiller-deploy-488961084-olkca-133912879 --stage1-name=coreos.com/rkt/stage1-coreos]: exit status 1
stdout:
stderr: image: searching for app image coreos.com/rkt/stage1-coreos
image: remote fetching from URL "https://github.com/coreos/rkt/releases/download/vlatest/stage1-coreos-latest-linux-amd64.aci"
prepare: bad HTTP status code: 404

The kubernetes events pointed to being unable to download stage1-coreos I manually tested what kubelet was trying to download with rkt

#  rkt fetch coreos.com/rkt/stage1-coreos
image: searching for app image coreos.com/rkt/stage1-coreos
image: remote fetching from URL "https://github.com/coreos/rkt/releases/download/vlatest/stage1-coreos-latest-linux-amd64.aci"
image: keys already exist for prefix "coreos.com/rkt/stage1-coreos", not fetching again
image: downloading signature from https://github.com/coreos/rkt/releases/download/vlatest/stage1-coreos-latest-linux-amd64.aci.asc
fetch: bad HTTP status code: 404

What did you expect to see? 200 Success

What did you see instead? 404 NotFound

TerraTech commented 7 years ago

Should this be posted at coreos/bugs instead?

euank commented 7 years ago

This is a cross cutting issue; I think having it here is fine.

There are two issues here. 1) No latest tag is published for stage1 images; only specific versions are published (e.g. coreos.com/rkt/stage1-coreos:1.14.0 is published, :latest is not). AFAIK this is intentional.

2) The rktnetes instructions assume that the stage1 image was already fetched into the local store or, failing at that, was referenced with a valid version. Those docs should probably be improved to clarify that you'll either need to specify that version or have already loaded the stage1 image somehow (be it by rkt configuration or rkt fetch of a local file or what have you).

I think this can be fixed just by improving the documentation. Does that sound good to you?

TerraTech commented 7 years ago

@euank sounds good, thank you for the clarification.

As an aside, if I'm running rkt 1.14.0, is it ok to run stage1 images that are greater than current rkt version? If so, are there any downsides? e.g. latest right now is 1.18.0