Update Tectonic 1.7.5+tectonic.1 ➝ 1.7.9-tectonic.4 hangs

rbaumgar commented 6 years ago

Issue Report Template

Tectonic Version

Tectonic 1.7.5+tectonic.1

Environment

Vagrant 1.9.8 VirtualBox 5.1.32 r120294 OS RHEL 7.2

What hardware/cloud provider/hypervisor is being used with Tectonic? VirtualBox

Expected Behavior

try to update from 1.7.5+tectonic.1 to 1.7.9-tectonic.4

Actual Behavior

Reproduction Steps

vagrant up
login to console
administration - Cluster Settings
Check for Update - Start Upgrade...

Other Information

Log: I0305 10:28:06.869431 1 kube_version_operator.go:732] component daemonset/kube-system/kubelet does not exist in local cache I0305 10:28:06.869627 1 kube_version_operator.go:756] component deployment/tectonic-system/tectonic-ingress-controller does not exist in local cache I0305 10:28:17.169075 1 kube_version_operator.go:646] waiting for container images to be cached I0305 10:28:27.170050 1 kube_version_operator.go:663] all container images cached I0305 10:28:28.243097 1 kube_version_operator.go:577] Updating component kube-apiserver I0305 10:28:29.370631 1 kube_version_operator.go:597] Component daemonset/kube-system/kube-apiserver already updated, moving on I0305 10:28:30.378127 1 kube_version_operator.go:577] Updating component kube-scheduler I0305 10:28:31.608609 1 kube_version_operator.go:597] Component deployment/kube-system/kube-scheduler already updated, moving on I0305 10:28:32.615245 1 kube_version_operator.go:577] Updating component kube-controller-manager I0305 10:28:33.753655 1 kube_version_operator.go:597] Component deployment/kube-system/kube-controller-manager already updated, moving on I0305 10:28:34.769026 1 kube_version_operator.go:577] Updating component kube-proxy I0305 10:28:35.867620 1 kube_version_operator.go:597] Component daemonset/kube-system/kube-proxy already updated, moving on I0305 10:28:36.879036 1 kube_version_operator.go:577] Updating component Node Updater I0305 10:28:36.924891 1 kube_version_operator.go:597] Component on-host/no-namespace/kubelet already updated, moving on I0305 10:28:37.961248 1 kube_version_operator.go:577] Updating component tectonic-identity E0305 10:28:38.074304 1 kube_version_operator.go:380] error updating to 1.7.9+tectonic.4: Failed update of component: deployment/tectonic-system/tectonic-identity due to: error creating patch: patch: spec: template: spec: affinity: podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution:

podAffinityTerm: labelSelector: matchLabels: k8s-app: tectonic-identity topologyKey: kubernetes.io/hostname weight: 100

conflicts with changes made from original to current: metadata: annotations: deployment.kubernetes.io/revision: "1" generation: 1 selfLink: /apis/extensions/v1beta1/namespaces/tectonic-system/deployments/tectonic-identity uid: ebc59c06-205a-11e8-b40e-080027b424ef spec: minReadySeconds: 30 replicas: 2 selector: matchLabels: component: identity k8s-app: tectonic-identity strategy: rollingUpdate: maxSurge: 1 type: RollingUpdate template: metadata: name: tectonic-identity spec: affinity: null containers:

command:
- /usr/local/bin/dex
- serve
- /etc/dex/config.yaml imagePullPolicy: IfNotPresent livenessProbe: failureThreshold: 3 httpGet: path: /identity/healthz port: 5556 scheme: HTTP initialDelaySeconds: 5 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 name: tectonic-identity ports:
- containerPort: 5556 protocol: TCP
- containerPort: 5557 protocol: TCP resources: limits: cpu: 100m memory: 50Mi requests: cpu: 100m memory: 50Mi terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts:
- mountPath: /etc/dex name: config
- mountPath: /etc/tectonic-identity-grpc-server-secret name: tectonic-identity-grpc-server-secret readOnly: true dnsPolicy: ClusterFirst imagePullSecrets:
name: coreos-pull-secret restartPolicy: Always schedulerName: default-scheduler securityContext: runAsNonRoot: true runAsUser: 65534 terminationGracePeriodSeconds: 30 volumes:
configMap: defaultMode: 420 items:
- key: config.yaml path: config.yaml name: tectonic-identity name: config
name: tectonic-identity-grpc-server-secret secret: defaultMode: 420 secretName: tectonic-identity-grpc-server-secret

I0305 10:28:40.139874 1 watcher.go:45] setting up local caches I0305 10:28:41.146363 1 watcher.go:222] local caches populated I0305 10:28:41.212093 1 kube_version_operator.go:732] component daemonset/kube-system/kubelet does not exist in local cache I0305 10:28:41.212754 1 kube_version_operator.go:756] component deployment/tectonic-system/tectonic-ingress-controller does not exist in local cache

kbrwn commented 6 years ago

@rbaumgar This error is likely because you made a change to this manifest that needs to be reverted before the update can proceed.

rbaumgar commented 6 years ago

can you give me a tip what manifest I should have changed? In the 2nd and 3rd try I did unzip tectonic-sandbox-1.7.5-tectonic.1.zip vagrant up login upgrade failure...

slords commented 6 years ago

I can confirm the behavior described. I've not changed anything other than bringing up the sandbox and attempting to upgrade to the latest version.

danielgblanco commented 6 years ago

I can also confirm, still there upgrading Tectonic 1.7.5+tectonic.1 ➝ 1.7.14-tectonic.1 on a freshly installed sandbox.

robszumski commented 6 years ago

The sandbox is a slightly modified installation that was never intended to be used with auto-updates, only as a demo environment.

In addition, the sandbox has lagged in getting updates because it does diverge from production installs and is currently discontinued. I would recommend using AWS, Azure or the other platforms to properly test upgrades. Be sure to run at least two masters for the true experience!

rbaumgar commented 6 years ago

The reason why I tried the update the on the sandbox was, because I want to use the latest version and their has not been a current version for the sandbox been available... :-( :-( :-(

Mit freundlichen Grüßen / Kind regards

Robert Baumgartner

Senior Solution Architect Middleware

Red Hat Austria GmbH https://www.redhat.com

Millenniumtower 24.Stock

Handelskai 94-96

A-1200 Wien

robert.baumgartner@redhat.com M: +43-660-5913883 https://red.ht/sig

On Thu, May 24, 2018 at 6:23 PM, Rob Szumski notifications@github.com wrote:

The sandbox is a slightly modified installation that was never intended to be used with auto-updates, only as a demo environment.

In addition, the sandbox has lagged in getting updates because it does diverge from production installs and is currently discontinued. I would recommend using AWS, Azure or the other platforms to properly test upgrades. Be sure to run at least two masters for the true experience!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coreos/tectonic-forum/issues/284#issuecomment-391776005, or mute the thread https://github.com/notifications/unsubscribe-auth/ASxwS-uH92MEBJFNGGqgR5JO0w0ltQ0Xks5t1t57gaJpZM4ScFte .

coreos / tectonic-forum