Open sarveshtamba opened 5 years ago
Hi, any help on this one would be greatly appreciated. Thanks in advance.
@ia3andy are you the right person for this issue? For fabric8/launcher-operator deployment I am facing "CrashLoopBackOff" for the pods.
In the file "deploy/operator.yaml" the image used is as below:- image: fabric8/launcher-operator:latest However, the latest fabric8/launcher-operator image has ARCHITECTURE=amd64 and OS=linux(https://hub.docker.com/r/fabric8/launcher-operator/tags), so we might need to build the image for ppc64le first and then continue the deployment using the built image.
Can you please confirm if this is the correct understanding?
I tried to build the image for ppc64le using steps in Develop section and it got built successfully with the operator-sdk build command.
[root@p006vm32 launcher-operator]# operator-sdk build fabric8/launcher-operator:9.1.2
INFO[0029] Building OCI image fabric8/launcher-operator:9.1.2
Sending build context to Docker daemon 183.2 MB
Step 1/5 : FROM alpine:3.8
Trying to pull repository registry.redhat.io/alpine ...
Trying to pull repository docker.io/library/alpine ...
3.8: Pulling from docker.io/library/alpine
0c6396bb25a4: Pull complete
Digest: sha256:04696b491e0cc3c58a75bace8941c14c924b9f313b03ce5029ebbc040ed9dcd9
Status: Downloaded newer image for docker.io/alpine:3.8
---> 18052731e50a
Step 2/5 : RUN apk upgrade --update --no-cache
---> Running in 846ac8e4d77c
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/ppc64le/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/ppc64le/APKINDEX.tar.gz
(1/2) Upgrading musl (1.1.19-r10 -> 1.1.19-r11)
(2/2) Upgrading musl-utils (1.1.19-r10 -> 1.1.19-r11)
Executing busybox-1.28.4-r3.trigger
OK: 5 MiB in 13 packages
---> 7216824410f3
Removing intermediate container 846ac8e4d77c
Step 3/5 : USER nobody
---> Running in 5381649be74c
---> c8052cd77c4e
Removing intermediate container 5381649be74c
Step 4/5 : ADD templates/*.yaml /usr/local/bin/templates/
---> 509ce5fd7e1e
Removing intermediate container a7e734a74387
Step 5/5 : ADD build/_output/bin/launcher-operator /usr/local/bin/launcher-operator
---> 32c8c0c50ded
Removing intermediate container fce538cd38f4
Successfully built 32c8c0c50ded
INFO[0058] Operator build complete.
[root@p006vm32 launcher-operator]# docker images | grep fabric8
fabric8/launcher-operator 9.1.2 32c8c0c50ded 40 seconds ago 48.7 MB
[root@p006vm32 launcher-operator]# oc get all
NAME AGE
task.tekton.dev/echo-hello-world 5d
NAME SUCCEEDED REASON STARTTIME COMPLETIONTIME
taskrun.tekton.dev/echo-hello-world-task-run False TaskRunTimeout 5d 5d
[root@p006vm32 launcher-operator]# vi ./deploy/operator.yaml
[root@p006vm32 launcher-operator]# docker inspect fabric8/launcher-operator:9.1.2
[
{
"Id": "sha256:32c8c0c50dedc9a1b6dd46356c16d72f07995ad7ff517757e2095cfe0b5cc2be",
"RepoTags": [
"fabric8/launcher-operator:9.1.2"
],
"RepoDigests": [],
"Parent": "sha256:509ce5fd7e1e0d458d358237b2f7669db7d7ca580b23707011b6235f918a159e",
"Comment": "",
"Created": "2019-09-17T08:50:00.142287463Z",
"Container": "fce538cd38f476125626cf5c72694dd2a3f596033c9b1b064bad731f18595f18",
"ContainerConfig": {
"Hostname": "846ac8e4d77c",
"Domainname": "",
"User": "nobody",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh",
"-c",
"#(nop) ADD file:30fe9b1b4f59428969888e33d2331e7501243691f08027c5152eee27b36e8eae in /usr/local/bin/launcher-operator "
],
"ArgsEscaped": true,
"Image": "sha256:509ce5fd7e1e0d458d358237b2f7669db7d7ca580b23707011b6235f918a159e",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": [],
"Labels": {}
},
"DockerVersion": "1.13.1",
"Author": "",
"Config": {
"Hostname": "846ac8e4d77c",
"Domainname": "",
"User": "nobody",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh"
],
"ArgsEscaped": true,
"Image": "sha256:509ce5fd7e1e0d458d358237b2f7669db7d7ca580b23707011b6235f918a159e",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": [],
"Labels": {}
},
"Architecture": "ppc64le",
"Os": "linux",
"Size": 48669181,
"VirtualSize": 48669181,
"GraphDriver": {
"Name": "overlay2",
"Data": {
"LowerDir": "/var/lib/docker/overlay2/98126c7461e113722f9ddb7937416b715ce5d339ef46cfb071ae2970dab871c1/diff:/var/lib/docker/overlay2/a37e26a1d02dabfc9e400bedd39bda8296bdaaf4fec6270f6146c6af88131891/diff:/var/lib/docker/overlay2/ad6c31ab9142e321fd07ca21dd0979a643ca776adc041be509916e23a126fef4/diff",
"MergedDir": "/var/lib/docker/overlay2/003c5dcf1bfc0a20781157a21fe66c3d81022760749671c4f053d984685c493d/merged",
"UpperDir": "/var/lib/docker/overlay2/003c5dcf1bfc0a20781157a21fe66c3d81022760749671c4f053d984685c493d/diff",
"WorkDir": "/var/lib/docker/overlay2/003c5dcf1bfc0a20781157a21fe66c3d81022760749671c4f053d984685c493d/work"
}
},
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:c90acc73c11e8a38a35855cd566e904cf7b13d4a3958e6ada2e85529e9fadba4",
"sha256:70cfd04ab5777e8625ad2543d3870e6610ad36a89313bf3bd2faa986663015fd",
"sha256:931bb96b156355889a233194eb0b20383e54ab1d57841329d5c8161afeb70a5e",
"sha256:39e9463790e6b8bce6c766c35d196a6fcaa2f93d2d8fcced4f15789bd9265680"
]
}
}
]
[root@p006vm32 launcher-operator]# oc create -R -f ./deploy
customresourcedefinition.apiextensions.k8s.io/launchers.launcher.fabric8.io created
deployment.apps/launcher-operator created
role.rbac.authorization.k8s.io/launcher-operator created
rolebinding.rbac.authorization.k8s.io/launcher-operator created
serviceaccount/launcher-operator created
[root@p006vm32 launcher-operator]# oc get all
NAME READY STATUS RESTARTS AGE
pod/launcher-operator-65564cd544-xmppd 0/1 CrashLoopBackOff 2 43s
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/launcher-operator 1 1 1 0 44s
NAME DESIRED CURRENT READY AGE
replicaset.apps/launcher-operator-65564cd544 1 1 0 44s
NAME AGE
task.tekton.dev/echo-hello-world 5d
NAME SUCCEEDED REASON STARTTIME COMPLETIONTIME
taskrun.tekton.dev/echo-hello-world-task-run False TaskRunTimeout 5d 5d
[root@p006vm32 launcher-operator]# kubectl describe pod launcher-operator-65564cd544-xmppd
Name: launcher-operator-65564cd544-xmppd
Namespace: launcher-operator
Priority: 0
PriorityClassName: <none>
Node: p006vm32/129.40.81.32
Start Time: Tue, 17 Sep 2019 05:02:21 -0400
Labels: name=launcher-operator
pod-template-hash=2112078100
Annotations: openshift.io/scc=anyuid
Status: Running
IP: 10.128.1.79
Controlled By: ReplicaSet/launcher-operator-65564cd544
Containers:
launcher-operator:
Container ID: docker://76415e9d2424fe4e3d9522c32c082f058177bb46af88a04e06c4b230e2f59512
Image: fabric8/launcher-operator:9.1.2
Image ID: docker://sha256:32c8c0c50dedc9a1b6dd46356c16d72f07995ad7ff517757e2095cfe0b5cc2be
Port: 60000/TCP
Host Port: 0/TCP
Command:
launcher-operator
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Tue, 17 Sep 2019 05:02:47 -0400
Finished: Tue, 17 Sep 2019 05:02:47 -0400
Ready: False
Restart Count: 2
Readiness: exec [stat /tmp/operator-sdk-ready] delay=4s timeout=1s period=10s #success=1 #failure=1
Environment:
WATCH_NAMESPACE: launcher-operator (v1:metadata.namespace)
POD_NAME: launcher-operator-65564cd544-xmppd (v1:metadata.name)
OPERATOR_NAME: launcher-operator
TEMPLATE_PATH: /usr/local/bin/templates
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from launcher-operator-token-jb27l (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
launcher-operator-token-jb27l:
Type: Secret (a volume populated by a Secret)
SecretName: launcher-operator-token-jb27l
Optional: false
QoS Class: BestEffort
Node-Selectors: node-role.kubernetes.io/compute=true
Tolerations: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 55s default-scheduler Successfully assigned launcher-operator/launcher-operator-65564cd544-xmppd to p006vm32
Normal Started 29s (x3 over 49s) kubelet, p006vm32 Started container
Warning BackOff 15s (x4 over 46s) kubelet, p006vm32 Back-off restarting failed container
Normal Pulled 2s (x4 over 51s) kubelet, p006vm32 Container image "fabric8/launcher-operator:9.1.2" already present on machine
Normal Created 2s (x4 over 50s) kubelet, p006vm32 Created container
However, the pod is still in "CrashLoopBackOff". On investigating found that the build/_output/bin/launcher-operator gets generated for x86_64 while the build/_output/bin/launcher-operator-local is for ppc64le.
[root@p006vm32 launcher-operator]# file build/_output/bin/launcher-operator*
build/_output/bin/launcher-operator: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
build/_output/bin/launcher-operator-local: ELF 64-bit LSB executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=2d72ea8c8d958d1865d7c45ea3a180cbad9c0ec1, not stripped
Investigating further on this.
Resolved the issue wherein 'launcher-operator' binary used in the image gets generated for x86_64 by default, by setting GOARCH variable for ppc64le arch as below:- export GOARCH=ppc64le
Cleaned up and built the image successfully with the operator-sdk build command, post which the Launcher operator got deployed successfully.
[root@p006vm32 launcher-operator]# oc get all -n launcher-operator
NAME READY STATUS RESTARTS AGE
pod/launcher-operator-65564cd544-swdh7 1/1 Running 0 2h
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE deployment.apps/launcher-operator 1 1 1 1 2h
NAME DESIRED CURRENT READY AGE replicaset.apps/launcher-operator-65564cd544 1 1 1 2h
- Currently working on step to Install the Launcher (via the installed operator). Facing issues with the example launcher yaml which makes use of image: fabric8/launcher-backend:latest, which for now is available for x86_64.
I am following the steps at:- https://github.com/fabric8-launcher/launcher-operator
Facing the following error at step 4:-
Investigated the same and found the following:-