operator-framework / operator-lifecycle-manager

A management framework for extending Kubernetes with Operators
https://olm.operatorframework.io
Apache License 2.0
1.72k stars 545 forks source link

Can't deploy OLM onto OpenShift #436

Closed djwhatle closed 6 years ago

djwhatle commented 6 years ago

I'm attempting to get OLM running and viewable from the web console and hitting some rough patches. My latest issue is with oc cluster up failing due to "unknown flag --service-catalog" as shown below.

30 minutes ago I was browsing the service-catalog web interface just fine, but failing to look at the rest of the OKD interface. Will write up another issue for that if I can reproduce soon.

Currently have commit 0a5a3... checked out.

[dwhatley@precision-t operator-lifecycle-manager]$ minishift version
minishift v1.23.0+91235ee

[dwhatley@precision-t operator-lifecycle-manager]$ minishift delete
You are deleting the Minishift VM: 'minishift'. Do you want to continue [y/N]?: y
Deleting the Minishift VM...
Minishift VM deleted.

[dwhatley@precision-t operator-lifecycle-manager]$ make run-local-shift
. ./scripts/build_local_shift.sh
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.10.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.10.0' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ... 
   Driver is available at /usr/local/bin/docker-machine-driver-kvm ... 
   Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting the OpenShift cluster using 'kvm' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    4 GB
   vCPUs :    2
   Disk size: 20 GB
-- Starting Minishift VM ................. OK
-- Checking for IP address ... OK
-- Checking for nameservers ... OK
-- Checking if external host is reachable from the Minishift VM ... 
   Pinging 8.8.8.8 ... OK
-- Checking HTTP connectivity from the VM ... 
   Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
   Importing 'openshift/origin:v3.10.0' .............................. OK
   Importing 'openshift/origin-docker-registry:v3.10.0' .............. OK
   Importing 'openshift/origin-haproxy-router:v3.10.0' ............... OK
-- OpenShift cluster will be configured with ...
   Version: v3.10.0
-- Copying oc binary from the OpenShift container image to VM ... OK
-- Starting OpenShift cluster -- Extra 'oc' cluster up flags (experimental) ... 
   '--service-catalog'
.Error during 'cluster up' execution: Error starting the cluster. ssh command error:
command : /var/lib/minishift/bin/oc cluster up --base-dir /var/lib/minishift/base --public-hostname 192.168.42.121 --routing-suffix 192.168.42.121.nip.io --service-catalog
err     : exit status 1
output  : Error: unknown flag: --service-catalog
djwhatle commented 6 years ago

I think the command should probably be:

oc cluster up --enable="service-catalog" instead of oc cluster up --service-catalog

https://github.com/operator-framework/operator-lifecycle-manager/blob/master/scripts/build_local_shift.sh#L7

Edit: The above should apply for 3.10, but I'm also thinking we may want to enable other cluster components to get OLM to work in the Web UI on 3.10, so trying now with --enable="*"

Edit 2: Changing --service-catalog to --enable=* in build_local_shift.sh got me a running cluster again. Still struggling to get the OKD console to work with ./scripts/run_console_local.sh, but will open a separate issue for that.

njhale commented 6 years ago

@djwhatle Good catch. Also, OLM currently requires Kubernetes v1.11 and I believe the version of OpenShift (v3.10) that minishift starts corresponds to Kube v1.10.

djwhatle commented 6 years ago

@njhale I see! Did OLM work with OpenShift <=v3.10 at one point? Is minikube with Kubernetes v1.11 the only recommended option right now?

njhale commented 6 years ago

At some point OLM did work with <=v3.10. We've confirmed with our internal CI environment that it works with v3.11. Until minishift can run a v3.11 cluster it looks like minikube is the best option.

ecordell commented 6 years ago

You could try 0.5.0 against 3.10? I can't recall if that will definitely work or not: https://github.com/operator-framework/operator-lifecycle-manager/tree/master/deploy/aos-olm/manifests/0.5.0

djwhatle commented 6 years ago

@ecordell is there a variable I can tweak somewhere to switch make run-local or make run-local-shift over to deploying 0.5.0?

Edit: grabbing release 0.5.0 zip and running that. Crossing fingers.

djwhatle commented 6 years ago

@ecordell checked out 0.5.0 tag and tried running against 3.10. Appears to have trouble logging into 3.10 cluster (tries to use -u system:admin) and also has trouble building even when you tweak the script to login with admin/foo instead.

Here are the build errors:

Sending build context to Docker daemon 92.06 MB
Step 1/6 : FROM golang:1.10
Trying to pull repository docker.io/library/golang ... 
1.10: Pulling from docker.io/library/golang
55cbf04beb70: Pull complete 
1607093a898c: Pull complete 
9a8ea045c926: Pull complete 
d4eee24d4dac: Pull complete 
9c35c9787a2f: Pull complete 
1701d74f449a: Pull complete 
ed9b85f3273c: Pull complete 
Digest: sha256:b38130c9826dc4eff5375f7deeac6dc9c2b9947f194229fac34712f549d03361
Status: Downloaded newer image for docker.io/golang:1.10
 ---> 5b1054129196
Step 2/6 : WORKDIR /go/src/github.com/operator-framework/operator-lifecycle-manager
 ---> 624abfbb1d14
Removing intermediate container e937dad76936
Step 3/6 : COPY . .
 ---> 5aecf0418b7f
Removing intermediate container 7efe9901077c
Step 4/6 : RUN make build && cp bin/alm /bin/alm && cp bin/catalog /bin/catalog && cp bin/servicebroker /bin/servicebroker
 ---> Running in 2ab4373bd690

/bin/bash: docker: command not found
/bin/bash: docker: command not found
/bin/bash: docker: command not found
/bin/bash: docker: command not found
if [ 1 = 1true ]; then \
        GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go test -o bin/alm -c -covermode=count -coverpkg ./pkg/... github.com/operator-framework/operator-lifecycle-manager/cmd/alm; \
else \
        GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o bin/alm github.com/operator-framework/operator-lifecycle-manager/cmd/alm; \
fi
vendor/github.com/emicklei/go-restful/container.go:17:2: cannot find package "github.com/emicklei/go-restful/log" in any of:
        /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/github.com/emicklei/go-restful/log (vendor tree)
        /usr/local/go/src/github.com/emicklei/go-restful/log (from $GOROOT)
        /go/src/github.com/emicklei/go-restful/log (from $GOPATH)
Makefile:40: recipe for target 'bin/alm' failed
make: *** [bin/alm] Error 1
The command '/bin/sh -c make build && cp bin/alm /bin/alm && cp bin/catalog /bin/catalog && cp bin/servicebroker /bin/servicebroker' returned a non-zero code: 2
make: *** [Makefile:74: run-local-shift] Error 2
[dwhatley@precision-t operator-lifecycle-manager]$
djwhatle commented 6 years ago

Latest attempt with an oc cluster up OpenShift I created without minishift:

Is this a known issue? ConfigMap too long error

The ConfigMap "ocs" is invalid: metadata.annotations: Too long: must have at most 262144 characters

Full output of oc apply -f deploy/upstream/manifests/0.6.0/

$ oc version
oc v3.11.0-alpha.0+36137de-1235
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://<my_cluster_ip>:8443
kubernetes v1.11.0+d4cacc0

$ oc apply -f deploy/upstream/manifests/0.6.0/
serviceaccount/olm-operator-serviceaccount created
clusterrolebinding.rbac.authorization.k8s.io/olm-operator-binding-kube-system created
customresourcedefinition.apiextensions.k8s.io/clusterserviceversions.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/catalogsources.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/installplans.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/subscriptions.operators.coreos.com created
catalogsource.operators.coreos.com/ocs created
deployment.apps/alm-operator created
deployment.apps/catalog-operator created
clusterrole.rbac.authorization.k8s.io/aggregate-olm-edit created
clusterrole.rbac.authorization.k8s.io/aggregate-olm-view created
The ConfigMap "ocs" is invalid: metadata.annotations: Too long: must have at most 262144 characters

Edit (RESOLVED): Resolved the "must have at most 262144 characters" issue by using oc create -f instead of oc apply -f based on the following from @njhale:

@njhale: try oc replace or oc create. Apply creates an annotation containing the previous config map.

Posted docs update PR: https://github.com/operator-framework/operator-lifecycle-manager/pull/449