apb not listed after successful push

[cunningt]

Seeing this on Mac OS X using oc cluster up (non-minishift). APB tool built from source, prepare/build/push are successful but when I apb list or check the catalog in the console, I can't find the apb I've just pushed.

tcunning@tcunning-OSX:/tmp/foo-apb$ apb push

version: 1.0 name: foo-apb description: This is a sample application generated by apb init bindable: False async: optional metadata: displayName: foo plans:

• name: default description: This default plan deploys foo-apb free: True metadata: {} parameters: [] Found registry IP at: 172.30.1.1:5000 Finished writing dockerfile. Building APB using tag: [172.30.1.1:5000/openshift/foo-apb] Successfully built APB image: 172.30.1.1:5000/openshift/foo-apb Pushing the image, this could take a minute... Successfully pushed image: 172.30.1.1:5000/openshift/foo-apb Contacting the ansible-service-broker at: https://asb-1338-ansible-service-broker.127.0.0.1.nip.io/ansible-service-broker/v2/bootstrap Successfully bootstrapped Ansible Service Broker Successfully relisted the Service Catalog tcunning@tcunning-OSX:/tmp/foo-apb$ apb list | grep foo tcunning@tcunning-OSX:/tmp/foo-apb$

[dymurray]

I think we need to get better about handling errors from docker push.. I will take a look at the very least printing out the error. Usually you can see what the error is if you manually do docker push 172.30.1.1:5000/openshift/foo-apb

[cunningt]

The error here was that the oc user I was using did not have rights to push to openshift - but it was pretty difficult to figure that out - definitely agree that printing out the errors would be really useful to the user.

[eriknelson]

@dymurray agreed re: the error output. Unfortunately it's not as simple as catching an exception, we're going to need to open a pipe and parse the stream that docker is outputting, unless there is a better way to deal with that client that I'm unaware of. An error status would be ideal.

[dymurray]

@eriknelson yeah I was actually trying to figure out a fix yesterday. One solution I had was to simply parse the output for Error and print the adjacent string. It wasn't great though so I want to spend some more time investigating the docker client package.

[jonnyfiveiq]

I think I have the same issue, after a successful push the image is not in the catalog, I do a APB list and nothing,,

I did try apb push --push-to-broker --broker asb-1338-ansible-service-broker.192.168.64.26.nip.io and this seem to register the image but as "apb-push-cat-test", this then shows up in the catalog....though after an extended amount of time,,,and apb list does not speed that up, though does show the item.

[dymurray]

@jonnyfiveiq I am willing to bet this is due to your broker configuration. I think that the broker is failing to bootstrap images from the internal registry if the image was successfully pushed and you didn't see it in apb list.

Can you please paste the output of oc get configmap broker-config -n ansible-service-broker -o yaml?

[jonnyfiveiq]

Sure, here is the output

apiVersion: v1 data: broker-config: | registry:

• type: "dockerhub" name: "dh" url: "https://registry.hub.docker.com" org: "ansibleplaybookbundle" tag: "latest" white_list:
  • ".*-apb$"
• type: local_openshift name: localregistry namespaces: ['openshift'] white_list:
  • ".*-apb$" dao: etcd_host: asb-etcd.ansible-service-broker.svc etcd_port: 2379 etcd_ca_file: /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt etcd_client_cert: /var/run/asb-etcd-auth/client.crt etcd_client_key: /var/run/asb-etcd-auth/client.key log: logfile: /var/log/ansible-service-broker/asb.log stdout: true level: debug color: true openshift: host: "" ca_file: "" bearer_token_file: "" image_pull_policy: "IfNotPresent" sandbox_role: "edit" namespace: ansible-service-broker keep_namespace: false keep_namespace_on_error: true broker: dev_broker: true bootstrap_on_startup: true refresh_interval: "600s" launch_apb_on_bind: false output_request: true recovery: true ssl_cert_key: /etc/tls/private/tls.key ssl_cert: /etc/tls/private/tls.crt auto_escalate: false auth:
    • type: basic enabled: false kind: ConfigMap metadata: creationTimestamp: 2018-03-08T13:16:47Z labels: app: ansible-service-broker name: broker-config namespace: ansible-service-broker resourceVersion: "2153" selfLink: /api/v1/namespaces/ansible-service-broker/configmaps/broker-config uid: f47ab82d-22d2-11e8-a026-16a627774c1f

[jonnyfiveiq]

So my steps to reproduce are as follows;

Follow minishift start instructions and addon install located here https://github.com/eriknelson/minishift-addons/tree/asb-updates/add-ons/ansible-service-broker

Then apply permissions as follows;

oc login -u system:admin oc adm policy add-cluster-role-to-user cluster-admin developer oc adm policy add-cluster-role-to-user access-asb-role developer oc login -u developer -p developer

then;

apb init myapp apb build apb prepare apb push

nothing shows in catalog but terminal shows success.

[cunningt]

@jonnyfiveiq The problem I had was that I didn't have permissions using the developer user to push an image to the openshift project. Maybe give pushing a 172.30.1.1/openshift/:1.0 image a try after docker -u login developer?

[jonnyfiveiq]

I think my issue was linked to not having -apb suffix in the apb name declaration...has very odd behaviour.

[mkanoor]

@cunningt How does you add "push" rights for a oc user in open shift? I am having a similar issue and I would appreciate your help.

[eriknelson]

@mkanoor There are a number of required permissions before you are going to be able to use the apb tool to push to the local registry. If you are using the minishift-addon with minishift, the developer user is automatically permissioned with the ability to push.

If you have somehow manually deployed the broker and need to permission a user, you can use this template: curl "https://raw.githubusercontent.com/ansibleplaybookbundle/ansible-playbook-bundle/master/templates/openshift-permissions.template.yaml" | oc process -p USER=$YOUR_USERNAME -f - | oc create -f -. Obviously the user you are logged in as requires you to have sufficient privileges to create and apply these roles.

[mkanoor]

@eriknelson I am on Mac using OpenShift with Docker.

oc v3.7.1+ab0f056
kubernetes v1.7.6+a08f5eeb62
features: Basic-Auth

Server https://127.0.0.1:8443
openshift v3.7.1+a8deba5-34
kubernetes v1.7.6+a08f5eeb62

I have installed apb using pip

oc cluster up --service-catalog=true
oc login -u system:admin
oc adm policy add-cluster-role-to-user cluster-admin developer
oc login << as developer>>
I run this shell script https://github.com/openshift/ansible-service-broker/blob/master/scripts/run_latest_build.sh
app bootstrap --username developer --password password

That still gives me a 403, it seems like its missing some roles.

I added curl "https://raw.githubusercontent.com/ansibleplaybookbundle/ansible-playbook-bundle/master/templates/openshift-permissions.template.yaml" | oc process -p USER=$YOUR_USERNAME -f - | oc create -f -

I still get a 403, I ran into the bootstrap because the apb push was failing in bootstrap.

[eriknelson]

@jonnyfiveiq We added an issue to track a proposed change to remove that name restriction, and I opened a PR to make the change. Issue: https://github.com/openshift/ansible-service-broker/issues/823, PR: https://github.com/openshift/ansible-service-broker/pull/827. EDIT: Change has been merged.

In an effort to avoid this becoming a kitchen sink issue for all apb push related problems, @cunningt @jonnyfiveiq, is it fair to consider this issue closed as resolved, and to open a more specific issue: the apb push command incorrectly reports success when the docker push has failed under the covers?

@mkanoor I opened a distinct issue to track your specific problem here: #242

[jonnyfiveiq]

yes my issue is closed, naming the APB with -APB fixes my issues with PUSH, of which I saw two, 1) where the apb would not list, or if it did list 2) it would not pull image. With the -APB added to name, I have not seen either issue since. thanks

[mkanoor]

@eriknelson @dymurray I am using open shift I see the apb push being successful but the apb list doesn't show the apb? I tried @jonnyfiveiq 's suggestion of using suffix -apb, that didn't help either.

Successfully pushed image: 172.30.1.1:5000/openshift/demo-apb
Contacting the ansible-service-broker at: https://asb-1338-ansible-service-broker.127.0.0.1.nip.io/ansible-service-broker/v2/bootstrap
Successfully bootstrapped Ansible Service Broker
Successfully relisted the Service Catalog  

apb list | grep demo-apb

comes up empty

[eriknelson]

@mkanoor's issue is solved and we're going to close this and open an issue that tracks the specific problem of apb push error reporting. Thanks all.