RDS - k8s Missing arguments

[jhernandezb]

Seems like the broker is not passing some defaults to the provisioner and for this reason fails to create the new stack.

• aws_cloudformation_role_arn
• region
• VpcId
• SBArtifactS3Bucket

Is this the intended behavior ?

I've managed to work with the following config.

apiVersion: servicecatalog.k8s.io/v1beta1
kind: ServiceInstance
metadata:
  name: rds
spec:
  clusterServiceClassExternalName: dh-rdsmysql
  clusterServicePlanExternalName: dev
  parameters:
      aws_access_key: "use-role"
      aws_secret_key: "use-role"
      aws_cloudformation_role_arn: "arn:aws:iam::99999999999:role/aws-servicebroker-cfn-deploy-role"
      region: "us-east-1"
      VpcId: "vpc-a0a0a0a0"
      SBArtifactS3Bucket: "awsservicebroker"
      AccessCidr: "172.20.0.0/16"

[jaymccon]

At present there are no defaults for those fields, typically they are configured at a cluster level using broker secrets, details on how to do this are documented here: https://github.com/awslabs/aws-servicebroker/blob/master/docs/getting-started-openshift.md#manually-creating-secrets-to-autofill-aws-service-parameters

That said it makes sense to me to default aws_access_key, aws_secret_key to "use-role" and SBArtifactS3Bucket to "awsservicebroker". The others are account/environment linked, so I don't see a way to provide useful deafults.

[jaymccon]

I just double checked and it looks like the install scripts do default these values.

I'm going to close this, let us know if the defaults are not working for you and we'll investigate further.

[alkar]

I've stumbled upon the same issue today. I followed the instructions here to set it up.

I can get an SQS queue but not a Postgres RDS:

$ kubectl -n dh-rdspostgresql-prov-62t9x logs apb-ba07622c-1d9a-4021-bfe7-238e69e40618
+ [[ provision --extra-vars {"_apb_plan_id":"dev","_apb_service_class_id":"46e348160adf2c73d8a6c37e2e8a653b","_apb_service_instance_id":"27b93807-8b69-11e8-a28b-f633de2c3e35","cluster":"kubernetes","namespace":"demo"} == *\s\2\i\/\a\s\s\e\m\b\l\e* ]]
+ ACTION=provision
+ shift
+ playbooks=/opt/apb/actions
+ CREDS=/var/tmp/bind-creds
+ TEST_RESULT=/var/tmp/test-result
+ whoami
+ oc-login.sh
Attempting to login with a service account...
Logged into "https://kubernetes.default:443" as "XXXXXXXXXXXXXXXXXXXXXXXXXXX" using the token provided.

Using "default".  You can switch projects with:

 'oc project <projectname>'
Welcome! See 'oc help' to get started.
+ set +x
ls: cannot access /etc/apb-secrets: No such file or directory
+ [[ -e /opt/apb/actions/provision.yaml ]]
+ [[ -e /opt/apb/actions/provision.yml ]]
+ ANSIBLE_ROLES_PATH=/etc/ansible/roles:/opt/ansible/roles
+ ansible-playbook /opt/apb/actions/provision.yml --extra-vars '{"_apb_plan_id":"dev","_apb_service_class_id":"46e348160adf2c73d8a6c37e2e8a653b","_apb_service_instance_id":"27b93807-8b69-11e8-a28b-f633de2c3e35","cluster":"kubernetes","namespace":"demo"}'
 [WARNING]: provided hosts list is empty, only localhost is available

PLAY [AWS Provision] ***********************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [ansible.kubernetes-modules : Install latest openshift client] ************
skipping: [localhost]

TASK [ansibleplaybookbundle.asb-modules : debug] *******************************
skipping: [localhost]

TASK [aws-provision-apb : include plan vars] ***********************************
ok: [localhost]

TASK [aws-provision-apb : generate b62 hash] ***********************************
changed: [localhost]

TASK [aws-provision-apb : set fact] ********************************************
ok: [localhost]

TASK [aws-provision-apb : Log stack suffix] ************************************
ok: [localhost] => {
    "msg": "stack_suffix: 5xo4o77qXo927uZzOlvuPEZ60L3"
}

TASK [aws-provision-apb : Create Resources] ************************************
fatal: [localhost]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'aws_cloudformation_role_arn' is undefined\n\nThe error appears to have been in '/opt/ansible/roles/aws-provision-apb/tasks/main.yml': line 14, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- block:\n  - cloudformation:\n    ^ here\n"}

TASK [aws-provision-apb : Writing error message] *******************************
changed: [localhost]

TASK [aws-provision-apb : Fail if there is CFN log errors] *********************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "[APB Error] - the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'aws_cloudformation_role_arn' is undefined\n\nThe error appears to have been in '/opt/ansible/roles/aws-provision-apb/tasks/main.yml': line 14, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- block:\n  - cloudformation:\n    ^ here\n"}
 [WARNING]: Could not create retry file '/opt/apb/actions/provision.retry'.
[Errno 13] Permission denied: u'/opt/apb/actions/provision.retry'

PLAY RECAP *********************************************************************
localhost                  : ok=6    changed=2    unreachable=0    failed=2

+ EXIT_CODE=2
+ set +ex
+ '[' -f /var/tmp/test-result ']'
+ '[' -f /var/tmp/bind-creds ']'
+ exit 2

The secret created by the install script is there:

$ kubectl -n aws-service-broker get secrets aws-secret
NAME         TYPE      DATA      AGE
aws-secret   Opaque    6         1h

and it includes aws_cloudformation_role_arn along with the other keys set in the variable file.

However, it doesn't appear as a mount on the Pod?

$ kubectl -n dh-rdspostgresql-prov-62t9x describe pod apb-ba07622c-1d9a-4021-bfe7-238e69e40618
Name:         apb-ba07622c-1d9a-4021-bfe7-238e69e40618
Namespace:    dh-rdspostgresql-prov-62t9x
Node:         ip-172-20-53-201.eu-west-1.compute.internal/172.20.53.201
Start Time:   Thu, 19 Jul 2018 16:33:59 +0100
Labels:       apb-action=provision
              apb-fqname=dh-rdspostgresql
              apb-pod-name=apb-ba07622c-1d9a-4021-bfe7-238e69e40618
Annotations:  <none>
Status:       Failed
IP:           100.117.231.10
Containers:
  apb:
    Container ID:  docker://03ae75538b7b9c017f3b120aef85ac3cd461429c2de6c73dc35863e49462130e
    Image:         docker.io/awsservicebroker/rdspostgresql-apb:latest
    Image ID:      docker-pullable://awsservicebroker/rdspostgresql-apb@sha256:532850b38e7a8d024f9b6ee2aaca97d84939ecfbe28b1463d71aab88b672d046
    Port:          <none>
    Host Port:     <none>
    Args:
      provision
      --extra-vars
      {"_apb_plan_id":"dev","_apb_service_class_id":"46e348160adf2c73d8a6c37e2e8a653b","_apb_service_instance_id":"27b93807-8b69-11e8-a28b-f633de2c3e35","cluster":"kubernetes","namespace":"demo"}
    State:      Terminated
      Reason:   Error
      Message:  [CloudFormation Error] - the field args has an invalid value, which appears to include a variable that is undefined. The error was: aws_cloudformation_role_arn is undefined

The error appears to have been in /opt/ansible/roles/aws-provision-apb/tasks/main.yml: line 14, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- block:
  - cloudformation:
    ^ here

      Exit Code:    2
      Started:      Thu, 19 Jul 2018 16:34:00 +0100
      Finished:     Thu, 19 Jul 2018 16:34:03 +0100
    Ready:          False
    Restart Count:  0
    Environment:
      POD_NAME:       apb-ba07622c-1d9a-4021-bfe7-238e69e40618 (v1:metadata.name)
      POD_NAMESPACE:  dh-rdspostgresql-prov-62t9x (v1:metadata.namespace)
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from apb-ba07622c-1d9a-4021-bfe7-238e69e40618-token-pj8nd (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          False
  PodScheduled   True
Volumes:
  apb-ba07622c-1d9a-4021-bfe7-238e69e40618-token-pj8nd:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  apb-ba07622c-1d9a-4021-bfe7-238e69e40618-token-pj8nd
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

The ServiceInstance then gets stuck and I can't delete it without uninstalling the broker first.

Worth noting that this has been setup with IAM access keys and not instance roles.