Cannot read property 'runtime-images' of undefined

nodesocket commented 5 years ago

BUG REPORT

Migrating this from serverless repo https://github.com/serverless/serverless-kubeless/issues/140#issuecomment-526726537 as they state it is a problem with Kubeless.

Running Kubernetes 1.13 on Amazon EKS with Kubelss v1.0.4.

What happened:

serverless deploy -v
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Unable to find required information for authenticating against the cluster
Unable to find required information for authenticating against the cluster

  Type Error ---------------------------------------------

  TypeError: Cannot read property 'runtime-images' of undefined

What you expected to happen:

Deploy successfully to Kubernetes cluster running on AWS EKS.

Details:

Setting the two required envar's KUBECONFIG and AWS_PROFILE.

Then I am able to run kubectl commands:

kubectl get pods --namespace kubeless
NAME                                          READY   STATUS    RESTARTS   AGE
kafka-0                                       1/1     Running   0          7m41s
kafka-trigger-controller-69cf947d6b-q62jg     1/1     Running   0          7m41s
kubeless-controller-manager-646876977-qb46g   3/3     Running   0          3d19h
zoo-0                                         1/1     Running   0          7m40s

I am using aws-iam-authenticator could that be the issue?

andresmgot commented 5 years ago

I am using aws-iam-authenticator could that be the issue?

Yes, the amount of authentication methods supported by the plugin is quite limited. That's likely caused by an old version of the package kubernetes-client that we are using there. An upgrade will probably help but we never had the time to contribute to that. External contributions are welcomed! :)

nodesocket commented 5 years ago

@andresmgot I am happy to try and assist as this is blocking me from run serverless on a Kubernetes cluster on Amazon EKS. Can you perhaps point me in the right direction?

andresmgot commented 5 years ago

@nodesocket sure. Let me guide you through the code. Note that this only affects to the serveleress-kubeless plugin so all the code is in that repository. So the code is failing here:

https://github.com/serverless/serverless-kubeless/blob/master/lib/config.js#L65

  get(key, opt) {
    if (opt && opt.parse) {
      return JSON.parse(this.configMag.data[key]);
    }
    return this.configMag.data[key];
  }

That is because configMap is undefined because it was not able to retrieve the Kubeless configuration. TBH, that should fail before, when it's not able to resolve configMap but that's a different issue.

This configMap is in the same file. And it's not able to retrieve it because here:

    this.connectionOptions = Object.assign(
      helpers.getConnectionOptions(helpers.loadKubeConfig()),
      { url, json: true }
    );

Those connection options are not properly configured to use aws-iam-authenticator. The code that resolves the authentication info is that function helpers.getConnectionOptions. You can find the code here:

https://github.com/serverless/serverless-kubeless/blob/master/lib/helpers.js#L129

As you can see in that function, the supported mechanisms to authenticate are either using a token, a user and pasword or a client-certificate.

So having that we can either:

Discard that getConnectionOptions implementation and use a similar method of the library kubernetes-client for NodeJS. I assume that they have something to be able to connect to the cluster and retrieve resources like the configMap we are talking.
Improve the getConnectionOptions method to support other types of auth, like the one you mention.

Hope that helps, I can answer more questions if you have them.

nodesocket commented 5 years ago

@andresmgot thanks for the reply. So, just to get a test working I can generate a standard token using https://docs.aws.amazon.com/cli/latest/reference/eks/get-token.html. Then do I set this token so severless will use it?

andresmgot commented 5 years ago

yes, that should work. Have you tried?

nodesocket commented 5 years ago

@andresmgot I am not sure how to set the Kube token so serverless uses it. I was able to successfully deploy the test py function using kubeless described in the quick start guide https://kubeless.io/docs/quick-start/.

However, when running serverless deploy it fails with:

Serverless: Packaging service...
Serverless: Excluding development dependencies...
Unable to find required information for authenticating against the cluster
Unable to find required information for authenticating against the cluster

  Type Error ---------------------------------------------

  TypeError: Cannot read property 'runtime-images' of undefined

vmware-archive / kubeless

Cannot read property 'runtime-images' of undefined #1078

BUG REPORT