Error Deploying to AWS EKS

[james-ingold]

Hello,

I'm trying to deploy to AWS EKS. I've configured kubectl for my cluster in amazon. I'm getting the following error when running serverless deploy -v

Unable to find required information for authenticating against the cluster Unable to find required information for authenticating against the cluster /Users/jamesingold/Repos/juxly/perfcol-serverless/node_modules/serverless-kubeless/lib/config.js:58 return JSON.parse(this.configMag.data[key]); ^ TypeError: Cannot read property 'runtime-images' of undefined at Config.get (/Users/jamesingold/Repos/juxly/perfcol-serverless/node_modules/serverless-kubeless/lib/config.js:58:44) at Object.getRuntimeDepfile (/Users/jamesingold/Repos/juxly/perfcol-serverless/node_modules/serverless-kubeless/lib/helpers.js:183:34) at ReadStream.s.on (/Users/jamesingold/Repos/juxly/perfcol-serverless/node_modules/serverless-kubeless/deploy/kubelessDeploy.js:103:39) at ReadStream.emit (events.js:187:15)

It looks like there is an authentication issue, even though I'm logged in through kubectl { "kind": "Status", "apiVersion": "v1", "metadata": {

}, "status": "Failure", "message": "configmaps \"kubeless-config\" is forbidden: User \"system:anonymous\" cannot get configmaps in the namespace \"kubeless\"", "reason": "Forbidden", "details": { "name": "kubeless-config", "kind": "configmaps" }, "code": 403 }

[rddefauw]

What was the resolution?

[james-ingold]

I had to hard code a token in the serverless-kubeless library in helpers.js getToken https://github.com/serverless/serverless-kubeless/blob/master/lib/helpers.js

[japsu]

That's no solution.

The problem is likely that the way Kubeless talks to the Kubernetes Master API fails to take into account the custom authentication method required by EKS in ~/.kube/config:

- name: arn:aws:eks:eu-west-1:XXXXX:cluster/CLUSTERNAME
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
      - token
      - -i
      - CLUSTERNAME
      command: aws-iam-authenticator

Please reopen.

[andresmgot]

Yes @japsu, that was a workaround more than a solution.

At that time the library we are using to work with Kubernetes in this plugin didn't support that new way of authenticating but it seems that it's available now (since this October): https://github.com/godaddy/kubernetes-client/issues/306

I am reopening this issue but we will need help updating the library!

[streetmapp]

Has anyone been able to look at this? Unsure if the iam-authenticator method needs to be used any longer. In May AWS made the authentication easier by putting it in the Amazon EKS CLI. Not sure if that streamlines the work needed or not.

[nodesocket]

I am also having this issue. Setting the two required envar's KUBECONFIG and AWS_PROFILE.

Then I am able to run kubectl commands:

kubectl get pods --namespace kubeless
NAME                                          READY   STATUS    RESTARTS   AGE
kafka-0                                       1/1     Running   0          7m41s
kafka-trigger-controller-69cf947d6b-q62jg     1/1     Running   0          7m41s
kubeless-controller-manager-646876977-qb46g   3/3     Running   0          3d19h
zoo-0                                         1/1     Running   0          7m40s

Howerver, running:

serverless deploy -v
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Unable to find required information for authenticating against the cluster
Unable to find required information for authenticating against the cluster

  Type Error ---------------------------------------------

  TypeError: Cannot read property 'runtime-images' of undefined

[cscetbon]

@bradleybluebean @andresmgot I still have the same issue even on the last release of serverless (including the merged PR) :

$ sls version
Plugin: 3.6.1
SDK: 2.3.0
Components: 2.22.3
$ npm list serverless-kubeless
hello@1.0.0 /.../serverless-kubeless/examples/get-python
└── serverless-kubeless@0.7.3
$ k get configmaps kubeless-config -n kubeless                                                                                                                                  
NAME              DATA   AGE
kubeless-config   10     2m7s

$ serverless deploy                                                                                                                                                             
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Unable to find required information for authenticating against the cluster
Unable to find required information for authenticating against the cluster

  Error --------------------------------------------------

  Error: Request returned: 403 - configmaps "kubeless-config" is forbidden: User "system:anonymous" cannot get resource "configmaps" in API group "" in the namespace "kubeless"
    Response: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"configmaps \"kubeless-config\" is forbidden: User \"system:anonymous\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"kubeless\"","reason":"Forbidden","details":{"name":"kubeless-config","kind":"configmaps"},"code":403}
  false

[cscetbon]

hmm I just found out that the change is not included in the version that is installed when you don't specify any version. I don't why but the version that is installed is 0.7.3 ... I installed 0.9.2 and got it working. Should I create an issue about that ?

[andresmgot]

that depends on the content of the package.json. Probably it's not updated in the example you tried to deploy. You can update the version there and send a PR so the next person that tries it doesn't find the issue :)

[cscetbon]

what do you mean ? I'm saying npm install serverless-kubeless should install the last version when it installed 0.7.3. So I wonder if it's just that some versions were not deployed to the repo for some reasons

[andresmgot]

I mean, when doing npm install serverless-kubeless it installs the latest version:

▶ npm install serverless-kubeless
+ serverless-kubeless@0.9.2
added 108 packages from 164 contributors and audited 192 packages in 7.488s

14 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

The problem is when you do npm install in a project that has an old version. I assume that this example (hello@1.0.0 /.../serverless-kubeless/examples/get-python) has an old version of serverless-kubeless in its package.json. That's why you are seeing that version.

[cscetbon]

@andresmgot thanks for the answer, it makes sense !