search

jtblin / kube2iam

kube2iam provides different AWS IAM roles for pods running on Kubernetes
BSD 3-Clause "New" or "Revised" License
1.98k stars 319 forks source link

unable to fix access denied on s3 using kube2iam #274

Closed sanees2010 closed 4 years ago

sanees2010 commented 4 years ago

Hi All

i followed all previous posts before posting my issue here, i am unable to make kube2iam work, my pod still gets access denied when trying to list buckets from s3, 1 - on worker node i attached additional policy which is something like this { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": "sts:AssumeRole", "Resource": "*" } ] }

I created 1 extra role, which has s3 full access, and its trust relationship is something like this

{ "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com" }, "Action": "sts:AssumeRole" }, { "Sid": "", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam:::role/node-role" }, "Action": "sts:AssumeRole" } ] }

My daemonset looks like this

apiVersion: apps/v1 kind: DaemonSet metadata: name: kube2iam namespace: default labels: app: kube2iam spec: selector: matchLabels: name: kube2iam template: metadata: labels: name: kube2iam spec: serviceAccountName: kube2iam hostNetwork: true containers:

And my deployment looks like this

apiVersion: apps/v1 kind: Deployment metadata: name: agent-deployment annotations: iam.amazon.com/role: arn:aws:iam:::role/k8s-pods-s3-role spec: replicas: 1 selector: matchLabels: app: agent-pod template: metadata:

annotations:

        #iam.amazon.com/role: k8s-pods-s3-role
  labels:
    app: agent-pod
spec:
  containers:
  - name: agent-container
    image: 674528447826.dkr.ecr.eu-west-2.amazonaws.com/tcagent-latest

I am trying to run teamcity agent which should have access to s3, i have tried using annotation in both pod's spec and also at deployment level but not getting any success, however there is 1 thing which sorted issue for few, that is host-interface. i am running kubernetes on amazon ec2 using kops, i tried eni+, cbr0 and cni0. but nothing work. Can any one please help on this

When i check the logs for kube2iam daemon set i get something like this

time="2020-05-23T16:02:25Z" level=info msg="base ARN autodetected, arn:aws:iam:::role/" time="2020-05-23T16:02:25Z" level=info msg="Using instance IAMRole arn:aws:iam::674528447826:role/nodes.k8s.local as default" time="2020-05-23T16:02:25Z" level=info msg="Listening on port 81

time="2020-05-23T16:13:42Z" level=info msg="GET /latest/meta-data/iam/security-credentials/k8s-pods-s3-role (200) took 59976.000000 ns" req.method=GET req.path=/latest/meta-data/iam/security-credentials/k8s-pods-s3-role req.remote=100.96.1.18 res.duration=59976 res.status=200

sanees2010 commented 4 years ago

not needed any more