[Feature]: For running on AWS, add IAM Roles for service accounts (IRSA) support

[aaroniscode]

Checklist

• [X] I've searched for similar issues and couldn't find anything matching
• [x] I've discussed this feature request in the K8sGPT Slack and got positive feedback

Is this feature request related to a problem?

Yes

Problem Description

According to AWS IAM Docs Managing access keys for IAM users:

As a best practice, use temporary security credentials (such as IAM roles) instead of creating long-term credentials like access keys.

It's a best practice when running Kubernetes applications on AWS to use IAM roles for service accounts. This works out of the box for Amazon EKS clusters. For self-managed clusters like kops or Cluster API, you can use the open source Amazon EKS Pod Identity Webhook which enables these non-EKS clusters to use IAM Roles for service accounts.

AWS customers should avoid using Access keys within Kubernetes clusters.

Solution Description

While I haven't done a full feature design, I think there are 2 things required for this feature:

1. When choosing spec.ai.backend: amazonbedrock, make the secret optional.
2. Extend the K8sGPT CRD to include a configuration option to annotate the service account. This annotation takes the form eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/my-role

Benefits

This will be a much more secure solution for running the K8sGPT Operator in AWS accounts and will align with AWS Security Best Practices.

Potential Drawbacks

None. This should be an optional configuration and would still allow Access keys. Access keys are a valid use case when running a local Kubernetes cluster like kind or running the cluster outside of AWS and using Amazon Bedrock as a backend.

Additional Information

In Nov 2023, Amazon introduced EKS Pod Identity as a way to simplify IAM permissions for customers running workloads on Amazon EKS. This feature is not open source, like IAM Roles for service accounts, and only works on Amazon EKS clusters.

While out of scope for this feature request, depending on the implementation of this feature, EKS Pod Identity may work when a secret (containing Access keys) is optional. Today access keys are not optional when using the Amazon Bedrock backend. EKS Pod Identity is supported in the AWS SDK for Go v1 v1.47.11 and later and k8sgpt is currently using v1.51.8.

[arbreezy]

this is blocked by https://github.com/k8sgpt-ai/k8sgpt-operator/pull/274

[aaroniscode]

Will open a PR as soon as #434 is merged

[arbreezy]

it's been merged @aaroniscode

[aaroniscode]

thanks @arbreezy! Starting work on the PR