[sonatype-nexus] IRSA is not working

[snakebyte91]

IRSA should be supported with Nexus version 3.26 as per NEXUS-24019

Chart version: 4.4.3 Image: quay.io/travelaudience/docker-nexus:3.27.0

Got the following error while trying to create a S3 blob store:

org.sonatype.nexus.blobstore.s3.internal.S3BlobStoreException: Bucket exists but is not owned by you.

[DmitriyStoyanov]

Facing with the same issue in eks. But interesting, that i've connected to the pod, installed there aws-cli and triggered aws s3api get-bucket-acl --bucket bucket-name and it works, but from nexus it fails.

Checked in cloudtrail, and found, that when I used aws-cli, it uses role attached via serviceaccount, but when nexus try to trigger s3.getBucketAcl(bucket); it uses role attached to node instance instead of serviceaccount attached. tested on quay.io/travelaudience/docker-nexus:3.33.0-01

As I see during this configuration, env variables

      AWS_ROLE_ARN:                   arn:aws:iam::xxxx:role/nexus-bucket-role-xxx
      AWS_WEB_IDENTITY_TOKEN_FILE:    /var/run/secrets/eks.amazonaws.com/serviceaccount/token

configured automatically, when I have in my chart values:

serviceAccount:
  annotations:
    "eks.amazonaws.com/role-arn": "arn:aws:iam::xxxx:role/nexus-bucket-role-xxx"

with existed role.

Possibly current version of aws sdk https://github.com/sonatype/nexus-public/blob/master/buildsupport/other/pom.xml#L31

     <aws-java-sdk.version>1.11.822</aws-java-sdk.version>

does not support WebIdentityToken as it described here https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-technical-overview.html?

Looks like so, in Changelog of aws sdk v1 I have not found anything about it, but found it here Changelog of aws sdk v2 - AWS Security Token Service

Edited: hm, no it was added previously in another changelog file for 1.11.x in 1.11.704 version this env variable already exist

[DmitriyStoyanov]

Hm, tested locally update of aws sdk java to 1.12.57 with compiling locally nexus-blobstore-s3-3.33.1-01.jar and adding it to docker image, but it doesn't help to fix the issue

[gustavclausen]

Faced the same issue. The main problem is that the nexus user in the container doesn't have permission to read the web identity token that is mounted into the container on the aws-iam-token volume (path /var/run/secrets/eks.amazonaws.com/serviceaccount). The nexus user is running the Nexus process, not the root user.

Solution

We solved the issue by delegating volume permission to the group of the nexus user by updating the security context for the pod:

Get group ID of nexus user:

$ id -u nexus
1000

Update the security context for the pod in the Helm values:

nexus:
  securityContext:
-  fsGroup: 200
+  fsGroup: 1000

Additional information

Docker image: quay.io/travelaudience/docker-nexus:3.38.1-01 Helm chart version: 5.4.1