eclipse-hawkbit / hawkbit-extensions

Eclipse Public License 2.0
20 stars 30 forks source link

artifact-repository-s3 extension can add and read, but not delete artifacts #89

Closed forsberg closed 1 year ago

forsberg commented 1 year ago

Having a most peculiar problem with the artifact-repository-s3 extension, version 0.3.0M8.

Hawkbit runs in EKS Kubernetes, with an AWS IAM Role connected to the service account, i.e, AWS API calls authenticate via Web Identity Token.

The bucket has public access disabled, and the IAM Policy looks like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:GetObject*",
                "s3:GetBucket*",
                "s3:List*",
                "s3:DeleteObject*",
                "s3:PutObject*",
                "s3:Abort*"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket",
                "arn:aws:s3:::my-bucket/*"
            ],
            "Effect": "Allow"
        }
    ]
}

Note the s3:DeleteObject* in the Action list.

Adding and retrieving artifacts work as intended. Objects are added to the S3 bucket under /DEFAULT, and when I retrieve them via the DDI API, they have the right contents.

However, if I try to delete an artifact from the Upload view, Hawkbit throws an exception, with the relevant parts being:

 Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: VS2BAXWQ9866BGMN; S3 Extended Request ID: cwZxN9yBRP3e9MP6Wflw8x/8vSKOJhlGRXz2Kat0nJD54gpfPs9yp/f3fKr0QHDzcVWhUIejJcc=; Proxy: null)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)
    at com.amazonaws.services.s3.AmazonS3Client.deleteObject(AmazonS3Client.java:2302)
    at org.eclipse.hawkbit.artifact.repository.S3Repository.deleteBySha1(S3Repository.java:126)

I have tried assuming the role from the commandline and executed aws s3api delete-object my-bucket with the same bucket and key that's causing the exception, and it works flawlessly.

Hints most welcome, banging head against wall here. Very strange that adding and reading works, but not deletion - using the same S3 client, in the same pod, right after eachother.

Currently awaiting S3 bucket logs, but have little hope they will help.

forsberg commented 1 year ago

Right, misconfiguration in my cluster. Nothing to see here!