awslabs / data-solutions-framework-on-aws

An open-source framework that simplifies implementation of data solutions.
https://awslabs.github.io/data-solutions-framework-on-aws/
Apache License 2.0
105 stars 14 forks source link

Bug: kafkaApiMskIamProvider can't update properties in topic #676

Open armaseg opened 2 weeks ago

armaseg commented 2 weeks ago

When I try to update MSK topic properties (ex: num_partitions) I get an error related to Topic Authorization:

{
    "Status": "FAILED",
    "Reason": "Not authorized to access topics: [Topic authorization failed]\n\nLogs: /aws/lambda/MskIamAclStack-kafkaApiMskIamProviderOnEventHand-0SoRHAKIvYrI\n\n    at createErrorFromCode (/var/task/node_modules/kafkajs/src/protocol/error.js:581:10)\n    at Object.parse (/var/task/node_modules/kafkajs/src/protocol/requests/alterConfigs/v0/response.js:35:11)\n    at Connection.send (/var/task/node_modules/kafkajs/src/network/connection.js:433:35)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async [private:Broker:sendRequest] (/var/task/node_modules/kafkajs/src/broker/index.js:904:14)\n    at async Broker.alterConfigs (/var/task/node_modules/kafkajs/src/broker/index.js:631:12)\n    at async Promise.all (index 0)\n    at async /var/task/node_modules/kafkajs/src/admin/index.js:749:27\n    at async topicCrudOnEvent (/var/task/index.js:176:9)\n    at async Runtime.onEventHandler [as handler] (/var/task/index.js:288:29)",
    "StackId": "arn:aws:cloudformation:eu-west-1:123456789:stack/MskIamAclStack/e9c0bcc0-2cb5-11ef-bc01-063483407eaa",
    "RequestId": "b9f398ed-255a-4491-a9f3-cc5892f5dac7",
    "PhysicalResourceId": "85b4a0d2-3d93-4f33-8498-fb491545a568",
    "LogicalResourceId": "kafkaApinewtopic15A0F755B"
}

The error of kafkajs is:

{
    "level": "ERROR",
    "timestamp": "2024-06-20T14:59:09.924Z",
    "logger": "kafkajs",
    "message": "[Connection] Response AlterConfigs(key: 33, version: 1)",
    "broker": "b-3.foo.c9.kafka.eu-west-1.amazonaws.com:9098",
    "clientId": "client-CR-my-cluster-3-6",
    "error": "Not authorized to access topics: [Topic authorization failed]",
    "correlationId": 4,
    "size": 57
}