Open jcarvalho opened 3 months ago
I can confirm by patching the amazon_msk.py
file to add the following check in this line causes the integration to work correctly again:
if not 'BrokerNodeInfo' in node_info:
continue
Would be great to get this upstream to remove the local patch :slightly_smiling_face:
Would love to see this get some progress!
+1 on this issue, would really love to see this addressed!
Amazon MSK has recently launched support for KRaft Clusters, which adds the Controller Nodes to the output of the
ListNodes
API Call.These node entries do not have a
brokerNodeInfo
entry, which causes the Agent Integration to crash with the following error:Ideally, the integration should also scrape the Controller Nodes (which may also expose Prometheus metrics), but it would at least be great to still support scraping the Broker Nodes when KRaft is in use.
Output of the info page
(Posting only relevant version + check information, happy to share more details over DM)
Additional environment details (Operating System, Cloud provider, etc):
Steps to reproduce the issue:
Describe the results you received:
The check fails with the exception above and no metrics are published to Datadog.
Describe the results you expected:
Ideally: The metrics for both the Controller and the Brokers are published to Datadog. Desired: The metrics for the Brokers are published to Datadog.
Additional information you deem important (e.g. issue happens only occasionally):
Returned data for the
ListNodes
call in our KRaft-enabled cluster (redacted URLs and Account/Subnet IDs):