search

prometheus-community / helm-charts

Prometheus community Helm charts
Apache License 2.0
5.12k stars 5.02k forks source link

error msg="Failed to get regions" when create datasource #4928

Open baduba opened 3 weeks ago

baduba commented 3 weeks ago

Hi, I updated the tag and after updating, I lost communication between grafana and cloudwatch, I enabled debug mode and saw get regions errors, I configured helm to automatically upload the datasource with region and it worked, after a few days it stopped working and didn't work. I can't get it to work after that, I've already recreated the environment several times, I've already returned the stack version to 60 I've already followed the manual below to see if I didn't make a mistake somewhere and everything is correct, it just stopped working. https://plainenglish.io/blog/understanding-webidentityerr-error-in-aws-lb

logger=accesscontrol t=2024-10-21T15:23:24.325086981Z level=debug msg="Evaluating permissions" id=user:1 orgID=1 permissions="action:datasources:query scopes:"
logger=secrets.kvstore t=2024-10-21T15:23:24.325147093Z level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=CloudWatchTest2
logger=tsdb.cloudwatch endpoint=callResource pluginId=cloudwatch dsName=CloudWatchTest2 dsUID=cloudwatch2 uname=admin t=2024-10-21T15:23:24.335618333Z level=error msg="Error handling resource request" error="error getting accounts for current user or role: ListSinks error: WebIdentityErr: failed to retrieve credentials\ncaused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity\n\tstatus code: 403, request id: 77b679a2-1048-4e69-8e82-9866891cfb94"
logger=context userId=1 orgId=1 uname=admin t=2024-10-21T15:23:24.335706829Z level=error msg="Request Completed" method=GET path=/api/datasources/6/resources/accounts status=500 r time_ms=16 duration=16.253871ms size=521 referer=https://xxxxxxx/grafana/connections/datasources/edit/cloudwatch2 handler=/api/datasources/:id/resources/* status_source=downstream
logger=tsdb.cloudwatch endpoint=callResource pluginId=cloudwatch dsName=CloudWatchTest2 dsUID=cloudwatch2 uname=admin t=2024-10-21T15:23:24.343068525Z level=error msg="Failed to get regions: " error="WebIdentityErr: failed to retrieve credentials\ncaused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity\n\tstatus code: 403, request id: f739b2b2-8103-4539-bc3c-ed5c3277309f"
logger=sqlstore t=2024-10-21T15:23:26.094048802Z level=debug msg="ReadReplica not configured, using main SQLStore"
logger=sqlstore t=2024-10-21T15:23:26.094921065Z level=debug msg="ReadReplica not configured, using main SQLStore"
logger=sqlstore t=2024-10-21T15:23:26.09568902Z level=debug msg="ReadReplica not configured, using main SQLStore"
logger=accesscontrol t=2024-10-21T15:23:26.097495331Z level=debug msg="Evaluating permissions" id=user:1 orgID=1 permissions="action:datasources:query scopes:"
logger=datasources t=2024-10-21T15:23:26.097521823Z level=debug msg="Querying for data source via SQL store" uid=cloudwatch2 orgId=1
logger=secrets.kvstore t=2024-10-21T15:23:26.097979448Z level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=CloudWatchTest2