search

nerdswords / yet-another-cloudwatch-exporter

Prometheus exporter for AWS CloudWatch - Discovers services through AWS tags, gets CloudWatch metrics data and provides them as Prometheus metrics with AWS tags as labels
Apache License 2.0
959 stars 330 forks source link

DynamoDB config #1109

Open modship opened 1 year ago

modship commented 1 year ago

Is there an existing issue for this?

YACE version

0.53.0 (docker)

Config file

apiVersion: v1alpha1
discovery:
  jobs:
    - type: AWS/Lambda
      regions:
        - eu-west-3
      period: 300
      length: 300
      metrics:
        - name: Invocations
          statistics: [Sum]
        - name: Errors
          statistics: [Sum]
        - name: Throttles
          statistics: [Sum]
        - name: Duration
          statistics: [Average, Maximum, Minimum, p90]
    - type: AWS/DynamoDB
      dimensionNameRequirements:
        - TableName
        - GlobalSecondaryIndexName
        - Operation
      regions:
        - eu-west-3
      period: 300
      length: 300
      metrics:
        - name: SuccessfulRequestLatency
          statistics: [Average]

Current Behavior

It might not be a bug, but I'm having trouble getting the scraping to work for AWS/DynamoDB (It works fine with: https://github.com/prometheus/cloudwatch_exporter/tree/master and this configuration: https://github.com/prometheus/cloudwatch_exporter/blob/master/examples/DynamoDB.yml).

error : 

2023-08-17 16:29:31 {"account":"XXXXXXX","arn":"","error":"expected to discover resources but none were found","job_type":"AWS/DynamoDB","level":"error","msg":"No tagged resources made it through filtering","region":"eu-west-3","time":"2023-08-17T14:29:31Z"}

Expected Behavior

metrics of dynamoDB exported

Steps To Reproduce

No response

Anything else?

No response

vinbh commented 1 year ago

+1 I have the same issue, yace wont get DynamoDB metrics. CLoudwatch exporter with above config works.

modship commented 1 year ago

@vinbh i have solved for now with that : (instead of "discovery")

 customNamespace:
    - name: AWS/DynamoDB
      namespace: AWS/DynamoDB
      regions:
        - eu-west-3
      metrics:
        - name: SuccessfulRequestLatency
          statistics:
            - Average
        - name: ThrottledRequests
          statistics:
            - Sum
        ...
kimdubi commented 10 months ago

@modship hi, Can you share the whole config yaml? I really need it. Thank you

modship commented 10 months ago

@modship hi, Can you share the whole config yaml? I really need it. Thank you

im using ansible

yace_exporter_configuration:
  apiVersion: v1alpha1
  discovery:
    jobs:
      - type: AWS/EC2
        regions:
          - eu-west-3
        period: 300
        length: 300
        metrics:
          - name: CPUUtilization
            statistics: [Average]
          - name: NetworkIn
            statistics: [Average, Sum]
          - name: NetworkOut
            statistics: [Average, Sum]
          - name: NetworkPacketsIn
            statistics: [Sum]
          - name: NetworkPacketsOut
            statistics: [Sum]
          - name: DiskReadBytes
            statistics: [Sum]
          - name: DiskWriteBytes
            statistics: [Sum]
          - name: DiskReadOps
            statistics: [Sum]
          - name: DiskWriteOps
            statistics: [Sum]
          - name: StatusCheckFailed
            statistics: [Sum]
          - name: StatusCheckFailed_Instance
            statistics: [Sum]
          - name: StatusCheckFailed_System
            statistics: [Sum]
      - type: AWS/Lambda
        regions:
          - eu-west-3
        period: 300
        length: 300
        metrics:
          - name: Invocations
            statistics: [Sum]
          - name: Errors
            statistics: [Sum]
          - name: Throttles
            statistics: [Sum]
          - name: Duration
            statistics: [Average, Maximum, Minimum, p90]
      - type: AWS/ApiGateway
        regions:
          - eu-west-3
        period: 300
        length: 300
        metrics:
          - name: Latency
            statistics: [Average, Maximum, p95, p99]
          - name: Count
            statistics: [SampleCount, Sum]
          - name: 4xx
            statistics: [Sum]
          - name: 4XXError
            statistics: [Sum]
          - name: 5xx
            statistics: [Sum]
          - name: 5XXError
            statistics: [Sum]
          - name: IntegrationLatency
            statistics: [Average]
  customNamespace:
    - name: Bridge
      namespace: Bridge
      regions:
        - eu-west-3
      metrics:
        - name: bridge-owner-xma-balance
          statistics:
            - Average
          period: 300
          length: 300
        - name: bridge-owner-eth-balance
          statistics:
            - Average
          period: 300
          length: 300
        - name: tDAI-supply-diff
          statistics:
            - Average
          period: 300
          length: 300
        - name: WETH-supply-diff
          statistics:
            - Average
          period: 300
          length: 300
        - name: failed-burnRedeem
          statistics:
            - Average
          period: 300
          length: 300
        - name: failed-lockMint
          statistics:
            - Average
          period: 300
          length: 300
    - name: AWS/DynamoDB
      namespace: AWS/DynamoDB
      regions:
        - eu-west-3
      metrics:
        - name: SuccessfulRequestLatency
          statistics:
            - Average
        - name: ThrottledRequests
          statistics:
            - Sum
        - name: UserErrors
          statistics:
            - Sum
        - name: ConsumedReadCapacityUnits
          statistics:
            - Sum
        - name: ConsumedWriteCapacityUnits
          statistics:
            - Sum
        - name: ProvisionedReadCapacityUnits
          statistics:
            - Average
        - name: ProvisionedWriteCapacityUnits
          statistics:
            - Average
        - name: ReturnedBytes
          statistics:
            - Average
        - name: ReturnedItemCount
          statistics:
            - Sum
kimdubi commented 10 months ago

@modship thanks for your response. below is my configuration. but it never collects SuccessfulRequestLatency metric. Do you have any doubts about what the problem might be? I really need it .

    discovery:
          jobs:
          - type: dynamodb
            regions:
              - ap-northeast-2
            enableMetricData: true
            metrics:          
              - name: ThrottledRequests   
                statistics:
                - Sum
                period: 60
                length: 60         
              - name: ConsumedReadCapacityUnits
                statistics:
                - Sum
                period: 60
                length: 60        
              - name: ProvisionedReadCapacityUnits       
                statistics:
                - Sum
                period: 60
                length: 60     
              - name: ConsumedWriteCapacityUnits      
                statistics:
                - Sum
                period: 60
                length: 60
              - name: ProvisionedWriteCapacityUnits      
                statistics:
                - Sum
                period: 60
                length: 60          
    customNamespace:
          - name: dynamodb
            namespace: AWS/DynamoDB
            regions:
              - ap-northeast-2 
            metrics:
              - name: SuccessfulRequestLatency
                statistics:
                - Average
                period: 60
                length: 60
modship commented 10 months ago

@modship thanks for your response. below is my configuration. but it never collects SuccessfulRequestLatency metric. Do you have any doubts about what the problem might be? I really need it .

    discovery:
          jobs:
          - type: dynamodb
            regions:
              - ap-northeast-2
            enableMetricData: true
            metrics:          
              - name: ThrottledRequests   
                statistics:
                - Sum
                period: 60
                length: 60         
              - name: ConsumedReadCapacityUnits
                statistics:
                - Sum
                period: 60
                length: 60        
              - name: ProvisionedReadCapacityUnits       
                statistics:
                - Sum
                period: 60
                length: 60     
              - name: ConsumedWriteCapacityUnits      
                statistics:
                - Sum
                period: 60
                length: 60
              - name: ProvisionedWriteCapacityUnits      
                statistics:
                - Sum
                period: 60
                length: 60          
    customNamespace:
          - name: dynamodb
            namespace: AWS/DynamoDB
            regions:
              - ap-northeast-2 
            metrics:
              - name: SuccessfulRequestLatency
                statistics:
                - Average
                period: 60
                length: 60

maybe your indentation for - Average And name : AWS/DynamoDB

kimdubi commented 10 months ago

@modship This is my final revision. However, the SuccessfulRequestLatency metric is still not being collected. Anyway, thank you for your help!

    discovery:
          jobs:
          - type: dynamodb
            regions:
              - ap-northeast-2
            enableMetricData: true
            metrics:          
              - name: ThrottledRequests   
                statistics:
                - Sum
                period: 60
                length: 60         
              - name: ConsumedReadCapacityUnits
                statistics:
                - Sum
                period: 60
                length: 60        
              - name: ProvisionedReadCapacityUnits       
                statistics:
                - Sum
                period: 60
                length: 60     
              - name: ConsumedWriteCapacityUnits      
                statistics:
                - Sum
                period: 60
                length: 60
              - name: ProvisionedWriteCapacityUnits      
                statistics:
                - Sum
                period: 60
                length: 60          
    customNamespace:
          - name: AWS/DynamoDB
            namespace: AWS/DynamoDB
            regions:
              - ap-northeast-2        
            metrics:
              - name: SuccessfulRequestLatency
                statistics:
                  - Average
                period: 60
                length: 60