[metrics-report] Airbyte Metrics BigDecimal to Double Error

[wjmolina]

## Environment - **Airbyte version**: 0.41.0 - **OS Version / Instance**: Amazon Linux 2 - **Deployment**: Kubernetes 1.18.10 - **Source Connector and version**: N/A - **Destination Connector and version**: N/A - **Step where error happened**: Sync job ## Current Behavior I set ``` airbyte: metrics: enabled: true extraEnv: - name: METRIC_CLIENT value: datadog - name: DD_AGENT_HOST value: datadog-agent-cluster-agent.monitoring - name: DD_DOGSTATSD_PORT value: "5005" - name: PUBLISH_METRICS value: "true" ``` but the only thing being logged to DataDog is ``` EST_NUM_METRICS_EMITTED_BY_REPORTER ``` But I also see the following error: ``` 2023-04-06 00:26:23 ERROR i.a.m.r.Emitter(Emit):210 - Exception querying database for metric: java.lang.ClassCastException: class java.math.BigDecimal cannot be cast to class java.lang.Double (java.math.BigDecimal and java.lang.Double are in module java.base of loader 'bootstrap') at java.util.LinkedHashMap.forEach(LinkedHashMap.java:729) ~[?:?] at io.airbyte.metrics.reporter.OldestRunningJob.lambda$new$1(Emitter.java:67) ~[io.airbyte.airbyte-metrics-reporter-0.41.0.jar:?] at io.airbyte.metrics.reporter.Emitter.Emit(Emitter.java:207) ~[io.airbyte.airbyte-metrics-reporter-0.41.0.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577) ~[?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?] at java.lang.Thread.run(Thread.java:1589) ~[?:?] ``` ## Expected Behavior The exception is gone and I see the rest of the metrics. ## Logs I included them above.

Steps to Reproduce

I included them above.

Are you willing to submit a PR?

Yes

[lisaah]

Also see this on 0.44.5 on sync job, Amazon Linux 2 EBS deployment (using docker-compose).

[molloyjj]

➕ Also seeing this for metrics collected using otel airbyte version 0.45.22 airbyte/metrics-reporter:0.44.10. Error is being thrown in the metrics reporter.

[molloyjj]

Anyone who is still experiencing this, I resolved by just using the helm chart configuration and deploying my own otel pods.

I found by following this guide, it resulted in the first reported error. I am guessing there are some unknown version constraints which are causing the issue. Would be helpful in the future if there was a more up to date guide that can be followed.

This was the env generated for the metrics container that worked (airbyte version 0.45.22).

  containers:
    - env:
        - name: AIRBYTE_VERSION
          valueFrom:
            configMapKeyRef:
              key: AIRBYTE_VERSION
              name: airbyte-airbyte-env
        - name: DATABASE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: password
              name: airbyte-cloudsql-user-password
        - name: DATABASE_URL
          valueFrom:
            configMapKeyRef:
              key: DATABASE_URL
              name: airbyte-airbyte-env
        - name: DATABASE_USER
          valueFrom:
            secretKeyRef:
              key: DATABASE_USER
              name: airbyte-airbyte-secrets
        - name: CONFIGS_DATABASE_MINIMUM_FLYWAY_MIGRATION_VERSION
          valueFrom:
            configMapKeyRef:
              key: CONFIGS_DATABASE_MINIMUM_FLYWAY_MIGRATION_VERSION
              name: airbyte-airbyte-env
        - name: METRIC_CLIENT
          valueFrom:
            configMapKeyRef:
              key: METRIC_CLIENT
              name: airbyte-airbyte-env
        - name: OTEL_COLLECTOR_ENDPOINT
          valueFrom:
            configMapKeyRef:
              key: OTEL_COLLECTOR_ENDPOINT
              name: airbyte-airbyte-env
        - name: PUBLISH_METRICS
          value: 'true'
      image: 'airbyte/metrics-reporter:0.40.3'