search

waliaabhishek / ccloud-chargeback-helper

Confluent Cloud Billing Chargeback Helper
https://github.com/waliaabhishek/ccloud-chargeback-helper
MIT License
28 stars 9 forks source link

Container ccloud_chargeback_handler exit #5

Closed eddycastillon closed 1 year ago

eddycastillon commented 1 year ago

Hello @waliaabhishek

I am trying to explore this amazing feature for CC, however I stuck when trying to deploy the containers. After setup my env file a container is dead. I am deploying from master.

After execute :

docker-compose --env-file .my-env up --remove-orphans

The container called ccloud_chargeback_handler dead. Reviewing the logs it shows the following

exec /usr/local/bin/python: exec format error ccloud_chargeback_handler exited with code 1

No sure exactly how to handle this issue

Regards Eddy C

eddycastillon commented 1 year ago

I built the image again using the Dockerfile and it fixed my issue.

Thank you.

waliaabhishek commented 1 year ago

Are you running on windows by any chance ? They latest docker images only support linux/amd64 and linux/arm64 at the moment.

eddycastillon commented 1 year ago

It is a good point, I am running on wsl. Currently I fixed the issue, but I am not able to see data on grafana.

c9c6ef8b3b3d   prom/prometheus:v2.44.0                "/bin/sh -c 'chmod +…"   3 hours ago    Up 2 hours   9090/tcp                                                        prometheus_feeder
51d85236c26f   grafana/grafana:10.0.1                 "/run.sh"                3 hours ago    Up 2 hours   0.0.0.0:3000->3000/tcp, :::3000->3000/tcp                       grafana
d1fd5be6248b   ecc/ccloud_chargeback_handler:v1.0.2   "python main.py --co…"   3 hours ago    Up 2 hours   0.0.0.0:8000-8001->8000-8001/tcp, :::8000-8001->8000-8001/tcp   ccloud_chargeback_handler
de4420d13c7e   prom/prometheus:v2.44.0                "/bin/prometheus --c…"   11 hours ago   Up 2 hours   0.0.0.0:9091->9090/tcp, :::9091->9090/tcp                       prometheus_for_chargeback```

What could be I wrong? - I do not have a dedicated metric API, so I follow the comments.

CCLOUD_LOOKBACK_DAYS=200
CCLOUD_BILLING_API_KEY=abc 
CCLOUD_BILLING_API_SECRET=123
METRICS_API_SERVER_URL=http://prometheus_for_chargeback:9090 
METRICS_API_SERVER_ENABLE_AUTH=False
METRICS_API_SERVER_AUTH_TYPE=HTTPBasicAuth                              
METRICS_API_SERVER_AUTH_USERNAME=admin
METRICS_API_SERVER_AUTH_PASSWORD=password
CHARGEBACK_SERVER_URL=http://prometheus_for_chargeback:9090

From the ccloud_chargeback_handler's boot logs I see it connects to CC using the API_KEY and SECRET

2023-07-25 20:59:07,498 ccloud.org                INFO     Initial Fetch Date after checking chargeback status in Prometheus: 2023-01-06 21:00:00+00:00
2023-07-25 20:59:07,498 data_processing.data_handlers.ccloud_api_handler INFO     Starting CCloud Object refresh now -- 2023-07-25 20:59:07.498386
2023-07-25 20:59:07,498 data_processing.data_handlers.ccloud_api_handler INFO     Refreshing CCloud Service Accounts
2023-07-25 20:59:07,498 ccloud.connections        INFO     Reading from API: https://api.confluent.cloud/iam/v2/service-accounts
2023-07-25 20:59:07,805 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:07] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:07,807 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:07] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:08,533 ccloud.connections        INFO     Found 13 items in API response.
2023-07-25 20:59:08,535 ccloud.ccloud_api.service_accounts INFO     CCloud Service Accounts initialized successfully
2023-07-25 20:59:08,535 data_processing.data_handlers.ccloud_api_handler INFO     Refreshing CCloud User Accounts
2023-07-25 20:59:08,535 ccloud.connections        INFO     Reading from API: https://api.confluent.cloud/iam/v2/users
2023-07-25 20:59:09,390 ccloud.connections        INFO     Found 63 items in API response.
2023-07-25 20:59:09,395 ccloud.ccloud_api.user_accounts INFO     CCloud Users initialized successfully
2023-07-25 20:59:09,395 data_processing.data_handlers.ccloud_api_handler INFO     Refreshing CCloud API Keys
2023-07-25 20:59:09,395 ccloud.connections        INFO     Reading from API: https://api.confluent.cloud/iam/v2/api-keys
2023-07-25 20:59:10,602 ccloud.connections        INFO     Found 53 items in API response.
2023-07-25 20:59:10,606 ccloud.ccloud_api.api_keys INFO     CCloud API Keys initialized successfully
2023-07-25 20:59:10,606 data_processing.data_handlers.ccloud_api_handler INFO     Refreshing CCloud Environments
2023-07-25 20:59:10,606 ccloud.connections        INFO     Reading from API: https://api.confluent.cloud/org/v2/environments
2023-07-25 20:59:10,809 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:10] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:11,491 ccloud.connections        INFO     Found 1 items in API response.
2023-07-25 20:59:11,492 ccloud.ccloud_api.environments INFO     CCloud Environment List initialized successfully
2023-07-25 20:59:11,492 data_processing.data_handlers.ccloud_api_handler INFO     Refreshing CCloud Kafka Clusters
2023-07-25 20:59:11,492 ccloud.ccloud_api.clusters INFO     Checking CCloud Environment env-xxxo for any provisioned Kafka Clusters.
2023-07-25 20:59:11,492 ccloud.connections        INFO     Reading from API: https://api.confluent.cloud/cmk/v2/clusters
2023-07-25 20:59:12,442 ccloud.connections        INFO     Found 6 items in API response.
2023-07-25 20:59:12,445 ccloud.ccloud_api.clusters INFO     CCloud Kafka Clusters initialized successfully
2023-07-25 20:59:12,445 data_processing.data_handlers.ccloud_api_handler INFO     Refreshing CCloud Connectors
2023-07-25 20:59:12,445 ccloud.ccloud_api.connectors INFO     Checking Environment env-xxxo for any provisioned connectors.
2023-07-25 20:59:13,553 ccloud.ccloud_api.connectors INFO     Checking Environment env-xxxo for any provisioned connectors.
2023-07-25 20:59:13,814 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:13] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:14,619 ccloud.ccloud_api.connectors INFO     Checking Environment env-xxxo for any provisioned connectors.
2023-07-25 20:59:15,744 ccloud.ccloud_api.connectors INFO     Checking Environment env-xxxo for any provisioned connectors.
2023-07-25 20:59:16,821 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:16] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:16,863 ccloud.ccloud_api.connectors INFO     Checking Environment env-xxxo for any provisioned connectors.
2023-07-25 20:59:17,689 ccloud.ccloud_api.connectors INFO     Checking Environment env-xxxo for any provisioned connectors.
2023-07-25 20:59:18,605 ccloud.ccloud_api.connectors INFO     CCloud Kafka Connectors initialized successfully
2023-07-25 20:59:18,605 data_processing.data_handlers.ccloud_api_handler INFO     Refreshing CCloud KSQLDB Clusters
2023-07-25 20:59:18,605 ccloud.ccloud_api.ksqldb_clusters INFO     Checking CCloud Environment env-xxxo for any provisioned ksqlDB Clusters.
2023-07-25 20:59:18,605 ccloud.connections        INFO     Reading from API: https://api.confluent.cloud/ksqldbcm/v2/clusters
2023-07-25 20:59:19,554 ccloud.connections        INFO     Found 1 items in API response.
2023-07-25 20:59:19,557 ccloud.ccloud_api.ksqldb_clusters INFO     CCloud ksqlDB Cluster initialized successfully
2023-07-25 20:59:19,557 data_processing.data_handlers.ccloud_api_handler INFO     Finished CCloud Object refresh -- 2023-07-25 20:59:19.557486
2023-07-25 20:59:19,557 data_processing.data_handlers.ccloud_api_handler INFO     Reading Objects dataset for Timestamp: 2023-01-06 21:00:00+00:00
2023-07-25 20:59:19,558 data_processing.data_handlers.billing_api_handler INFO     Initialized the Billing API Handler with URL: https://api.confluent.cloud/billing/v1/costs
2023-07-25 20:59:19,559 ccloud.connections        INFO     Reading from API: https://api.confluent.cloud/billing/v1/costs
2023-07-25 20:59:19,828 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:19] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:20,563 ccloud.connections        INFO     Found 0 items in API response.
2023-07-25 20:59:20,567 data_processing.data_handlers.billing_api_handler INFO     Initialized the Billing API Handler with last available date: 2023-01-13 21:00:00+00:00
2023-07-25 20:59:20,567 data_processing.data_handlers.prom_metrics_api_handler INFO     Setting up Auth Type Supplied by the config file
2023-07-25 20:59:20,574 workflow_runner           INFO     Shutting down all threads
2023-07-25 20:59:20,574 workflow_runner           INFO     Waiting for State Sync ticker for Final sync before exit
Traceback (most recent call last):
  File "/app/main.py", line 23, in <module>
    execute_workflow(arg_flags)
  File "/app/helpers.py", line 35, in add_entry_exit_logs
    ret = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/app/workflow_runner.py", line 134, in execute_workflow
    ccloud_orgs = CCloudOrgList(
                  ^^^^^^^^^^^^^^
  File "<string>", line 4, in __init__
  File "/app/ccloud/org.py", line 225, in __post_init__
    temp = CCloudOrg(
           ^^^^^^^^^^
  File "<string>", line 4, in __init__
  File "/app/ccloud/org.py", line 127, in __post_init__
    self.chargeback_handler = CCloudChargebackHandler(
                              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 10, in __init__
  File "/app/data_processing/data_handlers/chargeback_handler.py", line 85, in __post_init__
    self.read_all(start_date=self.start_date, end_date=self.last_available_date)
  File "/app/helpers.py", line 35, in add_entry_exit_logs
    ret = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/app/data_processing/data_handlers/chargeback_handler.py", line 148, in read_all
    self.compute_output(time_slice=time_slice_item)
  File "/app/helpers.py", line 35, in add_entry_exit_logs
    ret = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/app/data_processing/data_handlers/chargeback_handler.py", line 273, in compute_output
    bill_row.Index[0].to_pydatetime(),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'to_pydatetime'
2023-07-25 20:59:22,835 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:22] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:25,838 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:25] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:28,845 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:28] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:31,848 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:31] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:34,853 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:34] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:37,859 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:37] "GET /is_ready HTTP/1.1" 200 -
2023-07-25 20:59:40,863 werkzeug                  INFO     172.25.0.5 - - [25/Jul/2023 20:59:40] "GET /is_ready HTTP/1.1" 200 -
eddycastillon commented 1 year ago

On my Ubuntu 22.04 I got the same error related to "exec /usr/local/bin/python: exec format error ". Anyway, I fixed the issue building again the image. So I have running the docker images in wsl and linux but neither show me data in grafana.

waliaabhishek commented 1 year ago

Not sure whats going on but I tried to update my docker images and added a small fix. Everything works on my system even when I delete the local images and fetch from Docker. But then again I am using a Mac to test and it might be working there. Can you pull the 1.0.3 tag and try it out ?

eddycastillon commented 1 year ago

Using the new image, the containers started without problems. However I am not able to see data related to costs in grafana.

waliaabhishek commented 1 year ago

The default data pull starts with 200 days ago. The average has been about a minute per day for all the calculations to be performed. Also, check the logs for the prometheus_feeder pod. That should be running and feeding data to prometheus.