search

tailwarden / komiser

Open-source cloud-environment inspector. Supporting AWS, GCP, Azure, and more! Your cloud resources will have nowhere to hide!
https://komiser.io
Other
3.92k stars 428 forks source link

Panic within GCP redis calculating costs #1418

Closed zdenekpizl-foxdeli closed 3 weeks ago

zdenekpizl-foxdeli commented 3 months ago

Describe the bug

panic: runtime error: index out of range [0] with length 0

goroutine 48 [running]:
github.com/tailwarden/komiser/providers/gcp/redis.calculateRedisCost(0xc0009623c0, 0xc00268a540?)
    /home/runner/work/komiser/komiser/providers/gcp/redis/pricing.go:79 +0x365
github.com/tailwarden/komiser/providers/gcp/redis.Instances({0x4231180, 0x6504b20}, {0x0, 0x0, {0x0, 0x0}, 0x0, 0x0, 0x0, 0x0, ...})
    /home/runner/work/komiser/komiser/providers/gcp/redis/instances.go:68 +0x714
github.com/tailwarden/komiser/providers/gcp.FetchResources.func1()
    /home/runner/work/komiser/komiser/providers/gcp/gcp.go:57 +0x99
github.com/tailwarden/komiser/providers.(*WorkerPool).worker(0xc000d9e000)
    /home/runner/work/komiser/komiser/providers/providers.go:85 +0x3a
created by github.com/tailwarden/komiser/providers.(*WorkerPool).Start in goroutine 50
    /home/runner/work/komiser/komiser/providers/providers.go:69 +0x25

To Reproduce Steps to reproduce the behavior:

  1. I run docker image with config.toml set to a GCP account
  2. it started, run fetching some objects apparently for a while and then died with message above

Expected behavior

Desktop (please complete the following information):

github-actions[bot] commented 3 months ago

Hey, thank you for opening your first Issue! 🙂 While a Tailwarden team member takes a look at your issue we would like to invite you to join our official Discord server, where you can interact directly with other contributors and Tailwarden team members. Link here: https://discord.tailwarden.com

zdenekpizl-foxdeli commented 3 months ago

attaching full logs from docker image run, it is interesting although the service's JSON key is for an account with compute/storage admin/owner rights, it reports issues whilst getting i.e. bucket size. Also, there are errors reported about disabled features. Why? Do not report disabled features, they are disabled on purpose and you should quietly skip them AFAIK.

INIT!!!
INFO[2024-05-13T17:56:58Z] Debug logging is enabled                     
INFO[2024-05-13T17:56:58Z] Data will be stored in SQLite                
INFO[2024-05-13T17:56:58Z] migrated to group #1 (20230413100000, 20230602100000, 20230619100000, 20231014100000) 
INFO[2024-05-13T17:56:58Z] Komiser version: 3.1.16, commit: 6732ea4912c9be443ae65a56826db1ea2429a7b3, buildt: 1714753332 
INFO[2024-05-13T17:56:58Z] Fetching resources workflow has started      
INFO[2024-05-13T17:56:58Z] Workflow triggered for client: StageCluster 
ERRO[2024-05-13T17:56:58Z] failed to create alloydb client               error="google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for more information"
INFO[2024-05-13T17:56:58Z] [StageCluster][GCP] google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for more information 
ERRO[2024-05-13T17:56:58Z] failed to create alloydb client               error="google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for more information"
INFO[2024-05-13T17:56:58Z] [StageCluster][GCP] google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for more information 
ERRO[2024-05-13T17:56:58Z] failed to collect labels                      error="rpc error: code = InvalidArgument desc = Location \"\" does not exist.\nerror details: name = RequestInfo id = 0x82aeaf58fc41e250 data ="
INFO[2024-05-13T17:56:58Z] [StageCluster][GCP] rpc error: code = InvalidArgument desc = Location "" does not exist.
error details: name = RequestInfo id = 0x82aeaf58fc41e250 data = 
ERRO[2024-05-13T17:56:58Z] failed to get app engine                      error="rpc error: code = NotFound desc = Requested entity was not found."
INFO[2024-05-13T17:56:58Z] Fetched resources                             account=StageCluster provider=GCP resources=0 service="App Engine"
ERRO[2024-05-13T17:56:58Z] failed to list all firestore collections      error="rpc error: code = NotFound desc = The database (default) does not exist for project StageCluster Please visit https://console.cloud.google.com/datastore/setup?project=StageCluster to add a Cloud Datastore or Cloud Firestore database."
INFO[2024-05-13T17:56:58Z] [StageCluster][GCP] rpc error: code = NotFound desc = The database (default) does not exist for project StageCluster Please visit https://console.cloud.google.com/datastore/setup?project=StageCluster to add a Cloud Datastore or Cloud Firestore database. 
INFO[2024-05-13T17:56:58Z] Fetched resources                             account=StageCluster provider=GCP resources=0 service="SQL Instances"
INFO[2024-05-13T17:56:58Z] Fetched resources                             account=StageCluster provider=GCP resources=13 service="IAM Service Accounts"
INFO[2024-05-13T17:56:58Z] Fetched resources                             account=StageCluster provider=GCP resources=0 service=BigQuery
INFO[2024-05-13T17:56:58Z] Fetched resources                             account=StageCluster provider=GCP resources=0 service=KMS
INFO[2024-05-13T17:56:59Z] Fetched resources                             account=StageCluster provider=GCP resources=0 service="Compute Disk Snapshot"
INFO[2024-05-13T17:56:59Z] Fetched resources                             account=StageCluster provider=GCP resources=0 service="IAM Custom Roles"
ERRO[2024-05-13T17:56:59Z] failed to get nex image                       error="rpc error: code = InvalidArgument desc = Invalid resource field value in the request.\nerror details: name = ErrorInfo reason = RESOURCE_PROJECT_INVALID domain = googleapis.com metadata = map[method:google.devtools.artifactregistry.v1.ArtifactRegistry.ListDockerImages service:artifactregistry.googleapis.com]"
INFO[2024-05-13T17:56:59Z] Fetched resources                             account=StageCluster provider=GCP resources=0 service="Artifacts Registry Docker Images"
ERRO[2024-05-13T17:56:59Z] failed to get next repo                       error="rpc error: code = InvalidArgument desc = Invalid resource field value in the request.\nerror details: name = ErrorInfo reason = RESOURCE_PROJECT_INVALID domain = googleapis.com metadata = map[method:google.devtools.artifactregistry.v1.ArtifactRegistry.ListRepositories service:artifactregistry.googleapis.com]"
INFO[2024-05-13T17:56:59Z] Fetched resources                             account=StageCluster provider=GCP resources=0 service="Artifacts Registry Repositories"
INFO[2024-05-13T17:56:59Z] Fetched resources                             account=StageCluster provider=GCP resources=0 service="Certificate Manager"
ERRO[2024-05-13T17:56:59Z] failed to get next package                    error="googleapi: Error 404:"
INFO[2024-05-13T17:56:59Z] Fetched resources                             account=StageCluster provider=GCP resources=0 service="Artifacts Registry Packages"
INFO[2024-05-13T17:56:59Z] Fetched resources                             account=StageCluster provider=GCP resources=0 service="Cloud Functions"
DEBU[2024-05-13T17:57:00Z] failed to get bucket size, skipping cost calculation  bucket=gitlab-runner-cache-2c2cd3e7 error="no data found"
INFO[2024-05-13T17:57:00Z] Fetched resources                             account=StageCluster provider=GCP resources=6 service="Compute Engine"
DEBU[2024-05-13T17:57:00Z] failed to get bucket size, skipping cost calculation  bucket=gitlab-runner-cache-4aa8eaf4 error="no data found"
DEBU[2024-05-13T17:57:01Z] failed to get bucket size, skipping cost calculation  bucket=gkestagebackup error="no data found"
DEBU[2024-05-13T17:57:01Z] failed to get bucket size, skipping cost calculation  bucket=private-data-sandbox error="no data found"
INFO[2024-05-13T17:57:02Z] Fetched resources                             account=StageCluster provider=GCP resources=19 service="Compute Disk"
INFO[2024-05-13T17:57:04Z] Fetched resources                             account=StageCluster provider=GCP resources=9 service=Bucket

panic: runtime error: index out of range [0] with length 0

goroutine 48 [running]:
github.com/tailwarden/komiser/providers/gcp/redis.calculateRedisCost(0xc0009623c0, 0xc00268a540?)
    /home/runner/work/komiser/komiser/providers/gcp/redis/pricing.go:79 +0x365
github.com/tailwarden/komiser/providers/gcp/redis.Instances({0x4231180, 0x6504b20}, {0x0, 0x0, {0x0, 0x0}, 0x0, 0x0, 0x0, 0x0, ...})
    /home/runner/work/komiser/komiser/providers/gcp/redis/instances.go:68 +0x714
github.com/tailwarden/komiser/providers/gcp.FetchResources.func1()
    /home/runner/work/komiser/komiser/providers/gcp/gcp.go:57 +0x99
github.com/tailwarden/komiser/providers.(*WorkerPool).worker(0xc000d9e000)
    /home/runner/work/komiser/komiser/providers/providers.go:85 +0x3a
created by github.com/tailwarden/komiser/providers.(*WorkerPool).Start in goroutine 50
    /home/runner/work/komiser/komiser/providers/providers.go:69 +0x25
Azanul commented 1 month ago

@zdenekpizl-foxdeli In which zone/location do you have your Redis instances?

zdenekpizl-foxdeli commented 1 month ago

Hi, location is europe-west3 aka Frankfurt, type is Basic, version is 6.x.

Regards ZP.

Azanul commented 1 month ago

@zdenekpizl-foxdeli Thanks a lot for pointing out the issues

  1. Redis price calculation: found & will be fixed in next release
  2. Bucket size: are your buckets empty?
  3. Error messages for disabled services: it does seem to be an expected error, we can show them as info or warning instead of error
zdenekpizl-foxdeli commented 1 month ago

Great, thank you.

ad buckets - yes, some of then could be empty because of lifecycle time by time.

Regards, ZP.