grafana / cloudcost-exporter

Prometheus Exporter for Cloud Provider agnostic cost metrics
Apache License 2.0
66 stars 4 forks source link

Compute + GKE Missing us-east1 Region #96

Closed Pokom closed 9 months ago

Pokom commented 9 months ago

When generating the pricing map, we're missing out on pricing information for us-east1 compute instances. All of the sku's properly to parse for the pricing map, so it's not the regex to find and filter SKU's/ One of the methods we have is getPricingInfoFromSku: https://github.com/grafana/cloudcost-exporter/blob/main/pkg/google/compute/pricing_map.go#L231-L238 This makes the assumption that the first pricing is the correct one. This works for most of the compute instances except those that exist in us-east. My hunch is that us-east1 machines are the same price as other regional compute instances within the US, so we need to better handle multiple regions in the pricing.

I altered the execution of the program to print out all of the sku's with multiple regions and output log file and uploaded it: log.txt

There's 6235 sku's:

cat log.txt| wc -l

Let's filter out sku's that aren't networking:

cat log.txt | grep -v -i network | wc -l
239

Okay so 239 sku's is much more favorable then 6235. Here's a sample:

Sku: name:"services/6F81-5844-456A/skus/F33C-AA89-DE6D" sku_id:"F33C-AA89-DE6D" description:"Spot Preemptible C3D Instance Ram running in Americas" category:{service_display_name:"Compute Engine" resource_family:"Compute" resource_group:"RAM" usage_type:"Preemptible"} service_regions:"us-central1" service_regions:"us-east1" service_regions:"us-west1" pricing_info:{effective_time:{seconds:1706607872 nanos:279308000} pricing_expression:{usage_unit:"GiBy.h" display_quantity:1 tiered_rates:{unit_price:{currency_code:"USD" nanos:1346000}} usage_unit_description:"gibibyte hour" base_unit:"By.s" base_unit_description:"byte second" base_unit_conversion_factor:3.8654705664e+12} currency_conversion_rate:1} service_provider_name:"Google" geo_taxonomy:{type:MULTI_REGIONAL regions:"us-central1" regions:"us-east1" regions:"us-west1"}
Sku: name:"services/6F81-5844-456A/skus/F449-33EC-A5EF" sku_id:"F449-33EC-A5EF" description:"E2 Instance Ram running in Americas" category:{service_display_name:"Compute Engine" resource_family:"Compute" resource_group:"RAM" usage_type:"OnDemand"} service_regions:"us-central1" service_regions:"us-east1" service_regions:"us-west1" pricing_info:{effective_time:{seconds:1706607872 nanos:279308000} pricing_expression:{usage_unit:"GiBy.h" display_quantity:1 tiered_rates:{unit_price:{currency_code:"USD" nanos:2923530}} usage_unit_description:"gibibyte hour" base_unit:"By.s" base_unit_description:"byte second" base_unit_conversion_factor:3.8654705664e+12} currency_conversion_rate:1} service_provider_name:"Google" geo_taxonomy:{type:MULTI_REGIONAL regions:"us-central1" regions:"us-east1" regions:"us-west1"}
Sku: name:"services/6F81-5844-456A/skus/FB05-036A-8982" sku_id:"FB05-036A-8982" description:"T2D AMD Instance Ram running in Americas" category:{service_display_name:"Compute Engine" resource_family:"Compute" resource_group:"RAM" usage_type:"OnDemand"} service_regions:"us-central1" service_regions:"us-east1" service_regions:"us-west1" pricing_info:{effective_time:{seconds:1706607872 nanos:279308000} pricing_expression:{usage_unit:"GiBy.h" display_quantity:1 tiered_rates:{unit_price:{currency_code:"USD" nanos:3686000}} usage_unit_description:"gibibyte hour" base_unit:"By.s" base_unit_description:"byte second" base_unit_conversion_factor:3.8654705664e+12} currency_conversion_rate:1} service_provider_name:"Google" geo_taxonomy:{type:MULTI_REGIONAL regions:"us-central1" regions:"us-east1" regions:"us-west1"}
Sku: name:"services/6F81-5844-456A/skus/FCA2-37F3-16BA" sku_id:"FCA2-37F3-16BA" description:"Commitment v1: N2 Cpu in Americas for 3 Year" category:{service_display_name:"Compute Engine" resource_family:"Compute" resource_group:"CPU" usage_type:"Commit3Yr"} service_regions:"us-central1" service_regions:"us-east1" service_regions:"us-west1" pricing_info:{effective_time:{seconds:1706607872 nanos:279308000} pricing_expression:{usage_unit:"h" display_quantity:1 tiered_rates:{unit_price:{currency_code:"USD" nanos:14225000}} usage_unit_description:"hour" base_unit:"s" base_unit_description:"second" base_unit_conversion_factor:3600} currency_conversion_rate:1} service_provider_name:"Google" geo_taxonomy:{type:MULTI_REGIONAL regions:"us-central1" regions:"us-east1" regions:"us-west1"}
Sku: name:"services/6F81-5844-456A/skus/FCC7-A462-88DC" sku_id:"FCC7-A462-88DC" description:"Commitment v1: M3 Memory-optimized Cpu in Americas for 3 Years" category:{service_display_name:"Compute Engine" resource_family:"Compute" resource_group:"CPU" usage_type:"Commit3Yr"} service_regions:"us-central1" service_regions:"us-east1" service_regions:"us-west1" pricing_info:{effective_time:{seconds:1706607872 nanos:279308000} pricing_expression:{usage_unit:"h" display_quantity:1 tiered_rates:{unit_price:{currency_code:"USD" nanos:10450000}} usage_unit_description:"hour" base_unit:"s" base_unit_description:"second" base_unit_conversion_factor:3600} currency_conversion_rate:1} service_provider_name:"Google" geo_taxonomy:{type:MULTI_REGIONAL regions:"us-central1" regions:"us-east1" regions:"us-west1"}

So the very short of it is we need to handle parsing sku's with multiple regions in such a way that ensures each region gets a princingInfo object added to it.