search

tomkerkhove / promitor

Bringing Azure Monitor metrics where you need them.
https://promitor.io
MIT License
249 stars 91 forks source link

Failure to create job name for scraping resources for list of metrics which includes cpu_percent #2101

Closed kad-meedel closed 2 years ago

kad-meedel commented 2 years ago

Report

I changed the promitor version from 2.6.1 to 2.7.0, and it failed to start with the following error(s)

Failure to create job name for scraping resources for list of metrics which includes cpu_percent

System.Globalization.CultureNotFoundException: Only the invariant culture is supported in globalization-invariant mode. See https://aka.ms/GlobalizationInvariantMode for more information. (Parameter 'name') en-US is an invalid culture identifier.

The validation of Promitor setup is ok!

But the next error is: [09:41:06 FTL] Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. System.Exception: Resource Discovery is not configured at GuardNet.Guard.For[TException](Func1 predicate, TException exception) at GuardNet.Guard.For[TException](Func1 predicate, String message) at Promitor.Agents.Scraper.Discovery.ResourceDiscoveryClient..ctor(HttpClient httpClient, IOptionsMonitor1 configuration, ILogger1 logger) in /src/Promitor.Agents.Scraper/ResourceDiscoveryClient.cs:line 29 ........ ........

Expected Behavior

changed back to version 2.6.1 and all is ok again

Actual Behavior

promitor wont start. See Report section

Steps to Reproduce the Problem

Use version 2.7.0 ( if your environment is like ours )

Component

Scraper

Version

2.7.0

Configuration

Configuration:

apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization

resources:

configMapGenerator:

images:

Logs

██████╗ ██████╗ ██████╗ ███╗ ███╗██╗████████╗ ██████╗ ██████╗ ██╔══██╗██╔══██╗██╔═══██╗████╗ ████║██║╚══██╔══╝██╔═══██╗██╔══██╗ ██████╔╝██████╔╝██║ ██║██╔████╔██║██║ ██║ ██║ ██║██████╔╝ ██╔═══╝ ██╔══██╗██║ ██║██║╚██╔╝██║██║ ██║ ██║ ██║██╔══██╗ ██║ ██║ ██║╚██████╔╝██║ ╚═╝ ██║██║ ██║ ╚██████╔╝██║ ██║ ╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═╝╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═╝ [09:41:02 INF] Booting up Promitor v2.7.0 running .NET .NET 6.0.6 - Thank you for using Promitor! [09:41:02 INF] Running .NET 6.0.6 on Linux (alpine.3.14-x64 | Linux 5.4.0-1086-azure #91~18.04.1-Ubuntu SMP Thu Jun 23 20:33:05 UTC 2022). [09:41:02 INF] Using configuration folder '/config/' fail: Promitor.Agents.Scraper.Startup[0] Failure to create job name for scraping resources for list of metrics which includes cpu_percent. System.Globalization.CultureNotFoundException: Only the invariant culture is supported in globalization-invariant mode. See https://aka.ms/GlobalizationInvariantMode for more information. (Parameter 'name') en-US is an invalid culture identifier. at System.Globalization.CultureInfo..ctor(String name, Boolean useUserOverride) at System.Globalization.CultureInfo..ctor(String name) at CronExpressionDescriptor.ExpressionParser..ctor(String expression, Options options) at CronExpressionDescriptor.ExpressionDescriptor.GetDescription(DescriptionTypeEnum type) at CronExpressionDescriptor.ExpressionDescriptor.GetDescription(String expression, Options options) at CronExpressionDescriptor.ExpressionDescriptor.GetDescription(String expression) at Microsoft.Extensions.DependencyInjection.SchedulingExtensions.GenerateResourceScrapingJobName(MetricsDeclaration metricsDeclaration, ILogger1 logger) in /src/Promitor.Agents.Scraper/SchedulingExtensions.cs:line 133 [09:41:05 INF] Starting validation of Promitor setup [09:41:05 INF] Scrape configuration found at '/config/metrics-declaration.yaml' ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Name │ Outcome │ Details ┃ ┠───────────────────────────────────┼─────────┼────────────────────────────────┨ ┃ Metrics Declaration Path │ Success │ Everything is well-configured. ┃ ┃ Azure Authentication │ Success │ Everything is well-configured. ┃ ┃ Metrics Declaration │ Success │ Everything is well-configured. ┃ ┃ Resource Discovery │ Success │ Everything is well-configured. ┃ ┃ StatsD Metric Sink │ Success │ Everything is well-configured. ┃ ┃ Prometheus Scraping Endpoint │ Success │ Everything is well-configured. ┃ ┃ Metric Sink │ │ ┃ ┃ Atlassian Statuspage Metric Sink │ Success │ Everything is well-configured. ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ Validation [09:41:05 INF] Promitor configuration is valid, we are good to go. [09:41:05 INF] Here's an overview of what was configured: ╭───────────────────┬───────────────────┬───────────────────┬──────────────────╮ │ Tenant Id │ Subscription Id │ Resource Group │ Azure Cloud │ │ │ │ Name (Default) │ │ ├───────────────────┼───────────────────┼───────────────────┼──────────────────┤ │ 349cc69a-c832-473 │ 5ada2490-9d8e-42a │ rg-pg-dev-test │ Azure Global │ │ d-ad8e-d405d111f3 │ 2-b80d-18269ab48e │ │ Cloud │ │ f4 │ 85 │ │ │ ╰───────────────────┴───────────────────┴───────────────────┴──────────────────╯ Azure Metadata ╭─────────────┬─────────────┬────────┬─────────────┬─────────────┬─────────────╮ │ Metric Name │ Resource │ Labels │ Azure │ Resources │ Resource │ │ │ Type │ │ Monitor │ To Scrape │ Discovery │ │ │ │ │ Metric │ │ Groups To │ │ │ │ │ │ │ Scrape │ ├─────────────┼─────────────┼────────┼─────────────┼─────────────┼─────────────┤ │ azure_postg │ Postgre Sql │ None │ cpu_percent │ pg-dev-test │ None │ │ resql_cpu_p │ │ │ │ -Flexible │ │ │ ercent │ │ │ │ │ │ │ azure_postg │ Postgre Sql │ None │ memory_perc │ pg-dev-test │ None │ │ resql_memor │ │ │ ent │ -Flexible │ │ │ y_percent │ │ │ │ │ │ │ azure_postg │ Postgre Sql │ None │ active_conn │ pg-dev-test │ None │ │ resql_activ │ │ │ ections │ -Flexible │ │ │ e_connectio │ │ │ │ │ │ │ ns_total │ │ │ │ │ │ │ azure_postg │ Postgre Sql │ None │ storage_per │ pg-dev-test │ None │ │ resql_stora │ │ │ cent │ -Flexible │ │ │ ge_percent │ │ │ │ │ │ │ azure_postg │ Postgre Sql │ None │ storage_use │ pg-dev-test │ None │ │ resql_stora │ │ │ d │ -Flexible │ │ │ ge_used_byt │ │ │ │ │ │ │ es │ │ │ │ │ │ │ azure_postg │ Postgre Sql │ None │ network_byt │ pg-dev-test │ None │ │ resql_netwo │ │ │ es_egress │ -Flexible │ │ │ rk_out_byte │ │ │ │ │ │ │ s │ │ │ │ │ │ │ azure_postg │ Postgre Sql │ None │ network_byt │ pg-dev-test │ None │ │ resql_netwo │ │ │ es_ingress │ -Flexible │ │ │ rk_in_bytes │ │ │ │ │ │ │ azure_postg │ Postgre Sql │ None │ storage_fre │ pg-dev-test │ None │ │ resql_stora │ │ │ e │ -Flexible │ │ │ ge_free │ │ │ │ │ │ │ azure_postg │ Postgre Sql │ None │ iops │ pg-dev-test │ None │ │ resql_iops │ │ │ │ -Flexible │ │ │ azure_postg │ Postgre Sql │ None │ backup_stor │ pg-dev-test │ None │ │ resql_backu │ │ │ age_used │ -Flexible │ │ │ p_storage_u │ │ │ │ │ │ │ sed │ │ │ │ │ │ │ azure_stora │ Storage │ None │ UsedCapacit │ sagiteaback │ None │ │ ge_account_ │ Account │ │ y │ up │ │ │ backup_capa │ │ │ │ │ │ │ city │ │ │ │ │ │ │ azure_stora │ Storage │ None │ Availabilit │ sagiteaback │ None │ │ ge_account_ │ Account │ │ y │ up │ │ │ backup_avai │ │ │ │ │ │ │ lability │ │ │ │ │ │ │ azure_stora │ Storage │ None │ UsedCapacit │ sagiteapage │ None │ │ ge_account_ │ Account │ │ y │ stest │ │ │ pages_capac │ │ │ │ │ │ │ ity │ │ │ │ │ │ │ azure_stora │ Storage │ None │ Availabilit │ sagiteapage │ None │ │ ge_account_ │ Account │ │ y │ stest │ │ │ pages_avail │ │ │ │ │ │ │ ability │ │ │ │ │ │ ╰─────────────┴─────────────┴────────┴─────────────┴─────────────┴─────────────╯ Configured Metrics [09:41:06 FTL] Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. System.Exception: Resource Discovery is not configured at GuardNet.Guard.For[TException](Func1 predicate, TException exception) at GuardNet.Guard.For[TException](Func1 predicate, String message) at Promitor.Agents.Scraper.Discovery.ResourceDiscoveryClient..ctor(HttpClient httpClient, IOptionsMonitor1 configuration, ILogger1 logger) in /src/Promitor.Agents.Scraper/ResourceDiscoveryClient.cs:line 29 at lambda_method21(Closure , IServiceProvider , Object[] ) at Microsoft.Extensions.Http.DefaultTypedHttpClientFactory1.CreateClient(HttpClient httpClient) at Microsoft.Extensions.DependencyInjection.HttpClientBuilderExtensions.AddTransientHelper[TClient](IServiceProvider s, IHttpClientBuilder builder) at Microsoft.Extensions.DependencyInjection.HttpClientBuilderExtensions.<>cDisplayClass10_01.<AddTypedClientCore>b__0(IServiceProvider s) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSite(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope) at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.b0(ServiceProviderEngineScope scope) at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType) at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider) at Microsoft.Extensions.DependencyInjection.SchedulingExtensions.<>cDisplayClass2_0.b1(IServiceProvider jobServices) in /src/Promitor.Agents.Scraper/SchedulingExtensions.cs:line 96 at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSite(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, RuntimeResolverContext context) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSite(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSite(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope) at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType) at System.Collections.Concurrent.ConcurrentDictionary2.GetOrAdd(TKey key, Func2 valueFactory) at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType) at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType) at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider) at Microsoft.Extensions.DependencyInjection.SchedulerServiceCollectionExtensions.<>cDisplayClass6_0.b0(IServiceProvider sp) in C:\projects\cronscheduler-aspnetcore\src\CronScheduler.Extensions\DependencyInjection\SchedulerServiceCollectionExtensions.cs:line 119 at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSite(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, RuntimeResolverContext context) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSite(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope) at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType) at System.Collections.Concurrent.ConcurrentDictionary2.GetOrAdd(TKey key, Func2 valueFactory) at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope) at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType) at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider) at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken) at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token) at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token) at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host) at Promitor.Agents.Scraper.Program.Main(String[] args) in /src/Promitor.Agents.Scraper/Program.cs:line 52

Platform

Microsoft Azure

Contact Details

bert.meedendorp@kadaster.nl

tomkerkhove commented 2 years ago

This is a duplicate of #2089 but it's next on the list!

tomkerkhove commented 2 years ago

Should be fixed in https://github.com/tomkerkhove/promitor/pull/2118