Closed ahmetb closed 4 years ago
Issues go stale after 90 days of inactivity.
Mark the issue as fresh by adding the comment /remove-lifecycle stale
.
Stale issues rot after an additional 30 days of inactivity and eventually close.
If this issue is safe to close now please do so by adding the comment /close
.
Send feedback to Knative Productivity Slack channel or file an issue in knative/test-infra.
/lifecycle stale
Stale issues rot after 30 days of inactivity.
Mark the issue as fresh by adding the comment /remove-lifecycle rotten
.
Rotten issues close after an additional 30 days of inactivity.
If this issue is safe to close now please do so by adding the comment /close
.
Send feedback to Knative Productivity Slack channel or file an issue in knative/test-infra.
/lifecycle rotten
More than a year after this issue was opened, I come to have exactly the same problem. Any help? Any place where I can find an explanation of what knative metrics are measuring?
/lifecycle frozen
@ahmetb @rutpal monitoring is deprecated as a feature upstream but there is a plan to update the docs with more info, moving stuff out of the serving/eventing repos to docs. This is a good issue to understand what people need, so definitely something to be addressed imho. For example if you check other systems eg. Kafka it is well documented what to measure etc. @abrennan89 @dprotaso WDYTH?
cc @evankanderson
yeah the monitoring bundle is deprecated but we're waiting for some docs to provide guidance. One worry I have though is the docs will become out of date and unmaintained like the thing it's meant to replace.
"Updating the docs with more info" means that there is any info at all about what the metrics are measuring? We have a Prometheus that collects metrics from Knative in order to control the fulfilment of a certain SLA. Having a description of the impact that each measure has in Knative will be of a great help.
@ahmetb those pages no longer exist due to the old observability plugin being removed @ 0.14, and subsequent removal of monitoring / metrics related docs that were out of date.
Stavros and I plan to work on new documentation / a roadmap for monitoring docs when we have time, so if you could please add any requirements etc to #2931 instead I'd appreciate it.
https://knative.dev/docs/serving/accessing-metrics/#grafana https://knative.dev/docs/serving/accessing-metrics/#prometheus
These two pages say "here's how you access {Grafana,Prometheus}".
As a user who's not familiar with any of these, I would've appreciated which {dashboard,metrics} I am supposed to look at to learn more about my application running on Knative.
For Grafana: Which dashboards are useful to me, what do I look at.
For Prometheus: How do I use this to monitor my apps, which metrics are belonging to my apps.
I am gonna paste below the list of metrics http://localhost:9090 Prometheus homepage shows (which is where the docs send the users). Assume you're a Knative uesr trying to choose among these metrics.
There are about 1300 of them, the word
knative
orrevision
is not mentioned even once. Clearly the monitoring is not helping here by not being verbose enough, let's help users understand which one of them they are gonna choose in the docs.APIServiceOpenAPIAggregationControllerQueue1_adds APIServiceOpenAPIAggregationControllerQueue1_depth APIServiceOpenAPIAggregationControllerQueue1_longest_running_processor_microseconds APIServiceOpenAPIAggregationControllerQueue1_queue_latency APIServiceOpenAPIAggregationControllerQueue1_queue_latency_count APIServiceOpenAPIAggregationControllerQueue1_queue_latency_sum APIServiceOpenAPIAggregationControllerQueue1_retries APIServiceOpenAPIAggregationControllerQueue1_unfinished_work_seconds APIServiceOpenAPIAggregationControllerQueue1_work_duration APIServiceOpenAPIAggregationControllerQueue1_work_duration_count APIServiceOpenAPIAggregationControllerQueue1_work_duration_sum APIServiceRegistrationController_adds APIServiceRegistrationController_depth APIServiceRegistrationController_longest_running_processor_microseconds APIServiceRegistrationController_queue_latency APIServiceRegistrationController_queue_latency_count APIServiceRegistrationController_queue_latency_sum APIServiceRegistrationController_retries APIServiceRegistrationController_unfinished_work_seconds APIServiceRegistrationController_work_duration APIServiceRegistrationController_work_duration_count APIServiceRegistrationController_work_duration_sum AvailableConditionController_adds AvailableConditionController_depth AvailableConditionController_longest_running_processor_microseconds AvailableConditionController_queue_latency AvailableConditionController_queue_latency_count AvailableConditionController_queue_latency_sum AvailableConditionController_retries AvailableConditionController_unfinished_work_seconds AvailableConditionController_work_duration AvailableConditionController_work_duration_count AvailableConditionController_work_duration_sum DiscoveryController_adds DiscoveryController_depth DiscoveryController_longest_running_processor_microseconds DiscoveryController_queue_latency DiscoveryController_queue_latency_count DiscoveryController_queue_latency_sum DiscoveryController_retries DiscoveryController_unfinished_work_seconds DiscoveryController_work_duration DiscoveryController_work_duration_count DiscoveryController_work_duration_sum activator_client_latency_bucket activator_client_latency_count activator_client_latency_sum activator_client_results activator_go_alloc activator_go_bucket_hash_sys activator_go_frees activator_go_gc_cpu_fraction activator_go_gc_sys activator_go_heap_alloc activator_go_heap_idle activator_go_heap_in_use activator_go_heap_objects activator_go_heap_released activator_go_heap_sys activator_go_last_gc activator_go_lookups activator_go_mallocs activator_go_mcache_in_use activator_go_mcache_sys activator_go_mspan_in_use activator_go_mspan_sys activator_go_next_gc activator_go_num_forced_gc activator_go_num_gc activator_go_other_sys activator_go_stack_in_use activator_go_stack_sys activator_go_sys activator_go_total_alloc activator_go_total_gc_pause_ns activator_request_concurrency activator_request_count activator_request_latencies_bucket activator_request_latencies_count activator_request_latencies_sum admission_quota_controller_adds admission_quota_controller_depth admission_quota_controller_longest_running_processor_microseconds admission_quota_controller_queue_latency admission_quota_controller_queue_latency_count admission_quota_controller_queue_latency_sum admission_quota_controller_unfinished_work_seconds admission_quota_controller_work_duration admission_quota_controller_work_duration_count admission_quota_controller_work_duration_sum aggregator_unavailable_apiservice aggregator_unavailable_apiservice_count apiserver_admission_controller_admission_duration_seconds_bucket apiserver_admission_controller_admission_duration_seconds_count apiserver_admission_controller_admission_duration_seconds_sum apiserver_admission_step_admission_duration_seconds_bucket apiserver_admission_step_admission_duration_seconds_count apiserver_admission_step_admission_duration_seconds_sum apiserver_admission_step_admission_duration_seconds_summary apiserver_admission_step_admission_duration_seconds_summary_count apiserver_admission_step_admission_duration_seconds_summary_sum apiserver_admission_webhook_admission_duration_seconds_bucket apiserver_admission_webhook_admission_duration_seconds_count apiserver_admission_webhook_admission_duration_seconds_sum apiserver_audit_event_total apiserver_audit_level_total apiserver_audit_requests_rejected_total apiserver_client_certificate_expiration_seconds_bucket apiserver_client_certificate_expiration_seconds_count apiserver_client_certificate_expiration_seconds_sum apiserver_current_inflight_requests apiserver_init_events_total apiserver_longrunning_gauge apiserver_registered_watchers apiserver_request_count apiserver_request_duration_seconds_bucket apiserver_request_duration_seconds_count apiserver_request_duration_seconds_sum apiserver_request_latencies_bucket apiserver_request_latencies_count apiserver_request_latencies_sum apiserver_request_latencies_summary apiserver_request_latencies_summary_count apiserver_request_latencies_summary_sum apiserver_request_total apiserver_response_sizes_bucket apiserver_response_sizes_count apiserver_response_sizes_sum apiserver_storage_data_key_generation_duration_seconds_bucket apiserver_storage_data_key_generation_duration_seconds_count apiserver_storage_data_key_generation_duration_seconds_sum apiserver_storage_data_key_generation_failures_total apiserver_storage_data_key_generation_latencies_microseconds_bucket apiserver_storage_data_key_generation_latencies_microseconds_count apiserver_storage_data_key_generation_latencies_microseconds_sum apiserver_storage_envelope_transformation_cache_misses_total authenticated_user_requests autoregister_adds autoregister_depth autoregister_longest_running_processor_microseconds autoregister_queue_latency autoregister_queue_latency_count autoregister_queue_latency_sum autoregister_retries autoregister_unfinished_work_seconds autoregister_work_duration autoregister_work_duration_count autoregister_work_duration_sum autoscaler_actual_pods autoscaler_client_latency_bucket autoscaler_client_latency_count autoscaler_client_latency_sum autoscaler_client_results autoscaler_desired_pods autoscaler_excess_burst_capacity autoscaler_go_alloc autoscaler_go_bucket_hash_sys autoscaler_go_frees autoscaler_go_gc_cpu_fraction autoscaler_go_gc_sys autoscaler_go_heap_alloc autoscaler_go_heap_idle autoscaler_go_heap_in_use autoscaler_go_heap_objects autoscaler_go_heap_released autoscaler_go_heap_sys autoscaler_go_last_gc autoscaler_go_lookups autoscaler_go_mallocs autoscaler_go_mcache_in_use autoscaler_go_mcache_sys autoscaler_go_mspan_in_use autoscaler_go_mspan_sys autoscaler_go_next_gc autoscaler_go_num_forced_gc autoscaler_go_num_gc autoscaler_go_other_sys autoscaler_go_stack_in_use autoscaler_go_stack_sys autoscaler_go_sys autoscaler_go_total_alloc autoscaler_go_total_gc_pause_ns autoscaler_panic_mode autoscaler_panic_request_concurrency autoscaler_reconcile_count autoscaler_reconcile_latency_bucket autoscaler_reconcile_latency_count autoscaler_reconcile_latency_sum autoscaler_requested_pods autoscaler_stable_request_concurrency autoscaler_target_concurrency_per_pod autoscaler_work_queue_depth autoscaler_workqueue_adds_total autoscaler_workqueue_depth autoscaler_workqueue_longest_running_processor_seconds_bucket autoscaler_workqueue_longest_running_processor_seconds_count autoscaler_workqueue_longest_running_processor_seconds_sum autoscaler_workqueue_queue_latency_seconds_bucket autoscaler_workqueue_queue_latency_seconds_count autoscaler_workqueue_queue_latency_seconds_sum autoscaler_workqueue_retries_total autoscaler_workqueue_unfinished_work_seconds_bucket autoscaler_workqueue_unfinished_work_seconds_count autoscaler_workqueue_unfinished_work_seconds_sum autoscaler_workqueue_work_duration_seconds_bucket autoscaler_workqueue_work_duration_seconds_count autoscaler_workqueue_work_duration_seconds_sum cadvisor_version_info cloudprovider_gce_api_request_duration_seconds_bucket cloudprovider_gce_api_request_duration_seconds_count cloudprovider_gce_api_request_duration_seconds_sum container_cpu_cfs_periods_total container_cpu_cfs_throttled_periods_total container_cpu_cfs_throttled_seconds_total container_cpu_load_average_10s container_cpu_system_seconds_total container_cpu_usage_seconds_total container_cpu_user_seconds_total container_file_descriptors container_fs_inodes_free container_fs_inodes_total container_fs_io_current container_fs_io_time_seconds_total container_fs_io_time_weighted_seconds_total container_fs_limit_bytes container_fs_read_seconds_total container_fs_reads_bytes_total container_fs_reads_merged_total container_fs_reads_total container_fs_sector_reads_total container_fs_sector_writes_total container_fs_usage_bytes container_fs_write_seconds_total container_fs_writes_bytes_total container_fs_writes_merged_total container_fs_writes_total container_last_seen container_memory_cache container_memory_failcnt container_memory_failures_total container_memory_mapped_file container_memory_max_usage_bytes container_memory_rss container_memory_swap container_memory_usage_bytes container_memory_working_set_bytes container_network_receive_bytes_total container_network_receive_errors_total container_network_receive_packets_dropped_total container_network_receive_packets_total container_network_transmit_bytes_total container_network_transmit_errors_total container_network_transmit_packets_dropped_total container_network_transmit_packets_total container_processes container_scrape_error container_spec_cpu_period container_spec_cpu_quota container_spec_cpu_shares container_spec_memory_limit_bytes container_spec_memory_reservation_limit_bytes container_spec_memory_swap_limit_bytes container_start_time_seconds container_tasks_state controller_client_latency_bucket controller_client_latency_count controller_client_latency_sum controller_client_results controller_go_alloc controller_go_bucket_hash_sys controller_go_frees controller_go_gc_cpu_fraction controller_go_gc_sys controller_go_heap_alloc controller_go_heap_idle controller_go_heap_in_use controller_go_heap_objects controller_go_heap_released controller_go_heap_sys controller_go_last_gc controller_go_lookups controller_go_mallocs controller_go_mcache_in_use controller_go_mcache_sys controller_go_mspan_in_use controller_go_mspan_sys controller_go_next_gc controller_go_num_forced_gc controller_go_num_gc controller_go_other_sys controller_go_stack_in_use controller_go_stack_sys controller_go_sys controller_go_total_alloc controller_go_total_gc_pause_ns controller_reconcile_count controller_reconcile_latency_bucket controller_reconcile_latency_count controller_reconcile_latency_sum controller_service_ready_count controller_service_ready_latency controller_work_queue_depth controller_workqueue_adds_total controller_workqueue_depth controller_workqueue_longest_running_processor_seconds_bucket controller_workqueue_longest_running_processor_seconds_count controller_workqueue_longest_running_processor_seconds_sum controller_workqueue_queue_latency_seconds_bucket controller_workqueue_queue_latency_seconds_count controller_workqueue_queue_latency_seconds_sum controller_workqueue_retries_total controller_workqueue_unfinished_work_seconds_bucket controller_workqueue_unfinished_work_seconds_count controller_workqueue_unfinished_work_seconds_sum controller_workqueue_work_duration_seconds_bucket controller_workqueue_work_duration_seconds_count controller_workqueue_work_duration_seconds_sum crdEstablishing_adds crdEstablishing_depth crdEstablishing_longest_running_processor_microseconds crdEstablishing_queue_latency crdEstablishing_queue_latency_count crdEstablishing_queue_latency_sum crdEstablishing_retries crdEstablishing_unfinished_work_seconds crdEstablishing_work_duration crdEstablishing_work_duration_count crdEstablishing_work_duration_sum crd_autoregistration_controller_adds crd_autoregistration_controller_depth crd_autoregistration_controller_longest_running_processor_microseconds crd_autoregistration_controller_queue_latency crd_autoregistration_controller_queue_latency_count crd_autoregistration_controller_queue_latency_sum crd_autoregistration_controller_retries crd_autoregistration_controller_unfinished_work_seconds crd_autoregistration_controller_work_duration crd_autoregistration_controller_work_duration_count crd_autoregistration_controller_work_duration_sum crd_finalizer_adds crd_finalizer_depth crd_finalizer_longest_running_processor_microseconds crd_finalizer_queue_latency crd_finalizer_queue_latency_count crd_finalizer_queue_latency_sum crd_finalizer_retries crd_finalizer_unfinished_work_seconds crd_finalizer_work_duration crd_finalizer_work_duration_count crd_finalizer_work_duration_sum crd_naming_condition_controller_adds crd_naming_condition_controller_depth crd_naming_condition_controller_longest_running_processor_microseconds crd_naming_condition_controller_queue_latency crd_naming_condition_controller_queue_latency_count crd_naming_condition_controller_queue_latency_sum crd_naming_condition_controller_retries crd_naming_condition_controller_unfinished_work_seconds crd_naming_condition_controller_work_duration crd_naming_condition_controller_work_duration_count crd_naming_condition_controller_work_duration_sum crd_openapi_controller_adds crd_openapi_controller_depth crd_openapi_controller_longest_running_processor_microseconds crd_openapi_controller_queue_latency crd_openapi_controller_queue_latency_count crd_openapi_controller_queue_latency_sum crd_openapi_controller_retries crd_openapi_controller_unfinished_work_seconds crd_openapi_controller_work_duration crd_openapi_controller_work_duration_count crd_openapi_controller_work_duration_sum endpoint_no_pod etcd_helper_cache_entry_count etcd_helper_cache_entry_total etcd_helper_cache_hit_count etcd_helper_cache_hit_total etcd_helper_cache_miss_count etcd_helper_cache_miss_total etcd_object_counts etcd_request_cache_add_duration_seconds_bucket etcd_request_cache_add_duration_seconds_count etcd_request_cache_add_duration_seconds_sum etcd_request_cache_add_latencies_summary etcd_request_cache_add_latencies_summary_count etcd_request_cache_add_latencies_summary_sum etcd_request_cache_get_duration_seconds_bucket etcd_request_cache_get_duration_seconds_count etcd_request_cache_get_duration_seconds_sum etcd_request_cache_get_latencies_summary etcd_request_cache_get_latencies_summary_count etcd_request_cache_get_latencies_summary_sum etcd_request_duration_seconds_bucket etcd_request_duration_seconds_count etcd_request_duration_seconds_sum etcd_request_latencies_summary etcd_request_latencies_summary_count etcd_request_latencies_summary_sum get_token_count get_token_fail_count go_gc_duration_seconds go_gc_duration_seconds_count go_gc_duration_seconds_sum go_goroutines go_info go_memstats_alloc_bytes go_memstats_alloc_bytes_total go_memstats_buck_hash_sys_bytes go_memstats_frees_total go_memstats_gc_cpu_fraction go_memstats_gc_sys_bytes go_memstats_heap_alloc_bytes go_memstats_heap_idle_bytes go_memstats_heap_inuse_bytes go_memstats_heap_objects go_memstats_heap_released_bytes go_memstats_heap_sys_bytes go_memstats_last_gc_time_seconds go_memstats_lookups_total go_memstats_mallocs_total go_memstats_mcache_inuse_bytes go_memstats_mcache_sys_bytes go_memstats_mspan_inuse_bytes go_memstats_mspan_sys_bytes go_memstats_next_gc_bytes go_memstats_other_sys_bytes go_memstats_stack_inuse_bytes go_memstats_stack_sys_bytes go_memstats_sys_bytes go_threads grpc_client_handled_total grpc_client_msg_received_total grpc_client_msg_sent_total grpc_client_started_total http_request_duration_microseconds http_request_duration_microseconds_count http_request_duration_microseconds_sum http_request_size_bytes http_request_size_bytes_count http_request_size_bytes_sum http_requests_total http_response_size_bytes http_response_size_bytes_count http_response_size_bytes_sum istio_build kubelet_certificate_manager_server_expiration_seconds kubelet_cgroup_manager_duration_seconds_bucket kubelet_cgroup_manager_duration_seconds_count kubelet_cgroup_manager_duration_seconds_sum kubelet_cgroup_manager_latency_microseconds kubelet_cgroup_manager_latency_microseconds_count kubelet_cgroup_manager_latency_microseconds_sum kubelet_container_log_filesystem_used_bytes kubelet_containers_per_pod_count_bucket kubelet_containers_per_pod_count_count kubelet_containers_per_pod_count_sum kubelet_docker_operations kubelet_docker_operations_duration_seconds_bucket kubelet_docker_operations_duration_seconds_count kubelet_docker_operations_duration_seconds_sum kubelet_docker_operations_errors kubelet_docker_operations_errors_total kubelet_docker_operations_latency_microseconds kubelet_docker_operations_latency_microseconds_count kubelet_docker_operations_latency_microseconds_sum kubelet_docker_operations_total kubelet_http_inflight_requests kubelet_http_requests_duration_seconds_bucket kubelet_http_requests_duration_seconds_count kubelet_http_requests_duration_seconds_sum kubelet_http_requests_total kubelet_network_plugin_operations_duration_seconds_bucket kubelet_network_plugin_operations_duration_seconds_count kubelet_network_plugin_operations_duration_seconds_sum kubelet_network_plugin_operations_latency_microseconds kubelet_network_plugin_operations_latency_microseconds_count kubelet_network_plugin_operations_latency_microseconds_sum kubelet_node_name kubelet_pleg_relist_duration_seconds_bucket kubelet_pleg_relist_duration_seconds_count kubelet_pleg_relist_duration_seconds_sum kubelet_pleg_relist_interval_microseconds kubelet_pleg_relist_interval_microseconds_count kubelet_pleg_relist_interval_microseconds_sum kubelet_pleg_relist_interval_seconds_bucket kubelet_pleg_relist_interval_seconds_count kubelet_pleg_relist_interval_seconds_sum kubelet_pleg_relist_latency_microseconds kubelet_pleg_relist_latency_microseconds_count kubelet_pleg_relist_latency_microseconds_sum kubelet_pod_start_duration_seconds_bucket kubelet_pod_start_duration_seconds_count kubelet_pod_start_duration_seconds_sum kubelet_pod_start_latency_microseconds kubelet_pod_start_latency_microseconds_count kubelet_pod_start_latency_microseconds_sum kubelet_pod_worker_duration_seconds_bucket kubelet_pod_worker_duration_seconds_count kubelet_pod_worker_duration_seconds_sum kubelet_pod_worker_latency_microseconds kubelet_pod_worker_latency_microseconds_count kubelet_pod_worker_latency_microseconds_sum kubelet_pod_worker_start_duration_seconds_bucket kubelet_pod_worker_start_duration_seconds_count kubelet_pod_worker_start_duration_seconds_sum kubelet_pod_worker_start_latency_microseconds kubelet_pod_worker_start_latency_microseconds_count kubelet_pod_worker_start_latency_microseconds_sum kubelet_running_container_count kubelet_running_pod_count kubelet_runtime_operations kubelet_runtime_operations_duration_seconds_bucket kubelet_runtime_operations_duration_seconds_count kubelet_runtime_operations_duration_seconds_sum kubelet_runtime_operations_errors kubelet_runtime_operations_errors_total kubelet_runtime_operations_latency_microseconds kubelet_runtime_operations_latency_microseconds_count kubelet_runtime_operations_latency_microseconds_sum kubelet_runtime_operations_total kubernetes_build_info machine_cpu_cores machine_memory_bytes net_conntrack_dialer_conn_attempted_total net_conntrack_dialer_conn_closed_total net_conntrack_dialer_conn_established_total net_conntrack_dialer_conn_failed_total net_conntrack_listener_conn_accepted_total net_conntrack_listener_conn_closed_total node_arp_entries node_boot_time node_context_switches node_cpu node_entropy_available_bits node_exporter_build_info node_filefd_allocated node_filefd_maximum node_filesystem_avail node_filesystem_device_error node_filesystem_files node_filesystem_files_free node_filesystem_free node_filesystem_readonly node_filesystem_size node_forks node_intr node_ipvs_connections_total node_ipvs_incoming_bytes_total node_ipvs_incoming_packets_total node_ipvs_outgoing_bytes_total node_ipvs_outgoing_packets_total node_load1 node_load15 node_load5 node_memory_Active node_memory_Active_anon node_memory_Active_file node_memory_AnonHugePages node_memory_AnonPages node_memory_Bounce node_memory_Buffers node_memory_Cached node_memory_CommitLimit node_memory_Committed_AS node_memory_DirectMap1G node_memory_DirectMap2M node_memory_DirectMap4k node_memory_Dirty node_memory_HugePages_Free node_memory_HugePages_Rsvd node_memory_HugePages_Surp node_memory_HugePages_Total node_memory_Hugepagesize node_memory_Hugetlb node_memory_Inactive node_memory_Inactive_anon node_memory_Inactive_file node_memory_KernelStack node_memory_Mapped node_memory_MemAvailable node_memory_MemFree node_memory_MemTotal node_memory_Mlocked node_memory_NFS_Unstable node_memory_PageTables node_memory_Percpu node_memory_SReclaimable node_memory_SUnreclaim node_memory_Shmem node_memory_ShmemHugePages node_memory_ShmemPmdMapped node_memory_Slab node_memory_SwapCached node_memory_SwapFree node_memory_SwapTotal node_memory_Unevictable node_memory_VmallocChunk node_memory_VmallocTotal node_memory_VmallocUsed node_memory_Writeback node_memory_WritebackTmp node_netstat_Icmp6_InCsumErrors node_netstat_Icmp6_InDestUnreachs node_netstat_Icmp6_InEchoReplies node_netstat_Icmp6_InEchos node_netstat_Icmp6_InErrors node_netstat_Icmp6_InGroupMembQueries node_netstat_Icmp6_InGroupMembReductions node_netstat_Icmp6_InGroupMembResponses node_netstat_Icmp6_InMLDv2Reports node_netstat_Icmp6_InMsgs node_netstat_Icmp6_InNeighborAdvertisements node_netstat_Icmp6_InNeighborSolicits node_netstat_Icmp6_InParmProblems node_netstat_Icmp6_InPktTooBigs node_netstat_Icmp6_InRedirects node_netstat_Icmp6_InRouterAdvertisements node_netstat_Icmp6_InRouterSolicits node_netstat_Icmp6_InTimeExcds node_netstat_Icmp6_OutDestUnreachs node_netstat_Icmp6_OutEchoReplies node_netstat_Icmp6_OutEchos node_netstat_Icmp6_OutErrors node_netstat_Icmp6_OutGroupMembQueries node_netstat_Icmp6_OutGroupMembReductions node_netstat_Icmp6_OutGroupMembResponses node_netstat_Icmp6_OutMLDv2Reports node_netstat_Icmp6_OutMsgs node_netstat_Icmp6_OutNeighborAdvertisements node_netstat_Icmp6_OutNeighborSolicits node_netstat_Icmp6_OutParmProblems node_netstat_Icmp6_OutPktTooBigs node_netstat_Icmp6_OutRedirects node_netstat_Icmp6_OutRouterAdvertisements node_netstat_Icmp6_OutRouterSolicits node_netstat_Icmp6_OutTimeExcds node_netstat_Icmp6_OutType133 node_netstat_Icmp6_OutType135 node_netstat_Icmp6_OutType143 node_netstat_IcmpMsg_InType3 node_netstat_IcmpMsg_InType8 node_netstat_IcmpMsg_OutType0 node_netstat_IcmpMsg_OutType3 node_netstat_Icmp_InAddrMaskReps node_netstat_Icmp_InAddrMasks node_netstat_Icmp_InCsumErrors node_netstat_Icmp_InDestUnreachs node_netstat_Icmp_InEchoReps node_netstat_Icmp_InEchos node_netstat_Icmp_InErrors node_netstat_Icmp_InMsgs node_netstat_Icmp_InParmProbs node_netstat_Icmp_InRedirects node_netstat_Icmp_InSrcQuenchs node_netstat_Icmp_InTimeExcds node_netstat_Icmp_InTimestampReps node_netstat_Icmp_InTimestamps node_netstat_Icmp_OutAddrMaskReps node_netstat_Icmp_OutAddrMasks node_netstat_Icmp_OutDestUnreachs node_netstat_Icmp_OutEchoReps node_netstat_Icmp_OutEchos node_netstat_Icmp_OutErrors node_netstat_Icmp_OutMsgs node_netstat_Icmp_OutParmProbs node_netstat_Icmp_OutRedirects node_netstat_Icmp_OutSrcQuenchs node_netstat_Icmp_OutTimeExcds node_netstat_Icmp_OutTimestampReps node_netstat_Icmp_OutTimestamps node_netstat_Ip6_FragCreates node_netstat_Ip6_FragFails node_netstat_Ip6_FragOKs node_netstat_Ip6_InAddrErrors node_netstat_Ip6_InBcastOctets node_netstat_Ip6_InCEPkts node_netstat_Ip6_InDelivers node_netstat_Ip6_InDiscards node_netstat_Ip6_InECT0Pkts node_netstat_Ip6_InECT1Pkts node_netstat_Ip6_InHdrErrors node_netstat_Ip6_InMcastOctets node_netstat_Ip6_InMcastPkts node_netstat_Ip6_InNoECTPkts node_netstat_Ip6_InNoRoutes node_netstat_Ip6_InOctets node_netstat_Ip6_InReceives node_netstat_Ip6_InTooBigErrors node_netstat_Ip6_InTruncatedPkts node_netstat_Ip6_InUnknownProtos node_netstat_Ip6_OutBcastOctets node_netstat_Ip6_OutDiscards node_netstat_Ip6_OutForwDatagrams node_netstat_Ip6_OutMcastOctets node_netstat_Ip6_OutMcastPkts node_netstat_Ip6_OutNoRoutes node_netstat_Ip6_OutOctets node_netstat_Ip6_OutRequests node_netstat_Ip6_ReasmFails node_netstat_Ip6_ReasmOKs node_netstat_Ip6_ReasmReqds node_netstat_Ip6_ReasmTimeout node_netstat_IpExt_InBcastOctets node_netstat_IpExt_InBcastPkts node_netstat_IpExt_InCEPkts node_netstat_IpExt_InCsumErrors node_netstat_IpExt_InECT0Pkts node_netstat_IpExt_InECT1Pkts node_netstat_IpExt_InMcastOctets node_netstat_IpExt_InMcastPkts node_netstat_IpExt_InNoECTPkts node_netstat_IpExt_InNoRoutes node_netstat_IpExt_InOctets node_netstat_IpExt_InTruncatedPkts node_netstat_IpExt_OutBcastOctets node_netstat_IpExt_OutBcastPkts node_netstat_IpExt_OutMcastOctets node_netstat_IpExt_OutMcastPkts node_netstat_IpExt_OutOctets node_netstat_IpExt_ReasmOverlaps node_netstat_Ip_DefaultTTL node_netstat_Ip_ForwDatagrams node_netstat_Ip_Forwarding node_netstat_Ip_FragCreates node_netstat_Ip_FragFails node_netstat_Ip_FragOKs node_netstat_Ip_InAddrErrors node_netstat_Ip_InDelivers node_netstat_Ip_InDiscards node_netstat_Ip_InHdrErrors node_netstat_Ip_InReceives node_netstat_Ip_InUnknownProtos node_netstat_Ip_OutDiscards node_netstat_Ip_OutNoRoutes node_netstat_Ip_OutRequests node_netstat_Ip_ReasmFails node_netstat_Ip_ReasmOKs node_netstat_Ip_ReasmReqds node_netstat_Ip_ReasmTimeout node_netstat_TcpExt_ArpFilter node_netstat_TcpExt_BusyPollRxPackets node_netstat_TcpExt_DelayedACKLocked node_netstat_TcpExt_DelayedACKLost node_netstat_TcpExt_DelayedACKs node_netstat_TcpExt_EmbryonicRsts node_netstat_TcpExt_IPReversePathFilter node_netstat_TcpExt_ListenDrops node_netstat_TcpExt_ListenOverflows node_netstat_TcpExt_LockDroppedIcmps node_netstat_TcpExt_OfoPruned node_netstat_TcpExt_OutOfWindowIcmps node_netstat_TcpExt_PAWSActive node_netstat_TcpExt_PAWSEstab node_netstat_TcpExt_PFMemallocDrop node_netstat_TcpExt_PruneCalled node_netstat_TcpExt_RcvPruned node_netstat_TcpExt_SyncookiesFailed node_netstat_TcpExt_SyncookiesRecv node_netstat_TcpExt_SyncookiesSent node_netstat_TcpExt_TCPACKSkippedChallenge node_netstat_TcpExt_TCPACKSkippedFinWait2 node_netstat_TcpExt_TCPACKSkippedPAWS node_netstat_TcpExt_TCPACKSkippedSeq node_netstat_TcpExt_TCPACKSkippedSynRecv node_netstat_TcpExt_TCPACKSkippedTimeWait node_netstat_TcpExt_TCPAbortFailed node_netstat_TcpExt_TCPAbortOnClose node_netstat_TcpExt_TCPAbortOnData node_netstat_TcpExt_TCPAbortOnLinger node_netstat_TcpExt_TCPAbortOnMemory node_netstat_TcpExt_TCPAbortOnTimeout node_netstat_TcpExt_TCPAckCompressed node_netstat_TcpExt_TCPAutoCorking node_netstat_TcpExt_TCPBacklogDrop node_netstat_TcpExt_TCPChallengeACK node_netstat_TcpExt_TCPDSACKIgnoredNoUndo node_netstat_TcpExt_TCPDSACKIgnoredOld node_netstat_TcpExt_TCPDSACKOfoRecv node_netstat_TcpExt_TCPDSACKOfoSent node_netstat_TcpExt_TCPDSACKOldSent node_netstat_TcpExt_TCPDSACKRecv node_netstat_TcpExt_TCPDSACKUndo node_netstat_TcpExt_TCPDeferAcceptDrop node_netstat_TcpExt_TCPDelivered node_netstat_TcpExt_TCPDeliveredCE node_netstat_TcpExt_TCPFastOpenActive node_netstat_TcpExt_TCPFastOpenActiveFail node_netstat_TcpExt_TCPFastOpenBlackhole node_netstat_TcpExt_TCPFastOpenCookieReqd node_netstat_TcpExt_TCPFastOpenListenOverflow node_netstat_TcpExt_TCPFastOpenPassive node_netstat_TcpExt_TCPFastOpenPassiveFail node_netstat_TcpExt_TCPFastRetrans node_netstat_TcpExt_TCPFromZeroWindowAdv node_netstat_TcpExt_TCPFullUndo node_netstat_TcpExt_TCPHPAcks node_netstat_TcpExt_TCPHPHits node_netstat_TcpExt_TCPHystartDelayCwnd node_netstat_TcpExt_TCPHystartDelayDetect node_netstat_TcpExt_TCPHystartTrainCwnd node_netstat_TcpExt_TCPHystartTrainDetect node_netstat_TcpExt_TCPKeepAlive node_netstat_TcpExt_TCPLossFailures node_netstat_TcpExt_TCPLossProbeRecovery node_netstat_TcpExt_TCPLossProbes node_netstat_TcpExt_TCPLossUndo node_netstat_TcpExt_TCPLostRetransmit node_netstat_TcpExt_TCPMD5Failure node_netstat_TcpExt_TCPMD5NotFound node_netstat_TcpExt_TCPMD5Unexpected node_netstat_TcpExt_TCPMTUPFail node_netstat_TcpExt_TCPMTUPSuccess node_netstat_TcpExt_TCPMemoryPressures node_netstat_TcpExt_TCPMemoryPressuresChrono node_netstat_TcpExt_TCPMinTTLDrop node_netstat_TcpExt_TCPOFODrop node_netstat_TcpExt_TCPOFOMerge node_netstat_TcpExt_TCPOFOQueue node_netstat_TcpExt_TCPOrigDataSent node_netstat_TcpExt_TCPPartialUndo node_netstat_TcpExt_TCPPureAcks node_netstat_TcpExt_TCPRcvCoalesce node_netstat_TcpExt_TCPRcvCollapsed node_netstat_TcpExt_TCPRcvQDrop node_netstat_TcpExt_TCPRenoFailures node_netstat_TcpExt_TCPRenoRecovery node_netstat_TcpExt_TCPRenoRecoveryFail node_netstat_TcpExt_TCPRenoReorder node_netstat_TcpExt_TCPReqQFullDoCookies node_netstat_TcpExt_TCPReqQFullDrop node_netstat_TcpExt_TCPRetransFail node_netstat_TcpExt_TCPSACKDiscard node_netstat_TcpExt_TCPSACKReneging node_netstat_TcpExt_TCPSACKReorder node_netstat_TcpExt_TCPSYNChallenge node_netstat_TcpExt_TCPSackFailures node_netstat_TcpExt_TCPSackMerged node_netstat_TcpExt_TCPSackRecovery node_netstat_TcpExt_TCPSackRecoveryFail node_netstat_TcpExt_TCPSackShiftFallback node_netstat_TcpExt_TCPSackShifted node_netstat_TcpExt_TCPSlowStartRetrans node_netstat_TcpExt_TCPSpuriousRTOs node_netstat_TcpExt_TCPSpuriousRtxHostQueues node_netstat_TcpExt_TCPSynRetrans node_netstat_TcpExt_TCPTSReorder node_netstat_TcpExt_TCPTimeWaitOverflow node_netstat_TcpExt_TCPTimeouts node_netstat_TcpExt_TCPToZeroWindowAdv node_netstat_TcpExt_TCPWantZeroWindowAdv node_netstat_TcpExt_TCPWinProbe node_netstat_TcpExt_TCPWqueueTooBig node_netstat_TcpExt_TCPZeroWindowDrop node_netstat_TcpExt_TW node_netstat_TcpExt_TWKilled node_netstat_TcpExt_TWRecycled node_netstat_Tcp_ActiveOpens node_netstat_Tcp_AttemptFails node_netstat_Tcp_CurrEstab node_netstat_Tcp_EstabResets node_netstat_Tcp_InCsumErrors node_netstat_Tcp_InErrs node_netstat_Tcp_InSegs node_netstat_Tcp_MaxConn node_netstat_Tcp_OutRsts node_netstat_Tcp_OutSegs node_netstat_Tcp_PassiveOpens node_netstat_Tcp_RetransSegs node_netstat_Tcp_RtoAlgorithm node_netstat_Tcp_RtoMax node_netstat_Tcp_RtoMin node_netstat_Udp6_IgnoredMulti node_netstat_Udp6_InCsumErrors node_netstat_Udp6_InDatagrams node_netstat_Udp6_InErrors node_netstat_Udp6_NoPorts node_netstat_Udp6_OutDatagrams node_netstat_Udp6_RcvbufErrors node_netstat_Udp6_SndbufErrors node_netstat_UdpLite6_InCsumErrors node_netstat_UdpLite6_InDatagrams node_netstat_UdpLite6_InErrors node_netstat_UdpLite6_NoPorts node_netstat_UdpLite6_OutDatagrams node_netstat_UdpLite6_RcvbufErrors node_netstat_UdpLite6_SndbufErrors node_netstat_UdpLite_IgnoredMulti node_netstat_UdpLite_InCsumErrors node_netstat_UdpLite_InDatagrams node_netstat_UdpLite_InErrors node_netstat_UdpLite_NoPorts node_netstat_UdpLite_OutDatagrams node_netstat_UdpLite_RcvbufErrors node_netstat_UdpLite_SndbufErrors node_netstat_Udp_IgnoredMulti node_netstat_Udp_InCsumErrors node_netstat_Udp_InDatagrams node_netstat_Udp_InErrors node_netstat_Udp_NoPorts node_netstat_Udp_OutDatagrams node_netstat_Udp_RcvbufErrors node_netstat_Udp_SndbufErrors node_network_receive_bytes node_network_receive_compressed node_network_receive_drop node_network_receive_errs node_network_receive_fifo node_network_receive_frame node_network_receive_multicast node_network_receive_packets node_network_transmit_bytes node_network_transmit_compressed node_network_transmit_drop node_network_transmit_errs node_network_transmit_fifo node_network_transmit_frame node_network_transmit_multicast node_network_transmit_packets node_nf_conntrack_entries node_nf_conntrack_entries_limit node_procs_blocked node_procs_running node_scrape_collector_duration_seconds node_scrape_collector_success node_sockstat_FRAG_inuse node_sockstat_FRAG_memory node_sockstat_RAW_inuse node_sockstat_TCP_alloc node_sockstat_TCP_inuse node_sockstat_TCP_mem node_sockstat_TCP_mem_bytes node_sockstat_TCP_orphan node_sockstat_TCP_tw node_sockstat_UDPLITE_inuse node_sockstat_UDP_inuse node_sockstat_UDP_mem node_sockstat_UDP_mem_bytes node_sockstat_sockets_used node_time node_timex_estimated_error_seconds node_timex_frequency_adjustment node_timex_loop_time_constant node_timex_maxerror_seconds node_timex_offset_seconds node_timex_pps_calibration_count node_timex_pps_error_count node_timex_pps_frequency node_timex_pps_jitter_count node_timex_pps_jitter_seconds node_timex_pps_shift_seconds node_timex_pps_stability node_timex_pps_stability_exceeded_count node_timex_status node_timex_sync_status node_timex_tai_offset node_timex_tick_seconds node_uname_info node_vmstat_allocstall_dma node_vmstat_allocstall_dma32 node_vmstat_allocstall_movable node_vmstat_allocstall_normal node_vmstat_balloon_deflate node_vmstat_balloon_inflate node_vmstat_balloon_migrate node_vmstat_compact_daemon_free_scanned node_vmstat_compact_daemon_migrate_scanned node_vmstat_compact_daemon_wake node_vmstat_compact_fail node_vmstat_compact_free_scanned node_vmstat_compact_isolated node_vmstat_compact_migrate_scanned node_vmstat_compact_stall node_vmstat_compact_success node_vmstat_drop_pagecache node_vmstat_drop_slab node_vmstat_htlb_buddy_alloc_fail node_vmstat_htlb_buddy_alloc_success node_vmstat_kswapd_high_wmark_hit_quickly node_vmstat_kswapd_inodesteal node_vmstat_kswapd_low_wmark_hit_quickly node_vmstat_nr_active_anon node_vmstat_nr_active_file node_vmstat_nr_anon_pages node_vmstat_nr_anon_transparent_hugepages node_vmstat_nr_bounce node_vmstat_nr_dirtied node_vmstat_nr_dirty node_vmstat_nr_dirty_background_threshold node_vmstat_nr_dirty_threshold node_vmstat_nr_file_pages node_vmstat_nr_free_cma node_vmstat_nr_free_pages node_vmstat_nr_inactive_anon node_vmstat_nr_inactive_file node_vmstat_nr_isolated_anon node_vmstat_nr_isolated_file node_vmstat_nr_kernel_stack node_vmstat_nr_mapped node_vmstat_nr_mlock node_vmstat_nr_page_table_pages node_vmstat_nr_shmem node_vmstat_nr_shmem_hugepages node_vmstat_nr_shmem_pmdmapped node_vmstat_nr_slab_reclaimable node_vmstat_nr_slab_unreclaimable node_vmstat_nr_unevictable node_vmstat_nr_unstable node_vmstat_nr_vmscan_immediate_reclaim node_vmstat_nr_vmscan_write node_vmstat_nr_writeback node_vmstat_nr_writeback_temp node_vmstat_nr_written node_vmstat_nr_zone_active_anon node_vmstat_nr_zone_active_file node_vmstat_nr_zone_inactive_anon node_vmstat_nr_zone_inactive_file node_vmstat_nr_zone_unevictable node_vmstat_nr_zone_write_pending node_vmstat_nr_zspages node_vmstat_numa_foreign node_vmstat_numa_hit node_vmstat_numa_interleave node_vmstat_numa_local node_vmstat_numa_miss node_vmstat_numa_other node_vmstat_oom_kill node_vmstat_pageoutrun node_vmstat_pgactivate node_vmstat_pgalloc_dma node_vmstat_pgalloc_dma32 node_vmstat_pgalloc_movable node_vmstat_pgalloc_normal node_vmstat_pgdeactivate node_vmstat_pgfault node_vmstat_pgfree node_vmstat_pginodesteal node_vmstat_pglazyfree node_vmstat_pglazyfreed node_vmstat_pgmajfault node_vmstat_pgmajfault_a node_vmstat_pgmajfault_f node_vmstat_pgmajfault_s node_vmstat_pgmigrate_fail node_vmstat_pgmigrate_success node_vmstat_pgpgin node_vmstat_pgpgout node_vmstat_pgrefill node_vmstat_pgrotated node_vmstat_pgscan_direct node_vmstat_pgscan_direct_throttle node_vmstat_pgscan_kswapd node_vmstat_pgskip_dma node_vmstat_pgskip_dma32 node_vmstat_pgskip_movable node_vmstat_pgskip_normal node_vmstat_pgsteal_direct node_vmstat_pgsteal_kswapd node_vmstat_pswpin node_vmstat_pswpout node_vmstat_slabs_scanned node_vmstat_swap_ra node_vmstat_swap_ra_hit node_vmstat_thp_collapse_alloc node_vmstat_thp_collapse_alloc_failed node_vmstat_thp_deferred_split_page node_vmstat_thp_fault_alloc node_vmstat_thp_fault_fallback node_vmstat_thp_file_alloc node_vmstat_thp_file_mapped node_vmstat_thp_split_page node_vmstat_thp_split_page_failed node_vmstat_thp_split_pmd node_vmstat_thp_split_pud node_vmstat_thp_swpout node_vmstat_thp_swpout_fallback node_vmstat_thp_zero_page_alloc node_vmstat_thp_zero_page_alloc_failed node_vmstat_unevictable_pgs_cleared node_vmstat_unevictable_pgs_culled node_vmstat_unevictable_pgs_mlocked node_vmstat_unevictable_pgs_munlocked node_vmstat_unevictable_pgs_rescued node_vmstat_unevictable_pgs_scanned node_vmstat_unevictable_pgs_stranded node_vmstat_workingset_activate node_vmstat_workingset_nodereclaim node_vmstat_workingset_refault node_vmstat_zone_reclaim_failed non_structural_schema_condition_controller_adds non_structural_schema_condition_controller_depth non_structural_schema_condition_controller_longest_running_processor_microseconds non_structural_schema_condition_controller_queue_latency non_structural_schema_condition_controller_queue_latency_count non_structural_schema_condition_controller_queue_latency_sum non_structural_schema_condition_controller_retries non_structural_schema_condition_controller_unfinished_work_seconds non_structural_schema_condition_controller_work_duration non_structural_schema_condition_controller_work_duration_count non_structural_schema_condition_controller_work_duration_sum pilot_conflict_inbound_listener pilot_conflict_outbound_listener_http_over_current_tcp pilot_conflict_outbound_listener_http_over_https pilot_conflict_outbound_listener_tcp_over_current_http pilot_conflict_outbound_listener_tcp_over_current_tcp pilot_destrule_subsets pilot_duplicate_envoy_clusters pilot_eds_no_instances pilot_endpoint_not_ready pilot_inbound_updates pilot_info pilot_k8s_cfg_events pilot_k8s_reg_events pilot_no_ip pilot_proxy_convergence_time_bucket pilot_proxy_convergence_time_count pilot_proxy_convergence_time_sum pilot_proxy_queue_time_bucket pilot_proxy_queue_time_count pilot_proxy_queue_time_sum pilot_services pilot_virt_services pilot_vservice_dup_domain pilot_xds pilot_xds_eds_all_locality_endpoints pilot_xds_eds_instances pilot_xds_push_time_bucket pilot_xds_push_time_count pilot_xds_push_time_sum pilot_xds_pushes process_cpu_seconds_total process_max_fds process_open_fds process_resident_memory_bytes process_start_time_seconds process_virtual_memory_bytes process_virtual_memory_max_bytes prometheus_build_info prometheus_config_last_reload_success_timestamp_seconds prometheus_config_last_reload_successful prometheus_engine_queries prometheus_engine_queries_concurrent_max prometheus_engine_query_duration_seconds prometheus_engine_query_duration_seconds_count prometheus_engine_query_duration_seconds_sum prometheus_notifications_alertmanagers_discovered prometheus_notifications_dropped_total prometheus_notifications_queue_capacity prometheus_notifications_queue_length prometheus_rule_evaluation_duration_seconds prometheus_rule_evaluation_duration_seconds_count prometheus_rule_evaluation_duration_seconds_sum prometheus_rule_evaluation_failures_total prometheus_rule_group_duration_seconds prometheus_rule_group_duration_seconds_count prometheus_rule_group_duration_seconds_sum prometheus_rule_group_iterations_missed_total prometheus_rule_group_iterations_total prometheus_sd_azure_refresh_duration_seconds prometheus_sd_azure_refresh_duration_seconds_count prometheus_sd_azure_refresh_duration_seconds_sum prometheus_sd_azure_refresh_failures_total prometheus_sd_consul_rpc_duration_seconds prometheus_sd_consul_rpc_duration_seconds_count prometheus_sd_consul_rpc_duration_seconds_sum prometheus_sd_consul_rpc_failures_total prometheus_sd_dns_lookup_failures_total prometheus_sd_dns_lookups_total prometheus_sd_ec2_refresh_duration_seconds prometheus_sd_ec2_refresh_duration_seconds_count prometheus_sd_ec2_refresh_duration_seconds_sum prometheus_sd_ec2_refresh_failures_total prometheus_sd_file_read_errors_total prometheus_sd_file_scan_duration_seconds prometheus_sd_file_scan_duration_seconds_count prometheus_sd_file_scan_duration_seconds_sum prometheus_sd_gce_refresh_duration prometheus_sd_gce_refresh_duration_count prometheus_sd_gce_refresh_duration_sum prometheus_sd_gce_refresh_failures_total prometheus_sd_kubernetes_events_total prometheus_sd_marathon_refresh_duration_seconds prometheus_sd_marathon_refresh_duration_seconds_count prometheus_sd_marathon_refresh_duration_seconds_sum prometheus_sd_marathon_refresh_failures_total prometheus_sd_openstack_refresh_duration_seconds prometheus_sd_openstack_refresh_duration_seconds_count prometheus_sd_openstack_refresh_duration_seconds_sum prometheus_sd_openstack_refresh_failures_total prometheus_sd_triton_refresh_duration_seconds prometheus_sd_triton_refresh_duration_seconds_count prometheus_sd_triton_refresh_duration_seconds_sum prometheus_sd_triton_refresh_failures_total prometheus_target_interval_length_seconds prometheus_target_interval_length_seconds_count prometheus_target_interval_length_seconds_sum prometheus_target_scrape_pool_sync_total prometheus_target_scrapes_exceeded_sample_limit_total prometheus_target_scrapes_sample_duplicate_timestamp_total prometheus_target_scrapes_sample_out_of_bounds_total prometheus_target_scrapes_sample_out_of_order_total prometheus_target_sync_length_seconds prometheus_target_sync_length_seconds_count prometheus_target_sync_length_seconds_sum prometheus_treecache_watcher_goroutines prometheus_treecache_zookeeper_failures_total prometheus_tsdb_blocks_loaded prometheus_tsdb_compaction_chunk_range_bucket prometheus_tsdb_compaction_chunk_range_count prometheus_tsdb_compaction_chunk_range_sum prometheus_tsdb_compaction_chunk_samples_bucket prometheus_tsdb_compaction_chunk_samples_count prometheus_tsdb_compaction_chunk_samples_sum prometheus_tsdb_compaction_chunk_size_bucket prometheus_tsdb_compaction_chunk_size_count prometheus_tsdb_compaction_chunk_size_sum prometheus_tsdb_compaction_duration_seconds_bucket prometheus_tsdb_compaction_duration_seconds_count prometheus_tsdb_compaction_duration_seconds_sum prometheus_tsdb_compactions_failed_total prometheus_tsdb_compactions_total prometheus_tsdb_compactions_triggered_total prometheus_tsdb_head_active_appenders prometheus_tsdb_head_chunks prometheus_tsdb_head_chunks_created_total prometheus_tsdb_head_chunks_removed_total prometheus_tsdb_head_gc_duration_seconds prometheus_tsdb_head_gc_duration_seconds_count prometheus_tsdb_head_gc_duration_seconds_sum prometheus_tsdb_head_max_time prometheus_tsdb_head_min_time prometheus_tsdb_head_samples_appended_total prometheus_tsdb_head_series prometheus_tsdb_head_series_created_total prometheus_tsdb_head_series_not_found prometheus_tsdb_head_series_removed_total prometheus_tsdb_reloads_failures_total prometheus_tsdb_reloads_total prometheus_tsdb_retention_cutoffs_failures_total prometheus_tsdb_retention_cutoffs_total prometheus_tsdb_tombstone_cleanup_seconds_bucket prometheus_tsdb_tombstone_cleanup_seconds_count prometheus_tsdb_tombstone_cleanup_seconds_sum prometheus_tsdb_wal_corruptions_total prometheus_tsdb_wal_fsync_duration_seconds prometheus_tsdb_wal_fsync_duration_seconds_count prometheus_tsdb_wal_fsync_duration_seconds_sum prometheus_tsdb_wal_truncate_duration_seconds prometheus_tsdb_wal_truncate_duration_seconds_count prometheus_tsdb_wal_truncate_duration_seconds_sum rest_client_request_duration_seconds_bucket rest_client_request_duration_seconds_count rest_client_request_duration_seconds_sum rest_client_request_latency_seconds_bucket rest_client_request_latency_seconds_count rest_client_request_latency_seconds_sum rest_client_requests_total scrape_duration_seconds scrape_samples_post_metric_relabeling scrape_samples_scraped ssh_tunnel_open_count ssh_tunnel_open_fail_count storage_operation_duration_seconds_bucket storage_operation_duration_seconds_count storage_operation_duration_seconds_sum storage_operation_errors_total storage_operation_status_count up volume_manager_total_volumes webhook_client_latency_bucket webhook_client_latency_count webhook_client_latency_sum webhook_client_results webhook_go_alloc webhook_go_bucket_hash_sys webhook_go_frees webhook_go_gc_cpu_fraction webhook_go_gc_sys webhook_go_heap_alloc webhook_go_heap_idle webhook_go_heap_in_use webhook_go_heap_objects webhook_go_heap_released webhook_go_heap_sys webhook_go_last_gc webhook_go_lookups webhook_go_mallocs webhook_go_mcache_in_use webhook_go_mcache_sys webhook_go_mspan_in_use webhook_go_mspan_sys webhook_go_next_gc webhook_go_num_forced_gc webhook_go_num_gc webhook_go_other_sys webhook_go_stack_in_use webhook_go_stack_sys webhook_go_sys webhook_go_total_alloc webhook_go_total_gc_pause_ns webhook_reconcile_count webhook_reconcile_latency_bucket webhook_reconcile_latency_count webhook_reconcile_latency_sum webhook_request_count webhook_request_latencies_bucket webhook_request_latencies_count webhook_request_latencies_sum webhook_work_queue_depth webhook_workqueue_adds_total webhook_workqueue_depth webhook_workqueue_longest_running_processor_seconds_bucket webhook_workqueue_longest_running_processor_seconds_count webhook_workqueue_longest_running_processor_seconds_sum webhook_workqueue_queue_latency_seconds_bucket webhook_workqueue_queue_latency_seconds_count webhook_workqueue_queue_latency_seconds_sum webhook_workqueue_retries_total webhook_workqueue_unfinished_work_seconds_bucket webhook_workqueue_unfinished_work_seconds_count webhook_workqueue_unfinished_work_seconds_sum webhook_workqueue_work_duration_seconds_bucket webhook_workqueue_work_duration_seconds_count webhook_workqueue_work_duration_seconds_sum workqueue_adds_total workqueue_depth workqueue_longest_running_processor_seconds workqueue_queue_duration_seconds_bucket workqueue_queue_duration_seconds_count workqueue_queue_duration_seconds_sum workqueue_retries_total workqueue_unfinished_work_seconds workqueue_work_duration_seconds_bucket workqueue_work_duration_seconds_count workqueue_work_duration_seconds_sum