Is there any way to remove empty labels?

[wrfly]

Hello there, I want to collect container metrics to prometheus via cadvisor, but the /metrics exposed is too large, there are about 70+ containers in the host, and the /metrics's size is 43MB.

root@vmserver:~# wget 'http://10.6.6.10:8080/metrics' --spider
Spider mode enabled. Check if remote file exists.
--2018-03-22 10:25:41--  http://10.6.6.10:8080/metrics
Connecting to 10.6.6.10:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 45233291 (43M) [text/plain]
Remote file exists.

root@vmserver:~#

Here is a single metric:

container_fs_write_seconds_total{container_label_Author="",container_label_annotation_io_kubernetes_container_hash="",container_label_annotation_io_kubernetes_container_ports="",container_label_annotation_io_kubernetes_container_restartCount="",container_label_annotation_io_kubernetes_container_terminationMessagePath="",container_label_annotation_io_kubernetes_container_terminationMessagePolicy="",container_label_annotation_io_kubernetes_pod_terminationGracePeriod="",container_label_annotation_kubernetes_io_config_hash="",container_label_annotation_kubernetes_io_config_seen="",container_label_annotation_kubernetes_io_config_source="",container_label_annotation_scheduler_alpha_kubernetes_io_critical_pod="",container_label_app="",container_label_com_docker_compose_config_hash="a34802588ec530fc35eaa306b7e675d00d3a6dca2dec18bad114e515804fd1b1",container_label_com_docker_compose_container_number="1",container_label_com_docker_compose_oneoff="False",container_label_com_docker_compose_project="xuanwu",container_label_com_docker_compose_service="neo4j-server",container_label_com_docker_compose_version="1.18.0",container_label_com_microscaling_docker_dockerfile="",container_label_com_microscaling_license="",container_label_component="",container_label_controller_revision_hash="",container_label_daemon="",container_label_grafanak8sapp="",container_label_io_kubernetes_container_logpath="",container_label_io_kubernetes_container_name="",container_label_io_kubernetes_docker_type="",container_label_io_kubernetes_pod_name="",container_label_io_kubernetes_pod_namespace="",container_label_io_kubernetes_pod_uid="",container_label_io_kubernetes_sandbox_id="",container_label_k8s_app="",container_label_maintainer="",container_label_mon="",container_label_mon_cluster="",container_label_org_label_schema_build_date="",container_label_org_label_schema_description="",container_label_org_label_schema_name="",container_label_org_label_schema_schema_version="",container_label_org_label_schema_url="",container_label_org_label_schema_vcs_ref="",container_label_org_label_schema_vcs_url="",container_label_org_label_schema_vendor="",container_label_org_label_schema_version="",container_label_pod_template_generation="",container_label_pod_template_hash="",container_label_restartcount="",container_label_rook_cluster="",container_label_rook_file_system="",container_label_rook_object_store="",container_label_tier="",device="/dev/sda3",id="/docker/b7a429742fadd45a875dac2ce03d200adfebd39c7fa9133e92867369eb003531",image="library/neo4j:3.2.2",name="neo4j-server"} 0

How can I remove the labels that the container didn't use to reduce the file size of /metric ?

Or set something for customizing the labels.

Thanks.

[dashpole]

Is this the cAdvisor embedded in the kubelet, or a stand-alone cAdvisor?

[wrfly]

@dashpole stand-alone

[dashpole]

There isn't a way to do this right now. There was someone who was trying to add a whitelist for labels a little while back: https://github.com/google/cadvisor/pull/1735, but it wasn't finished. I would be happy to review a PR that does something similar.