mweinelt / kea-exporter

Export Kea Metrics in the Prometheus Exposition Format
MIT License
34 stars 17 forks source link

Fails to start with `ValueError: Incorrect label names` #29

Closed jmbaur closed 2 years ago

jmbaur commented 2 years ago

Running the latest tag (v0.5.0) I get this error on startup:

kea-exporter[10555]: Listening on http://0.0.0.0:9547
kea-exporter[10555]: Traceback (most recent call last):
kea-exporter[10555]:   File "/nix/store/cms5b30wjy5y96zwnspki4lh4zsnv1gh-kea-exporter-0.5.0/bin/.kea-exporter-wrapped", line 9, in <module>
kea-exporter[10555]:     sys.exit(cli())
kea-exporter[10555]:   File "/nix/store/apiha8lds5nbj1039kxj90vc4xr5q0sf-python3.10-click-8.1.3/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
kea-exporter[10555]:     return self.main(*args, **kwargs)
kea-exporter[10555]:   File "/nix/store/apiha8lds5nbj1039kxj90vc4xr5q0sf-python3.10-click-8.1.3/lib/python3.10/site-packages/click/core.py", line 1055, in main
kea-exporter[10555]:     rv = self.invoke(ctx)
kea-exporter[10555]:   File "/nix/store/apiha8lds5nbj1039kxj90vc4xr5q0sf-python3.10-click-8.1.3/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
kea-exporter[10555]:     return ctx.invoke(self.callback, **ctx.params)
kea-exporter[10555]:   File "/nix/store/apiha8lds5nbj1039kxj90vc4xr5q0sf-python3.10-click-8.1.3/lib/python3.10/site-packages/click/core.py", line 760, in invoke
kea-exporter[10555]:     return __callback(*args, **kwargs)
kea-exporter[10555]:   File "/nix/store/cms5b30wjy5y96zwnspki4lh4zsnv1gh-kea-exporter-0.5.0/lib/python3.10/site-packages/kea_exporter/cli.py", line 22, in cli
kea-exporter[10555]:     exporter.update()
kea-exporter[10555]:   File "/nix/store/cms5b30wjy5y96zwnspki4lh4zsnv1gh-kea-exporter-0.5.0/lib/python3.10/site-packages/kea_exporter/kea.py", line 637, in update
kea-exporter[10555]:     metric.labels(**labels).set(value)
kea-exporter[10555]:   File "/nix/store/zcrn80lfchkiggn63yvjj6cm0pz213rv-python3.10-prometheus-client-0.14.1/lib/python3.10/site-packages/prometheus_client/metrics.py", >
kea-exporter[10555]:     raise ValueError('Incorrect label names')
kea-exporter[10555]: ValueError: Incorrect label names
systemd[1]: prometheus-kea-exporter.service: Main process exited, code=exited, status=1/FAILURE

Adding v6-allocation-fail-no-pools, v6-allocation-fail-shared-network, v6-allocation-fail-classes, & v6-allocation-fail-subnet to the metric_dhcp6_subnet_ignore list does cause the exporter to not crash. Was adding these to the ignore list part of the proposed changes from #26? Let me know if seeing my dhcp6 config would be helpful.

mweinelt commented 2 years ago

Thanks. I'll hope to be able to look into this on the weekend. Until the you can pin to a version < 0.5 as a workaround.

mweinelt commented 2 years ago

Let me know if seeing my dhcp6 config would be helpful.

I don't think that will be necessary.

We should probably extend the NixOS test, because it currently does not exhibit this error, because the config is too simple to expose these kinds of metrics.

glendekoning commented 2 years ago

I have had something similar to this with v4 allocation failed too. Am yet to debug it. I guess your right my config when I was testing the patches wasn't very complex nor was I able to test the failed allocations.

glendekoning commented 2 years ago

I think it is at the subnet level

glendekoning commented 2 years ago

So i was able to get some debugs from my dhcp4 server when it failed. 'v4-allocation-fail' looks to be the issue i think it could be at the subnet level, as we have not accommodated this. The question is do we need to track the global counts when we have the subnet breakdown.

i was thinking of ignoring all global v4-allocation-fail and only tracking subnet level?

Listening on http://0.0.0.0:9547
Unhandled metric 'v4-allocation-fail', please open an issue at https://github.com/mweinelt/kea-exporter/issues
Traceback (most recent call last):
  File "/usr/bin/kea-exporter", line 8, in <module>
    sys.exit(cli())
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/kea_exporter/cli.py", line 22, in cli
    exporter.update()
  File "/usr/lib/python3.10/site-packages/kea_exporter/kea.py", line 637, in update
    metric.labels(**labels).set(value)
  File "/usr/lib/python3.10/site-packages/prometheus_client/metrics.py", line 182, in labels
    raise ValueError('Incorrect label names')
ValueError: Incorrect label names
# Subnet Statistic
      "subnet[1].v4-allocation-fail": [
        [
          1,
          "2022-10-25 20:49:32.007366"
        ]
      ],
      "subnet[1].v4-allocation-fail-classes": [
        [
          1,
          "2022-10-25 20:49:32.007691"
        ]
      ],
      "subnet[1].v4-allocation-fail-subnet": [
        [
          1,
          "2022-10-25 20:49:32.007131"
        ]
      ],
      "subnet[1].v4-reservation-conflicts": [
        [
          0,
          "2022-10-25 20:48:24.516653"
        ]
      ],
# Global Statistic
      "v4-allocation-fail": [
        [
          1,
          "2022-10-25 20:49:32.007357"
        ],
        [
          0,
          "2022-10-25 20:48:24.243656"
        ]
      ],
      "v4-allocation-fail-classes": [
        [
          1,
          "2022-10-25 20:49:32.007676"
        ],
        [
          0,
          "2022-10-25 20:48:24.243657"
        ]
      ],
      "v4-allocation-fail-no-pools": [
        [
          0,
          "2022-10-25 20:48:24.243659"
        ]
      ],
      "v4-allocation-fail-shared-network": [
        [
          0,
          "2022-10-25 20:48:24.243665"
        ]
      ],
      "v4-allocation-fail-subnet": [
        [
          1,
          "2022-10-25 20:49:32.007118"
        ],
        [
          0,
          "2022-10-25 20:48:24.243666"
        ]
      ],