NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.16k stars 13.43k forks source link

prometheus-ipmi failes to get local ipmi data #278720

Open TuxCoder opened 7 months ago

TuxCoder commented 7 months ago

Describe the bug

Wrong permission management in systemd service file. with message: error="error running ipmimonitoring: exit status 1: /nix/store/zfvw9wzhlz1a55wjyl9g75l6zhyjxy14-freeipmi-1.6.11/sbin/ipmi-sensors: permission denied\n"

Steps To Reproduce

Steps to reproduce the behavior:

  1. have a server with ipmi support
  2. services.prometheus.exporters.ipmi.enable=true
  3. curl "localhost:9290/metrics"

Expected behavior

get Ipmi monitor data

Additional context

prometheus-ipmi uses freeipmi underneath that needs more access as an unprivileged user as described here: https://github.com/prometheus-community/ipmi_exporter#running-as-unprivileged-user

currently the service is configured as a separate user eg. ipmi-exporter. without a configuration or requirement of root.

I got it working with this hack:

systemd.services.prometheus-ipmi-exporter.serviceConfig = {
  PrivateDevices = false;
  DeviceAllow = lib.mkOverride 10 true;
};

I'm not quite sure how to fix this nicely without running this service by default as root. otherwise we could add this params with a config parameter here: nixos/modules/services/monitoring/prometheus/exporters/ipmi.nix eg.

{
  ...
  extraOpts.export_local = mkEnable "enable local export";
  serviceOpts.serviceConfig = {
    ...
  } // (mkIf cfg.export_local {
      User="root";
      PrivateDevices = false;
      DeviceAllow = lib.mkOverride 10 true; # allow all devices
  });
}

I tried to look with strace what goes wrong and it lookgs like it tries to access a bunch of different devices, one of the main ones is /dev/ipmi0, but also some things in /sys/* and other places

Notify maintainers

@WilliButz - exportes @snaar - made ipmi exporter

Thanks and write if you need anything to that I could test, I'm not so into systemd permission management.


Add a :+1: reaction to issues you find important.

snaar commented 7 months ago

I personally just gave up on figuring out permissions and run it as basically root. I run it on a machine where it doesn't matter from security point of view.

snaar commented 7 months ago

I was hesitant to make it run as root by default for everyone, since I feel like as user you need do be explicitly aware that you are doing something like that. Maybe it could have been documented better, but otherwise I'm not sure it's good idea to have it "quietly" run as root for everyone.

snaar commented 7 months ago

Actually I misremembered, for my current setup I go over network to get ipmi data, so it's not read from local machine and root is not needed. This is even more reason to not have root as default.

snaar commented 7 months ago

(This is a common case where your motherboard has out-of-band management controller that you can hit over IP to get IPMI data out of.)

TuxCoder commented 7 months ago

hello @snaar thanks for you opinion, as I tried to get it running, I was really disappointed that there is a nix package but it was not working. And it took me quite some time to figure out how it works, as only user=root is not enough.

So better doc would be very welcome.

Probably to add a "dirty" config parameter as described above, only a bit better naming So not run it as root by default, but give kind of the option for id in a documented way would be nice.