hubblo-org / scaphandre

⚡ Energy consumption metrology agent. Let "scaph" dive and bring back the metrics that will help you make your systems and applications more sustainable !
Apache License 2.0
1.61k stars 109 forks source link

Export metrics from Proxmox's VMs #222

Open cs35-owncloud opened 2 years ago

cs35-owncloud commented 2 years ago

Hello,

first thanks for this tool ! I used the bug label for this issue, but it's mostly an information I couldn't find on exporter-qemu page, sorry if it doesn't make sense.

Bug description

Running scaphandre on Proxmox does not seem to be working.

To Reproduce

  1. Install scaphandre on both Proxmox and one VM (for testing purpose one is enough)
  2. Edit your VM's configuration to share the scaphandre directory (i.e /etc/pve/qemu-server/xx.conf), add an args line (here /mnt/scaphandre should be my source directory for scaphandre's data between Proxmox and the VMs)
    args: -fsdev local,security_model=passthrough,id=fsdev0,path=/mnt/scaphandre -device virtio-9p-pci,id=fs0,fsdev=fsdev0,mount_tag=scaphandre,bus=pci.0,addr=0x4
  3. Run  scaphandre qemuon Proxmox
  4. Create the mountpoint on the VM
    mkdir /var/scaphandre
  5. mount it
    mount -t 9p scaphandre /var/scaphandre/
  6. Run scaphandre --vm prometheus on the VM
  7. Error given
# scaphandre --vm prometheus                       
scaphandre::sensors::powercap_rapl: Powercap_rapl path is: /var/scaphandre
Scaphandre prometheus exporter
Sending ⚡ metrics
Press CTRL-C to stop scaphandre
thread 'main' panicked at 'Trick: if you are running on a vm, do not forget to use --vm parameter invoking scaphandre at the command line', src/sensors/mod.rs:263:18
stack backtrace:
   0:     0x55f713f360f0 - std::backtrace_rs::backtrace::libunwind::trace::h72c2fb8038f1bbee
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/../../backtrace/src/backtrace/libunwind.rs:96
   1:     0x55f713f360f0 - std::backtrace_rs::backtrace::trace_unsynchronized::h1e3b084883f1e78c
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/../../backtrace/src/backtrace/mod.rs:66
   2:     0x55f713f360f0 - std::sys_common::backtrace::_print_fmt::h3bf6a7ebf7f0394a
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/sys_common/backtrace.rs:79
   3:     0x55f713f360f0 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h2e8cb764b7fe02e7
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/sys_common/backtrace.rs:58
   4:     0x55f713f590cc - core::fmt::write::h7a1184eaee6a8644
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/fmt/mod.rs:1080
   5:     0x55f713f2f8c2 - std::io::Write::write_fmt::haeeb374d93a67eac
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/io/mod.rs:1516
   6:     0x55f713f3859d - std::sys_common::backtrace::_print::h1d14a7f6ad632dc8
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/sys_common/backtrace.rs:61
   7:     0x55f713f3859d - std::sys_common::backtrace::print::h301abac8bb2e3e81
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/sys_common/backtrace.rs:48
   8:     0x55f713f3859d - std::panicking::default_hook::{{closure}}::hde0cb80358a6920a
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:208
   9:     0x55f713f38248 - std::panicking::default_hook::h9b1a691049a0ec8f
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:227
  10:     0x55f713f38c81 - std::panicking::rust_panic_with_hook::h2bdec87b60580584
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:577
  11:     0x55f713f38829 - std::panicking::begin_panic_handler::{{closure}}::h101ca09d9df5db47
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:484
  12:     0x55f713f3655c - std::sys_common::backtrace::__rust_end_short_backtrace::h3bb85654c20113ca
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/sys_common/backtrace.rs:153
  13:     0x55f713f387e9 - rust_begin_unwind
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
  14:     0x55f713f56cf1 - core::panicking::panic_fmt::h48c31e1e3d550146
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/panicking.rs:85
  15:     0x55f713f56a83 - core::option::expect_failed::hf3f43f1792267e24
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/option.rs:1226
  16:     0x55f7139e90c7 - scaphandre::sensors::Topology::add_cpu_cores::hea3766cbbc1cd056
  17:     0x55f71398bf6d - <scaphandre::sensors::powercap_rapl::PowercapRAPLSensor as scaphandre::sensors::Sensor>::generate_topology::hee3bfcc8d178a939
  18:     0x55f71398c3aa - <scaphandre::sensors::powercap_rapl::PowercapRAPLSensor as scaphandre::sensors::Sensor>::get_topology::h2298c1e3ecf308a3
  19:     0x55f7139d6af9 - <scaphandre::exporters::prometheus::PrometheusExporter as scaphandre::exporters::Exporter>::run::hd33a047640847d94
  20:     0x55f713947ea6 - scaphandre::run::hc810d2790be74778
  21:     0x55f7139333c7 - scaphandre::main::h3c7daf56cbaf0841
  22:     0x55f713931893 - std::sys_common::backtrace::__rust_begin_short_backtrace::hb0630873a13c9c16
  23:     0x55f7139318a9 - std::rt::lang_start::{{closure}}::h71210feee44b24fd
  24:     0x55f713f39197 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::he179d32a5d10d957
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/ops/function.rs:259
  25:     0x55f713f39197 - std::panicking::try::do_call::hcb3d5e7be089b2b4
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:381
  26:     0x55f713f39197 - std::panicking::try::h7ac93b0cd56fb701
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:345
  27:     0x55f713f39197 - std::panic::catch_unwind::h7b40e396c93a4fcd
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panic.rs:382
  28:     0x55f713f39197 - std::rt::lang_start_internal::h142b9cc66267fea1
                               at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/rt.rs:51
  29:     0x55f713933572 - main
  30:     0x7f675a518d0a - __libc_start_main
  31:     0x55f71393123a - _start
  32:                0x0 - <unknown>

Expected behavior

Well obviously it cannot work since the scaphandre directory on the Proxmox is not something like /var/lib/libvirt/scaphandre/DOMAIN_NAME as described in the export-qemu documentation.

So it's not really a bug, but is it possible to configure the scaphandre qemu directory ? Is there any workaround to run scaphandre on Proxmox and get data from the VMs ? Thanks !

Environment

Proxmox

VMs

Kernels will be updated soon, but they are not relevant to the issue since I get the RAPL data on the Proxmox when running scaphandre stdout -t 15:

# scaphandre stdout -t 15
Scaphandre stdout exporter
Sending ⚡ metrics
Measurement step is: 2s
Host:   0 W
    package     dram
Top 5 consumers:
Power       PID Exe
No processes found yet or filter returns no value.
------------------------------------------------------------

Host:   30.287504 W
    package     dram
Socket0 30.287504 W |   22.99262 W  

Top 5 consumers:
Power       PID Exe
0.264464 W  1553    "kvm"
0.062967 W  66  "ksmd"
0.03778 W   2149    "sshd"
0 W 1   "systemd"
0 W 2   "kthreadd"
------------------------------------------------------------
bpetit commented 1 year ago

Hi !

Thanks for raising this topic.

This is more of an enhancement as this context has never been tested before.

Thatoo commented 3 weeks ago

Do you know any tutorial to install Scaphandre on proxmox host and its VM?