Empty server_pcicfg_uncore in pcm_caladan_init()

[ur4t]

Environment

Hardware

• i7-6700 CPU
• Intel 82599ES NIC

OS & kernel

• Ubuntu 18.04 with kernel 4.15
• Ubuntu 20.04 with kernel 5.4

build/config

Debug enabled, other options disabled, following all the instructions mentioned in github readme.

iokerneld

sudo gdb iokerneld
(gdb) r
Starting program: /home/user/caladan/iokerneld
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
CPU 04| <6> entering 'iokernel' init phase
CPU 04| <6> init -> base
CPU 04| <5> cpu: detected 8 cores, 1 nodes
CPU 04| <5> time: detected 3407 ticks / us
CPU 04| <6> init: entering 'early' init
CPU 04| <6> init: entering 'normal' init
CPU 04| <6> init: entering 'late' init
[  0.000301] CPU 04| <6> init -> ksched
[  0.000325] CPU 04| <6> init -> sched
[  0.000329] CPU 04| <5> sched: CPU configuration...
        node 0: [0,4][1,5][2,6][3,7]
[  0.000341] CPU 04| <5> sched: dataplane on 4, control on 0
[  0.000346] CPU 04| <6> init -> simple
[  0.000358] CPU 04| <6> init -> numa
[  0.000365] CPU 04| <6> init -> ias
IBRS and IBPB supported  : yes
STIBP supported          : yes
Spec arch caps supported : no
IBRS enabled in the kernel   : no
STIBP enabled in the kernel  : no
[New Thread 0x7ffff7045700 (LWP 134361)]
[New Thread 0x7ffff6844700 (LWP 134362)]
[New Thread 0x7ffff6043700 (LWP 134363)]
terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check: __n (which is 0) >= this->size() (which is 0)

Thread 1 "iokerneld" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff7a71859 in __GI_abort () at abort.c:79
#2  0x00007ffff7d2f951 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff7d3b47c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff7d3b4e7 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff7d3b799 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff7d323eb in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00005555556b2ec9 in std::vector<std::shared_ptr<pcm::ServerPCICFGUncore>, std::allocator<std::shared_ptr<pcm::ServerPCICFGUncore> > >::_M_range_check (__n=0, this=<optimized out>)
    at /usr/include/c++/9/ext/atomicity.h:69
#8  std::vector<std::shared_ptr<pcm::ServerPCICFGUncore>, std::allocator<std::shared_ptr<pcm::ServerPCICFGUncore> > >::at (__n=0, this=<optimized out>)
    at /usr/include/c++/9/bits/stl_vector.h:1091
#9  pcm_caladan_init (socket=0) at pcm-caladan.cpp:28
#10 0x000055555560050f in ias_bw_init () at iokernel/ias_bw.c:260
#11 0x0000555555601a38 in ias_init () at iokernel/ias.c:555
#12 0x00005555555eb4d2 in run_init_handlers (phase=0x5555556f286b "iokernel", h=0x555555770aa0 <iok_init_handlers>, nr=13) at iokernel/main.c:65
#13 0x00005555555eb968 in main (argc=1, argv=0x7fffffffe6a8) at iokernel/main.c:190
(gdb)

test.cpp

#include "cpucounters.h"
#include <cstdio>
int main(void) {
    printf("1\n");
    auto a = pcm::PCM::getInstance();
    printf("2\n");
    a->server_pcicfg_uncore.at(0); //throws std::out_of_range
    //a->server_pcicfg_uncore[0];
    printf("3\n");
}

Compile this test with g++ test.cpp -I caladan/deps/pcm/ -L caladan/deps/pcm -o test-pcm -lPCM -lpthread. Run sudo ./test-pcm and here is the output:

1
IBRS and IBPB supported  : yes
STIBP supported          : yes
Spec arch caps supported : no
IBRS enabled in the kernel   : no
STIBP enabled in the kernel  : no
2
terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check: __n (which is 0) >= this->size() (which is 0)
Aborted

[joshuafried]

We don't currently have support for memory bandwidth monitoring on non-Xeon CPUs. You can launch iokerneld with the nobw flag (sudo ./iokerneld ias nobw).

[ur4t]

Solved. Thanks a lot.