OpenFastPath / ofp

OpenFastPath project
BSD 3-Clause "New" or "Revised" License
349 stars 126 forks source link

Segmentation fault when running examples #202

Closed bestgen closed 6 years ago

bestgen commented 6 years ago

Hello. When I try to run the webserver examples of OFP, the system is not able to open threads and crashed. The error messages are:

[admin@localhost ofp]$ sudo ./example/webserver/webserver -i ens33 -c 4 -f ./example/webserver/ofp.cli &
[6] 5865
[admin@localhost ofp]$ RLIMIT_CORE: 0/-1
Setting to max: 0
HW time counter freq: 3408013176 hz

_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishm.c:881:_odp_ishm_reserve():No huge pages, fall back to normal pages. check: /proc/sys/vm/nr_hugepages.
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
PKTIO: initialized loop interface.
PKTIO: initialized pcap interface.
PKTIO: initialized ipc interface.
PKTIO: initialized socket mmap, use export ODP_PKTIO_DISABLE_SOCKET_MMAP=1 to disable.
PKTIO: initialized socket mmsg,use export ODP_PKTIO_DISABLE_SOCKET_MMSG=1 to disable.
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory

ODP system info
---------------
ODP API version: 1.17.0
CPU model:       Intel(R) Core(TM) i7-6700 CPU
CPU freq (hz):   3407504000
Cache line size: 64
Core count:      4

Running ODP appl: "webserver"
-----------------
IF-count:        1
Using IFs:       ens33

(The program stuck here and show segmentation fault when it was force-terminated )

It shows that ODP failed to allocated the OFP shared memory. Is it the result of not enough available memory? I set up OFP on the virtual machine based on Vmware, and I have allocate the virtual machine 4 cores and 2GB memory. If it is, how much is enough?

Besides that, I debug the codes and found the program get stuck in the sentence params->num_vrf = OFP_NUM_VRF; in the function ofp_init_global_param_from_file(), which is in line 108 of app_main.c. And the debug error shows No source available for "vfprintf() at 0x7ffff66466e4".It is weird since that sentence is just an assignment instruction.

Can someone please offer some help? Thank you.

JereLeppanen commented 6 years ago

Could you try the same with strace to see which system call, if any, is blocking?

$ sudo strace -f ./example/webserver/webserver -i ens33 -c 4 -f ./example/webserver/ofp.cli

bestgen commented 6 years ago

Thank you for your reply @JereLeppanen . I think it stuck in opening /usr/local/bin/ofp/etc/ofp.conf. ofp.conf, even etc doesn't exist beneath the ofp path or in the whole file system. /usr/local/etc only has the odp folder and has no ofp.conf file, either.

[pid  2848] fstat(15, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
[pid  2848] mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fcad2383000
[pid  2848] read(15, "processor\t: 0\nvendor_id\t: Genuin"..., 1024) = 1024
[pid  2848] read(15, "enuineIntel\ncpu family\t: 6\nmodel"..., 1024) = 1024
[pid  2848] close(15)                   = 0
[pid  2848] munmap(0x7fcad2383000, 4096) = 0
[pid  2848] write(1, "\nODP system info\n---------------"..., 33
ODP system info
---------------
) = 33
[pid  2848] write(1, "ODP API version: 1.17.0\n", 24ODP API version: 1.17.0
) = 24
[pid  2848] write(1, "CPU model:       Intel(R) Core(T"..., 47CPU model:       Intel(R) Core(TM) i7-6700 CPU
) = 47
[pid  2848] write(1, "CPU freq (hz):   3407645000\n", 28CPU freq (hz):   3407645000
) = 28
[pid  2848] write(1, "Cache line size: 64\n", 20Cache line size: 64
) = 20
[pid  2848] write(1, "Core count:      4\n\n", 20Core count:      4

) = 20
[pid  2848] write(1, "Running ODP appl: \"webserver\"\n--"..., 48Running ODP appl: "webserver"
-----------------
) = 48
[pid  2848] write(1, "IF-count:        1\n", 19IF-count:        1
) = 19
[pid  2848] write(1, "Using IFs:       ens33\n", 23Using IFs:       ens33
) = 23
[pid  2848] write(1, "\n", 1
)           = 1
[pid  2848] open("/usr/local/bin/ofp/etc/ofp.conf", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  2848] --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x100000000} ---
[pid  2848] +++ killed by SIGSEGV (core dumped) +++
<... accept resumed> 0x7ffe629ff960, 0x7ffe629ff8d8) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
--- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=2848, si_uid=0} ---
+++ killed by SIGTERM +++
Segmentation fault
[admin@localhost ofp]$ 
JereLeppanen commented 6 years ago

Well, the strace shows that the open() call for ofp.conf returned an error, so at least it's not stuck in a system call. Please try disabling the configuration file with ./configure --disable-libconfig.

Assuming you're using a not-very-old OFP master, you could also try with a newer version of ODP. The latest version is 1.19.0.1.

bestgen commented 6 years ago

Thanks @JereLeppanen . I found there are some updates of ofp as well as the user guide. I install the latest version and it works well. Thank you for your help.