LukeFenton commented 5 years ago

Using BGP reader to parse a single MRT file.

V1 - bgpreader -d singlefile -o upd-file,segfaultErrorTest_MRT - Running this produced no errors and parsed all of the updates within the file.

V2 - bgpreader -d singlefile -o upd-file=segfaultErrorTest_MRT - Running this produced this error 
'Segmentation fault'

This same error occurs when running through the V2 PyBGPStream using:

stream.set_data_interface("singlefile")
stream.set_data_interface_option("singlefile", "upd-file",  PATH_TO_MRT)

Produces this result:

bash: line 6: 32401 Segmentation fault      python2.7 -m ikpdb --ikpdb-port=15471 --ikpdb-working-directory=$project_path "$file"

alistairking commented 5 years ago

Could you provide the MRT file that causes the crash? Or does it happen with any file?

LukeFenton commented 5 years ago

I have tried it on 4 different MRTs this morning and am getting the same result. successful processing on V1 with a SegFault on V2

LukeFenton commented 5 years ago

Have tried one of the MRT files from the Routeviews archive. This appeared to work without the SegFault, though the files I have work on V1 not the V2

LukeFenton commented 5 years ago

Here is the file which is causing the Seg Fault.

SegFaultMRT.zip

alistairking commented 5 years ago

I've confirmed that I can reproduce this problem, and it affects the underlying parser library, libparsebgp. I'm going to transfer this issue over to that repo.

LukeFenton commented 5 years ago

I've confirmed that I can reproduce this problem, and it affects the underlying parser library, libparsebgp. I'm going to transfer this issue over to that repo.

Thanks for that. Hopefully this can be resolved.

kenkeys commented 5 years ago

Fixed in commit 6c1f385

LukeFenton commented 5 years ago

Hi, in order to use this update which package is needing to be re downloaded and reinstalled ?

digizeph commented 5 years ago

Hi, in order to use this update which package is needing to be re downloaded and reinstalled ?

You can pull the latest libbgpstream master branch (i.e. bgpstream v2), and reinstall it. It has updated to use more recent version of libparsebgp which includes this fix.

LukeFenton commented 5 years ago

I have tried to install this latest update and am getting this error.

/bin/sh ../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../.. -I../.. -I../../lib -I../../lib/utils -I../../common -I../../lib/formats/libparsebgp/lib -I../../lib/formats/libparsebgp/lib/bgp -I../../lib/formats/libparsebgp/lib/mrt -I../../lib/formats/libparsebgp/lib/bmp -g -O2 -pthread -MT bgpstream_parsebgp_common.lo -MD -MP -MF .deps/bgpstream_parsebgp_common.Tpo -c -o bgpstream_parsebgp_common.lo bgpstream_parsebgp_common.c libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../.. -I../.. -I../../lib -I../../lib/utils -I../../common -I../../lib/formats/libparsebgp/lib -I../../lib/formats/libparsebgp/lib/bgp -I../../lib/formats/libparsebgp/lib/mrt -I../../lib/formats/libparsebgp/lib/bmp -g -O2 -pthread -MT bgpstream_parsebgp_common.lo -MD -MP -MF .deps/bgpstream_parsebgp_common.Tpo -c bgpstream_parsebgp_common.c -fPIC -DPIC -o .libs/bgpstream_parsebgp_common.o bgpstream_parsebgp_common.c: In function ‘bgpstream_parsebgp_populate_record’: bgpstream_parsebgp_common.c:589:23: error: ‘PARSEBGP_TRUNCATED_MSG’ undeclared (first use in this function); did you mean ‘PARSEBGP_INVALID_MSG’? } else if (err == PARSEBGP_TRUNCATED_MSG) { ^~~~~~ PARSEBGP_INVALID_MSG bgpstream_parsebgp_common.c:589:23: note: each undeclared identifier is reported only once for each function it appears in make[3]: [bgpstream_parsebgp_common.lo] Error 1 make[3]: Leaving directory `/home/lf/src/v2_tools/libbgpstream-2.0.0/lib/formats' make[2]: [install-recursive] Error 1 make[2]: Leaving directory /home/lf/src/v2_tools/libbgpstream-2.0.0/lib/formats' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory/home/lf/src/v2_tools/libbgpstream-2.0.0/lib' make: *** [install-recursive] Error 1

digizeph commented 5 years ago

Hi @LukeFenton, I just tried an fresh install of bgpstream v2 on Mac, and it works fine:

git clone https://github.com/CAIDA/libbgpstream.git
cd libbgpstream
./autogen.sh
./configure --without-transport-kafka --without-kafka --without-betabmp
make

Could you check the version of the libparsebgp submodule at lib/formats/libparsebgp? It should point to this version (d9c2dd56c1db47522cea3630448321ce8a6e7f6f):

commit d9c2dd56c1db47522cea3630448321ce8a6e7f6f (HEAD)

LukeFenton commented 5 years ago

Hi I have used the above instructions and this has worked. However following the instructions on the v2 release page did not work.

I am now getting this error when installing pybgpstream using the v2 release instructions again. Are they out of date?

this is the error -

[lf@ip-172-31-19-137 pybgpstream-2.0.0]$ python setup.py build_ext running build_ext building '_pybgpstream' extension creating build creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/src gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c src/_pybgpstream_module.c -o build/temp.linux-x86_64-2.7/src/_pybgpstream_module.o In file included from src/_pybgpstream_module.c:27:0: src/_pybgpstream_bgpelem.h:30:10: fatal error: bgpstream_elem.h: No such file or directory

include "bgpstream_elem.h"

      ^~~~~~~~~~~~~~~~~~

compilation terminated. error: command 'gcc' failed with exit status 1

LukeFenton commented 5 years ago

Turns out i missed the make install part, Ignore the last post.

My apologies

L

CAIDA / libparsebgp

BGPReader Segmentation Fault V2 #46

include "bgpstream_elem.h"