search

travelping / upg-vpp

User Plane Gateway (UPG) based on VPP
Apache License 2.0
150 stars 51 forks source link

Segmentation Fault #263

Closed mohammedyaqub closed 2 years ago

mohammedyaqub commented 2 years ago

Hi,

We are trying to run travelping/upg-vpp UPF executable binary but facing segmentation fault.

Environment details.

Tried debugging through GDB below is the output. 

Program received signal SIGSEGV, Segmentation fault.
mspace_malloc (msp=0x0, bytes=1116) at /home/ubuntu/upg-vpp/vpp/src/vppinfra/dlmalloc.c:4371
4371      if (!PREACTION(ms)) {

The back trace output shows.

mspace_malloc (msp=0x0, bytes=1116) at /home/ubuntu/upg-vpp/vpp/src/vppinfra/dlmalloc.c:4371
#1  0x00007ffff7f26919 in mspace_get_aligned (msp=0x0, n_user_data_bytes=1116, align=16, align_offset=<optimized out>)
    at /home/ubuntu/upg-vpp/vpp/src/vppinfra/dlmalloc.c:4254
#2  0x00007ffff7f7ce57 in clib_mem_alloc_aligned_at_offset (size=1096, align=8, align_offset=0, 
    os_out_of_memory_on_failure=1) at /home/ubuntu/upg-vpp/vpp/src/vppinfra/mem.h:142
#3  vec_resize_allocate_memory (v=<optimized out>, length_increment=64, data_bytes=1096, header_bytes=<optimized out>, 
    data_align=8, numa_id=255) at /home/ubuntu/upg-vpp/vpp/src/vppinfra/vec.c:67
#4  0x00007ffff7f3e7e6 in _vec_resize_inline (v=0x0, length_increment=64, data_bytes=1024, header_bytes=64, 
    data_align=8, numa_id=255) at /home/ubuntu/upg-vpp/vpp/src/vppinfra/vec.h:170
#5  _hash_create (elts=64, h_user=<optimized out>) at /home/ubuntu/upg-vpp/vpp/src/vppinfra/hash.c:747
#6  _hash_set3 (v=0x0, key=93824993155599, value=0x0, old_value=0x0) at /home/ubuntu/upg-vpp/vpp/src/vppinfra/hash.c:845
#7  0x00005555555a0fe7 in upf_init ()
#8  0x00005555555a208f in main ()

Kindly help @ivan4th.

Thanks.

sergeymatov commented 2 years ago

seems like an OOM issue. Please check that VPP heap size is reasonably enough as well as OS memory fits it.

mohammedyaqub commented 2 years ago

Thank you for the response,

my apologize for late although increasing the size of main heap size the problem still appears to be the same.

- I tried to print the *vm there everything appeared to be `NULL ` below is the output.

(gdb) p *vm $1 = {cacheline0 = 0x55555566e600 "", clib_time = {total_cpu_time = 0, last_cpu_time = 0, clocks_per_second = 0, seconds_per_clock = 0, init_cpu_time = 0, init_reference_time = 0, last_verify_cpu_time = 0, last_verify_reference_time = 0, log2_clocks_per_second = 0, log2_clocks_per_frequency_verify = 0, damping_constant = 0}, time_offset = 0, time_last_barrier_release = 0, cpu_time_last_node_dispatch = 0, cpu_time_main_loop_start = 0, main_loop_count = 0, main_loop_vectors_processed = 0, main_loop_nodes_processed = 0, internal_node_vectors = 0, internal_node_calls = 0, internal_node_vectors_last_clear = 0, internal_node_calls_last_clear = 0, internal_node_last_vectors_per_main_loop = 0, vlib_node_runtime_perf_callbacks = { curr = 0x0, next = 0x0, spare = 0x0, lock = 0x0}, main_loop_exit_set = 0, main_loop_exit_now = 0, main_loop_exit = { regs = {0, 0, 0, 0, 0, 0, 0, 0}}, main_loop_error = 0x0, name = 0x0, heap_base = 0x0, heap_aligned_base = 0x0, heap_size = 0, buffer_main = 0x0, physmem_main = {flags = 0, base_addr = 0, max_size = 0, maps = 0x0, pmalloc_main = 0x0}, node_main = {nodes = 0x0, node_by_name = 0x0, flags = 0, nodes_by_type = {0x0, 0x0, 0x0, 0x0}, pending_local_interrupts = 0x0, pending_remote_interrupts = 0x0, pending_remote_interrupts_notify = 0x0, pending_interrupt_lock = 0x0, polling_threshold_vector_length = 0, interrupt_threshold_vector_length = 0, next_frames = 0x0, pending_frames = 0x0, timing_wheel = 0x0, signal_timed_event_data_pool = 0x0, data_from_advancing_timing_wheel = 0x0, time_next_process_ready = 0, processes = 0x0, current_process_index = 0, suspended_process_frames = 0x0, recycled_event_data_vectors = 0x0, input_node_counts_by_state = {0, 0, 0}, frame_size_hash = 0x0, frame_sizes = 0x0, time_last_runtime_stats_clear = 0, node_registrations = 0x0, node_by_error = 0x0}, cli_main = {commands = 0x0, command_index_by_path = 0x0, cli_command_registrations = 0x0, sort_vector = 0x0, perf_counter_cbs = 0x0, perf_counter_cbs_tmp = 0x0}, trace_main = {trace_buffer_pool = 0x0, last_main_loop_count = 0, filter_node_index = 0, filter_flag = 0, filter_count = 0, trace_enable = 0, nodes = 0x0, verbose = 0, trace_buffer_callback = 0x0, add_trace_callback = 0x0}, dispatch_pcap_main = {lock = 0x0, file_name = 0x0, n_packets_to_capture = 0, packet_type = PCAP_PACKET_TYPE_null, n_packets_captured = 0, flags = 0, file_descriptor = 0, n_pcap_data_written = 0, pcap_data = 0x0, packets_read = 0x0, timestamps = 0x0, min_packet_bytes = 0, max_packet_bytes = 0}, dispatch_pcap_enable = 0, dispatch_buffer_trace_nodes = 0x0, pcap_buffer = 0x0, pcap = {pcap_rx_enable = 0 '\000', pcap_tx_enable = 0 '\000', pcap_drop_enable = 0 '\000', pad1 = 0 '\000', max_bytes_per_pkt = 0, pcap_sw_if_index = 0, pcap_main = {lock = 0x0, file_name = 0x0, n_packets_to_capture = 0, packet_type = PCAP_PACKET_TYPE_null, n_packets_captured = 0, flags = 0, file_descriptor = 0, n_pcap_data_written = 0, pcap_data = 0x0, packets_read = 0x0, timestamps = 0x0, min_packet_bytes = 0, max_packet_bytes = 0}, filter_classify_table_index = 0}, trace_filter = { trace_filter_enable = 0 '\000', trace_classify_table_index = 0, trace_filter_set_index = 0}


- looks like the way i tried calling `upf_init()` was not correct please check #271 

Kindly suggest to overcome this issue.

Regards,
Yaqub.
sergeymatov commented 2 years ago

Please check out how does VPP perform it's startup. UPG is a plugin and it's init stage following after VPP internal modules startup and other plugins.