search

magma / magma

Platform for building access networks and modular network services
https://www.magmacore.org/
Other
1.69k stars 609 forks source link

[agw][lte] DevAssert on nb_ue_associated during UE removal #5193

Closed ulaskozat closed 3 years ago

ulaskozat commented 3 years ago

Your Environment

Describe the Issue

Issue arises during mme restarts because of an earlier failure. nb_ue_associated is decremented for an eNB during S1AP initialization because of UE that did not have yet an IMSI (but UE itself is not removed) as well as when the UE is actually removed (e.g., during timer recovery event).

Expected behavior Should not fail at assertion check.

Additional context

Core was generated by `/usr/local/bin/mme -c /var/opt/magma/tmp/mme.conf -s /var/opt/magma/tmp/spgw.co'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00005610230dbeab in s1ap_remove_ue (state=state@entry=0x60f00000b4f0, ue_ref=ue_ref@entry=0x603000047820)
    at /home/vagrant/magma/lte/gateway/c/oai/tasks/s1ap/s1ap_mme.c:519
519 /home/vagrant/magma/lte/gateway/c/oai/tasks/s1ap/s1ap_mme.c: No such file or directory.
[Current thread is 1 (Thread 0x7f38befd7700 (LWP 21624))]
(gdb) bt
#0  0x00005610230dbeab in s1ap_remove_ue (state=state@entry=0x60f00000b4f0, ue_ref=ue_ref@entry=0x603000047820)
    at /home/vagrant/magma/lte/gateway/c/oai/tasks/s1ap/s1ap_mme.c:519
#1  0x00005610231267da in s1ap_mme_release_ue_context (state=state@entry=0x60f00000b4f0, ue_ref_p=ue_ref_p@entry=0x603000047820, imsi64=imsi64@entry=1011234565066)
    at /home/vagrant/magma/lte/gateway/c/oai/tasks/s1ap/s1ap_mme_handlers.c:2493
#2  0x0000561023126ad5 in s1ap_mme_generate_ue_context_release_command (state=state@entry=0x60f00000b4f0, ue_ref_p=0x603000047820, cause=<optimized out>, 
    imsi64=imsi64@entry=1011234565066, assoc_id=<optimized out>, stream=<optimized out>, mme_ue_s1ap_id=8134, enb_ue_s1ap_id=1241)
    at /home/vagrant/magma/lte/gateway/c/oai/tasks/s1ap/s1ap_mme_handlers.c:1287
#3  0x0000561023126b49 in s1ap_handle_ue_context_release_command (state=state@entry=0x60f00000b4f0, 
    ue_context_release_command_pP=ue_context_release_command_pP@entry=0x629000118026, imsi64=imsi64@entry=1011234565066)
    at /home/vagrant/magma/lte/gateway/c/oai/tasks/s1ap/s1ap_mme_handlers.c:1413
#4  0x00005610230db5fe in handle_message (loop=<optimized out>, reader=<optimized out>, arg=<optimized out>)
    at /home/vagrant/magma/lte/gateway/c/oai/tasks/s1ap/s1ap_mme.c:191
#5  0x00007f38d296d7be in zloop_start () from /usr/lib/x86_64-linux-gnu/libczmq.so.4
#6  0x00005610230db281 in s1ap_mme_thread (args=<optimized out>) at /home/vagrant/magma/lte/gateway/c/oai/tasks/s1ap/s1ap_mme.c:319
#7  0x00007f38d514c4a4 in start_thread (arg=0x7f38befd7700) at pthread_create.c:456
#8  0x00007f38d14c0d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
(gdb) f 0
#0  0x00005610230dbeab in s1ap_remove_ue (state=state@entry=0x60f00000b4f0, ue_ref=ue_ref@entry=0x603000047820)
    at /home/vagrant/magma/lte/gateway/c/oai/tasks/s1ap/s1ap_mme.c:519
519 in /home/vagrant/magma/lte/gateway/c/oai/tasks/s1ap/s1ap_mme.c
(gdb) print enb_ref->nb_ue_associated
$1 = 0
(gdb) print *enb_ref
$2 = {s1_state = S1AP_READY, enb_name = '\000' <repeats 149 times>, enb_id = 5, default_paging_drx = 0 '\000', supported_ta_list = {list_count = 1 '\001', 
    supported_tai_items = {{tac = 1, bplmnlist_count = 1 '\001', bplmns = {{mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', mnc_digit3 = 15 '\017', mcc_digit3 = 1 '\001', 
            mnc_digit2 = 1 '\001', mnc_digit1 = 0 '\000'}, {mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', mnc_digit3 = 0 '\000', mcc_digit3 = 0 '\000', 
            mnc_digit2 = 0 '\000', mnc_digit1 = 0 '\000'}, {mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', mnc_digit3 = 0 '\000', mcc_digit3 = 0 '\000', 
            mnc_digit2 = 0 '\000', mnc_digit1 = 0 '\000'}, {mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', mnc_digit3 = 0 '\000', mcc_digit3 = 0 '\000', 
            mnc_digit2 = 0 '\000', mnc_digit1 = 0 '\000'}, {mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', mnc_digit3 = 0 '\000', mcc_digit3 = 0 '\000', 
            mnc_digit2 = 0 '\000', mnc_digit1 = 0 '\000'}, {mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', mnc_digit3 = 0 '\000', mcc_digit3 = 0 '\000', 
            mnc_digit2 = 0 '\000', mnc_digit1 = 0 '\000'}}}, {tac = 0, bplmnlist_count = 0 '\000', bplmns = {{mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', 
            mnc_digit3 = 0 '\000', mcc_digit3 = 0 '\000', mnc_digit2 = 0 '\000', mnc_digit1 = 0 '\000'}, {mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', 
            mnc_digit3 = 0 '\000', mcc_digit3 = 0 '\000', mnc_digit2 = 0 '\000', mnc_digit1 = 0 '\000'}, {mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', 
            mnc_digit3 = 0 '\000', mcc_digit3 = 0 '\000', mnc_digit2 = 0 '\000', mnc_digit1 = 0 '\000'}, {mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', 
            mnc_digit3 = 0 '\000', mcc_digit3 = 0 '\000', mnc_digit2 = 0 '\000', mnc_digit1 = 0 '\000'}, {mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', 
            mnc_digit3 = 0 '\000', mcc_digit3 = 0 '\000', mnc_digit2 = 0 '\000', mnc_digit1 = 0 '\000'}, {mcc_digit2 = 0 '\000', mcc_digit1 = 0 '\000', 
            mnc_digit3 = 0 '\000', mcc_digit3 = 0 '\000', mnc_digit2 = 0 '\000', mnc_digit1 = 0 '\000'}}} <repeats 15 times>}}, nb_ue_associated = 0, ue_id_coll = {
    mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, 
      __size = '\000' <repeats 39 times>, __align = 0}, size = 16, num_elements = 3, nodes = 0x608000009d00, lock_nodes = 0x61500000a280, 
    hashfunc = 0x56102355a9d6 <def_hashfunc>, name = 0x601000027630, is_allocated_by_malloc = false, log_enabled = true}, sctp_assoc_id = 611, next_sctp_stream = 1, 
  instreams = 2, outstreams = 2, ran_cp_ipaddr = "\n\026cI", '\000' <repeats 11 times>, ran_cp_ipaddr_sz = 4 '\004'}
ssanadhya commented 3 years ago

Fixed in https://github.com/magma/magma/pull/5279