ModemManager: crash on start

[CDuke]

Maintainer: @nickberry17 Environment: openwrt-21.02-snapshot-r15934-112d1052ac-ramips-mt7621-mikrotik_routerboard-m33g; ModemManager 1.14.10-1

Description: Modem manager crash on start

daemon.debug [6534]: [/dev/cdc-wdm0] Read max control message size from descriptors file: 4096
Sun Mar 28 14:56:58 2021 daemon.warn [6534]: <warn>  [modem0/sim0] couldn't load list of emergency numbers: Failed to parse CRSM query result '+CRSM: 148,8,""'
Sun Mar 28 14:56:59 2021 kern.info kernel: [   84.618907] do_page_fault(): sending SIGSEGV to ModemManager for invalid read access from 00000000
Sun Mar 28 14:56:59 2021 kern.info kernel: [   84.627863] epc = 00000000 in ModemManager[400000+eb000]
Sun Mar 28 14:56:59 2021 kern.info kernel: [   84.633211] ra  = 777483b5 in libmm-shared-foxconn.so[77747000+12000]
Sun Mar 28 14:56:59 2021 daemon.info procd: Instance modemmanager::instance1 s in a crash loop 6 crashes, 7 seconds since last crash

system.log kernel.log

[BKPepe]

Hi there,

I backported a few updates related to ModemManager and libqmi from OpenWrt master branch to OpenWrt 21.02 branch. Let me know, if that helps or not.

[CDuke]

No, it doesn't work Environment: openwrt-21.02-snapshot-r15952-6090337679-ramips-mt7621-mikrotik_routerboard-m33g; ModemManager 1.16.2-1

Mon Apr  5 19:25:58 2021 daemon.debug [6528]: opening device...
Mon Apr  5 19:25:58 2021 daemon.debug [6528]: [/dev/cdc-wdm0] Read max control message size from descriptors file: 4096
Mon Apr  5 19:25:58 2021 daemon.debug [6528]: [/dev/cdc-wdm0] channel destroyed
Mon Apr  5 19:25:59 2021 daemon.info [6528]: <info>  [device /sys/devices/platform/1e1c0000.xhci/usb2/2-1] creating modem with plugin 'dell' and '6' ports
Mon Apr  5 19:25:59 2021 daemon.info [6528]: <info>  [base-manager] modem for device '/sys/devices/platform/1e1c0000.xhci/usb2/2-1' successfully created
Mon Apr  5 19:25:59 2021 daemon.debug [6528]: opening device...
Mon Apr  5 19:25:59 2021 daemon.debug [6528]: [/dev/cdc-wdm0] Read max control message size from descriptors file: 4096
Mon Apr  5 19:26:00 2021 daemon.warn [6528]: <warn>  [modem0/sim0] couldn't load list of emergency numbers: Failed to parse CRSM query result '+CRSM: 148,8,""'
Mon Apr  5 19:26:00 2021 kern.info kernel: [   89.105219] do_page_fault(): sending SIGSEGV to ModemManager for invalid read access from 00000000
Mon Apr  5 19:26:00 2021 kern.info kernel: [   89.114208] epc = 00000000 in ModemManager[400000+f3000]
Mon Apr  5 19:26:00 2021 kern.info kernel: [   89.119566] ra  = 776943b5 in libmm-shared-foxconn.so[77693000+12000]
Mon Apr  5 19:26:00 2021 daemon.info procd: Instance modemmanager::instance1 s in a crash loop 6 crashes, 7 seconds since last crash 

system.log

[BKPepe]

Let's ping maintainers of ModemManager: @nickberry17 and @aleksander0m

[aleksander0m]

Can you get a debug log by running it manually with /usr/sbin/ModemManager --debug ?

[BKPepe]

Closing for now. Once you provide more details, you can re-open it.

[aleksander0m]

I'm truly interested in fixing this, we're only one debug log away! I've opened this issue in upstream https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/issues/362

Please either reopen here with the debug log, or otherwise push the log to the upstream bugreport.

Also, what device is this?

[aleksander0m]

@CDuke could you please help debug the issue further? thanks!

[CDuke]

Router - Mikrotik RBM33G Modem - Dell DW5821e

"Can you get a debug log by running it manually with /usr/sbin/ModemManager --debug ?" - modemmanager_debug.log

Very strange, when run ModemManager "manual threre is no crashing, but modems list is empty (mmcli -L)

[aleksander0m]

@CDuke not really strange, the problem is that the events adding new ports are emitted via hotplug scripts. Could you edit the MM daemon init file to run with --debug (so that logs go to syslog/logread) and then reboot? Or, if the modem is connected via USB, just unplug and replug?

[CDuke]

Sat Apr  3 16:13:16 2021 daemon.debug [6358]: <debug> [1617466396.497727] [modem0/sim0] loading emergency numbers...
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: <debug> [1617466396.497974] [modem0/ttyUSB0/at] device open count is 2 (open)
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: <debug> [1617466396.498397] [modem0/ttyUSB0/at] --> 'AT+CRSM=176,28599,0,0,15<CR>'
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: <debug> [1617466396.595586] [modem0/ttyUSB0/at] <-- '<CR><LF>+CRSM: 148,8,""<CR><LF><CR><LF>OK<CR><LF>'
Sat Apr  3 16:13:16 2021 daemon.warn [6358]: <warn>  [1617466396.596291] [modem0/sim0] couldn't load list of emergency numbers: Failed to parse CRSM query result '+CRSM: 148,8,""'
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Sent message... <<<<<< RAW: <<<<<<   length = 48 <<<<<<   data   = 03:00:00:00:30:00:00:00:0D:00:00:00:01:00:00:00:00:00:00:00:A2:89:CC:33:BC:BB:8B:4F:B6:B0:13:3E:C2:AA:E6:DF:02:00:00:00:00:00:00:00:00:00:00:00
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Sent message (translated)... <<<<<< Header: <<<<<<   length      = 48 <<<<<<   type        = command (0x00000003) <<<<<<   transaction = 13 <<<<<< Fragment header: <<<<<<   total   = 1 <<<<<<   current = 0 <<<<<< Contents: <<<<<<   service = 'basic-connect' (a289cc33-bcbb-8b4f-b6b0-133ec2aae6df) <<<<<<   cid     = 'subscriber-ready-status' (0x00000002) <<<<<<   type    = 'query' (0x00000000)
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: <debug> [1617466396.597800] [modem0/ttyUSB0/at] device open count is 1 (close)
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Received message... >>>>>> RAW: >>>>>>   length = 148 >>>>>>   data   = 03:00:00:80:94:00:00:00:0D:00:00:00:01:00:00:00:00:00:00:00:A2:89:CC:33:BC:BB:8B:4F:B6:B0:13:3E:C2:AA:E6:DF:02:00:00:00:00:00:00:00:64:00:00:00:01:00:00:00:1C:00:00:00:1E:00:00:00:3C:00:00:00:28:00:00:00:00:00:00:00:00:00:00:00:32:00:35:00:30:00:31:00:31:00:30:00:31:00:30:00:34:00:38:00:32:00:36:00:36:00:37:00:36:00:00:00:38:00:39:00:37:00:31:00:31:00:30:00:30:00:30:00:30:00:31:00:30:00:34:00:38:00:32:00:36:00:36:00:37:00:36:00:32:00:46:00
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Received message (translated)... >>>>>> Header: >>>>>>   length      = 148 >>>>>>   type        = command-done (0x80000003) >>>>>>   transaction = 13 >>>>>> Fragment header: >>>>>>   total   = 1 >>>>>>   current = 0 >>>>>> Contents: >>>>>>   status error = 'None' (0x00000000) >>>>>>   service      = 'basic-connect' (a289cc33-bcbb-8b4f-b6b0-133ec2aae6df) >>>>>>   cid          = 'subscriber-ready-status' (0x00000002) >>>>>> Fields: >>>>>>   ReadyState = 'initialized' >>>>>>   SubscriberId = '250110104826676' >>>>>>   SimIccId = '8971100001048266762F' >>>>>>   ReadyInfo = 'none' >>>>>>   TelephoneNumbersCount = '0' >>>>>>   TelephoneNumbers = ''
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Sent message... <<<<<< RAW: <<<<<<   length = 48 <<<<<<   data   = 03:00:00:00:30:00:00:00:0E:00:00:00:01:00:00:00:00:00:00:00:A2:89:CC:33:BC:BB:8B:4F:B6:B0:13:3E:C2:AA:E6:DF:05:00:00:00:00:00:00:00:00:00:00:00
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Sent message (translated)... <<<<<< Header: <<<<<<   length      = 48 <<<<<<   type        = command (0x00000003) <<<<<<   transaction = 14 <<<<<< Fragment header: <<<<<<   total   = 1 <<<<<<   current = 0 <<<<<< Contents: <<<<<<   service = 'basic-connect' (a289cc33-bcbb-8b4f-b6b0-133ec2aae6df) <<<<<<   cid     = 'pin-list' (0x00000005) <<<<<<   type    = 'query' (0x00000000)
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Received message... >>>>>> RAW: >>>>>>   length = 208 >>>>>>   data   = 03:00:00:80:D0:00:00:00:0E:00:00:00:01:00:00:00:00:00:00:00:A2:89:CC:33:BC:BB:8B:4F:B6:B0:13:3E:C2:AA:E6:DF:05:00:00:00:00:00:00:00:A0:00:00:00:02:00:00:00:01:00:00:00:04:00:00:00:08:00:00:00:01:00:00:00:01:00:00:00:04:00:00:00:08:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Received message (translated)... >>>>>> Header: >>>>>>   length      = 208 >>>>>>   type        = command-done (0x80000003) >>>>>>   transaction = 14 >>>>>> Fragment header: >>>>>>   total   = 1 >>>>>>   current = 0 >>>>>> Contents: >>>>>>   status error = 'None' (0x00000000) >>>>>>   service      = 'basic-connect' (a289cc33-bcbb-8b4f-b6b0-133ec2aae6df) >>>>>>   cid          = 'pin-list' (0x00000005) >>>>>> Fields: >>>>>>   PinDescPin1 = { >>>>>>       PinMode = '2' >>>>>>       PinFormat = '1' >>>>>>       PinLengthMin = '4' >>>>>>       PinLengthMax = '8' >>>>>>   }  >>>>>>   PinDescPin2 = { >>>>>>       PinMode = '1' >>>>>>       PinFormat = '1' >>>>>>       PinLengthMin = '4' >>>>>>       PinLengthMax = '8' >>>>>>   }  >>>>>>   PinDescDeviceSimPin = { >>>>>>       PinMode = '0' >>>>>>       PinFormat = '0' >>>>>>       PinLengthMin = '0' >>>>>>       PinLengthMax = '0' >>>>>>   }  >>>>>>   PinDescDeviceFirstSimPin = { >>>>>>       PinMode = '0' >>>>>>       Pin
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: <debug> [1617466396.691145] [modem0] loading UE mode of operation for EPS...
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: <debug> [1617466396.691332] [modem0/ttyUSB0/at] device open count is 2 (open)
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: <debug> [1617466396.691745] [modem0/ttyUSB0/at] --> 'AT+CEMODE?<CR>'
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: <debug> [1617466396.731650] [modem0/ttyUSB0/at] <-- '<CR><LF>+CEMODE: 2<CR><LF><CR><LF>OK<CR><LF>'
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Sent message... <<<<<< RAW: <<<<<<   length = 48 <<<<<<   data   = 03:00:00:00:30:00:00:00:0F:00:00:00:01:00:00:00:00:00:00:00:3D:01:DC:C5:FE:F5:4D:05:0D:3A:BE:F7:05:8E:9A:AF:03:00:00:00:00:00:00:00:00:00:00:00
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Sent message (translated)... <<<<<< Header: <<<<<<   length      = 48 <<<<<<   type        = command (0x00000003) <<<<<<   transaction = 15 <<<<<< Fragment header: <<<<<<   total   = 1 <<<<<<   current = 0 <<<<<< Contents: <<<<<<   service = 'ms-basic-connect-extensions' (3d01dcc5-fef5-4d05-0d3a-bef7058e9aaf) <<<<<<   cid     = 'lte-attach-configuration' (0x00000003) <<<<<<   type    = 'query' (0x00000000)
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: <debug> [1617466396.732961] [modem0/ttyUSB0/at] device open count is 1 (close)
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Received message... >>>>>> RAW: >>>>>>   length = 232 >>>>>>   data   = 03:00:00:80:E8:00:00:00:0F:00:00:00:01:00:00:00:00:00:00:00:3D:01:DC:C5:FE:F5:4D:05:0D:3A:BE:F7:05:8E:9A:AF:03:00:00:00:00:00:00:00:B8:00:00:00:03:00:00:00:1C:00:00:00:2C:00:00:00:48:00:00:00:38:00:00:00:80:00:00:00:38:00:00:00:00:00:00:00:00:00:00:00:01:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:01:00:00:00:03:00:00:00:2C:00:00:00:0A:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:44:00:55:00:4D:00:4D:00:59:00:00:00:00:00:00:00:02:00:00:00:03:00:00:00:2C:00:00:00:0A:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:44:00:55:00:4D:00:4D:00:59:00:00:00
Sat Apr  3 16:13:16 2021 daemon.debug [6358]: [/dev/cdc-wdm0] Received message (translated)... >>>>>> Header: >>>>>>   length      = 232 >>>>>>   type        = command-done (0x80000003) >>>>>>   transaction = 15 >>>>>> Fragment header: >>>>>>   total   = 1 >>>>>>   current = 0 >>>>>> Contents: >>>>>>   status error = 'None' (0x00000000) >>>>>>   service      = 'ms-basic-connect-extensions' (3d01dcc5-fef5-4d05-0d3a-bef7058e9aaf) >>>>>>   cid          = 'lte-attach-configuration' (0x00000003) >>>>>> Fields: >>>>>>   ConfigurationCount = '3' >>>>>>   Configurations = '{ >>>>>>     [0] = { >>>>>>           IpType = '0' >>>>>>           Roaming = '0' >>>>>>           Source = '1' >>>>>>           AccessString = '(null)' >>>>>>           UserName = '(null)' >>>>>>           Password = '(null)' >>>>>>           Compression = '0' >>>>>>           AuthProtocol = '0' >>>>>>     }, >>>>>>     [1] = { >>>>>>           IpType = '0' >>>>>>           Roaming = '1' >>>>>>           Source = '3' >>>>>>           AccessString = 'DUMMY' >>>>>>
Sat Apr  3 16:13:16 2021 kern.info kernel: [   88.659975] do_page_fault(): sending SIGSEGV to ModemManager for invalid read access from 00000000
Sat Apr  3 16:13:16 2021 kern.info kernel: [   88.668932] epc = 00000000 in ModemManager[400000+f3000]
Sat Apr  3 16:13:16 2021 kern.info kernel: [   88.674287] ra  = 776963b5 in libmm-shared-foxconn.so[77695000+12000]
Sat Apr  3 16:13:16 2021 daemon.info procd: Instance modemmanager::instance1 s in a crash loop 6 crashes, 7 seconds since last crash

modemmanager_daemon_debug.log

[aleksander0m]

Could you run the following to see if that also crashes?

$ sudo mbimcli -d /dev/cdc-wdm0 -p --ms-query-lte-attach-configuration --verbose

[aleksander0m]

Is this a custom compiled openwrt image? Did you build all libqmi, libmbim and ModemManager? Or did you skip building libqmi completely? If you didn't build ModemManager with QMI support, I have an explanation for the crash.

[CDuke]

mbimcli -d /dev/cdc-wdm0 -p --ms-query-lte-attach-configuration --verbose mbim_cli.log

"Is this a custom compiled openwrt image?" - no. I took from https://downloads.openwrt.org/releases/21.02-SNAPSHOT/targets/ramips/mt7621/ I have to install addtitional packages (follow this instruction https://openwrt.org/docs/guide-user/network/wan/wwan/modemmanager)

[aleksander0m]

Ok, so building ModemManager without QMI support triggers the issue in the exact same place, and confirms my assumptions detailed here https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/issues/362#note_902656

ModemManager[31902]: [/dev/cdc-wdm1] Sent message...
<<<<<< RAW:
<<<<<<   length = 48
<<<<<<   data   = 03:00:00:00:30:00:00:00:0F:00:00:00:01:00:00:00:00:00:00:00:3D:01:DC:C5:FE:F5:4D:05:0D:3A:BE:F7:05:8E:9A:AF:03:00:00:00:00:00:00:00:00:00:00:00
ModemManager[31902]: [/dev/cdc-wdm1] Sent message (translated)...
<<<<<< Header:
<<<<<<   length      = 48
<<<<<<   type        = command (0x00000003)
<<<<<<   transaction = 15
<<<<<< Fragment header:
<<<<<<   total   = 1
<<<<<<   current = 0
<<<<<< Contents:
<<<<<<   service = 'ms-basic-connect-extensions' (3d01dcc5-fef5-4d05-0d3a-bef7058e9aaf)
<<<<<<   cid     = 'lte-attach-configuration' (0x00000003)
<<<<<<   type    = 'query' (0x00000000)
ModemManager[31902]: <debug> [1620071887.869906] [modem1/ttyUSB0/at] device open count is 1 (close)
ModemManager[31902]: [/dev/cdc-wdm1] Received message...
>>>>>> RAW:
>>>>>>   length = 208
>>>>>>   data   = 03:00:00:80:D0:00:00:00:0F:00:00:00:01:00:00:00:00:00:00:00:3D:01:DC:C5:FE:F5:4D:05:0D:3A:BE:F7:05:8E:9A:AF:03:00:00:00:00:00:00:00:A0:00:00:00:03:00:00:00:1C:00:00:00:2C:00:00:00:48:00:00:00:2C:00:00:00:74:00:00:00:2C:00:00:00:03:00:00:00:00:00:00:00:03:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:01:00:00:00:03:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:02:00:00:00:03:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
ModemManager[31902]: [/dev/cdc-wdm1] Received message (translated)...
>>>>>> Header:
>>>>>>   length      = 208
>>>>>>   type        = command-done (0x80000003)
>>>>>>   transaction = 15
>>>>>> Fragment header:
>>>>>>   total   = 1
>>>>>>   current = 0
>>>>>> Contents:
>>>>>>   status error = 'None' (0x00000000)
>>>>>>   service      = 'ms-basic-connect-extensions' (3d01dcc5-fef5-4d05-0d3a-bef7058e9aaf)
>>>>>>   cid          = 'lte-attach-configuration' (0x00000003)
>>>>>> Fields:
>>>>>>   ConfigurationCount = '3'
>>>>>>   Configurations = '{
>>>>>>     [0] = {
>>>>>>           IpType = '3'
>>>>>>           Roaming = '0'
>>>>>>           Source = '3'
>>>>>>           AccessString = '(null)'
>>>>>>           UserName = '(null)'
>>>>>>           Password = '(null)'
>>>>>>           Compression = '0'
>>>>>>           AuthProtocol = '0'
>>>>>>     },
>>>>>>     [1] = {
>>>>>>           IpType = '0'
>>>>>>           Roaming = '1'
>>>>>>           Source = '3'
>>>>>>           AccessString = '(null)'
>>>>>>           UserName = '(null)'
>>>>>>           Password = '(null)'
>>>>>>           Compression = '0'
>>>>>>           AuthProtocol = '0'
>>>>>>     },
>>>>>>     [2] = {
>>>>>>           IpType = '0'
>>>>>>           Roaming = '2'
>>>>>>           Source = '3'
>>>>>>           AccessString = '(null)'
>>>>>>           UserName = '(null)'
>>>>>>           Password = '(null)'
>>>>>>           Compression = '0'
>>>>>>           AuthProtocol = '0'
>>>>>>     },
>>>>>>   }'

Thread 1 "ModemManager" received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in  ()
#1  0x00007ffff7f9a6a6 in location_load_capabilities (self=0x5555557ce4f0, callback=0x5555555db0dc <load_capabilities_ready>, user_data=0x7fffe8011ed0)
    at foxconn/mm-broadband-modem-mbim-foxconn.c:228
#2  0x00005555555db310 in interface_initialization_step (task=0x7fffe8011ed0) at mm-iface-modem-location.c:1813
#3  0x00005555555db8ac in mm_iface_modem_location_initialize
    (self=0x5555557ce4f0, cancellable=0x5555557b7660, callback=0x555555608c13 <iface_modem_location_initialize_ready>, user_data=0x55555579e020) at mm-iface-modem-location.c:1980
#4  0x00005555556098cf in initialize_step (task=0x55555579e020) at mm-broadband-modem.c:12243
#5  0x0000555555608a95 in iface_modem_3gpp_ussd_initialize_ready (self=0x5555557ce4f0, result=0x55555579dd50, task=0x55555579e020) at mm-broadband-modem.c:12132
#6  0x00007ffff7ca73d4 in  () at /usr/lib/libgio-2.0.so.0
#7  0x00007ffff7ca7409 in  () at /usr/lib/libgio-2.0.so.0
#8  0x00007ffff7ac9f30 in g_main_context_dispatch () at /usr/lib/libglib-2.0.so.0
#9  0x00007ffff7b1db59 in  () at /usr/lib/libglib-2.0.so.0
#10 0x00007ffff7ac9593 in g_main_loop_run () at /usr/lib/libglib-2.0.so.0
#11 0x0000555555586444 in main (argc=2, argv=0x7fffffffead8) at main.c:213

The fix in https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/merge_requests/510 solves the crash, I will have to backport it to the 1.16 branch manually, as other things changed in that plugin.

But, I think it should also be cleared here why this crashed in this way. This crash can only happen if:

• ModemManager is not built with QMI support (i.e --without-qmi)
• Or, if ModemManager is built with QMI support but QMI over MBIM wasn't enabled (with --enable-mbim-qmux).

I would bet the 2nd assumption is the one that got us here. How can we confirm which config options were used to build the images you used? Are the build logs for those images available somewhere?

[CDuke]

“ which config options were used to build the images you used?” - image of ModemManager or Openwrt?

[aleksander0m]

“ which config options were used to build the images you used?” - image of ModemManager or Openwrt?

We need to know which config options were used to build the ModemManager package and the libqmi package; or the debug build log if that exists

[CDuke]

@BKPepe, @nickberry17 Can you help answer?

[john-tho]

I cannot find verbose / debug build logs for the buildbot images / packages, or the package config files.

• https://buildbot.openwrt.org/openwrt-21.02/images/#/builders/61
• https://buildbot.openwrt.org/openwrt-21.02/packages/#/builders/25

In OpenWrt it looks like with default options, they are configured with:

• libqmi: --enable-mbim-qmux
• modemmanager: --with-mbim --with-qmi

test build for a device in the same mt7621 family wth mm on master:

libmbim configure:

./configure --target=mipsel-openwrt-linux --host=mipsel-openwrt-linux --build=x86_64-pc-linux-gnu --program-prefix="" --program-suffix="" --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/lib --sysconfdir=/etc --datadir=/usr/share --localstatedir=/var --mandir=/usr/man --infodir=/usr/info --disable-nls  --disable-static --disable-gtk-doc --disable-gtk-doc-html --disable-gtk-doc-pdf --disable-silent-rules --enable-more-warnings=yes --without-udev --without-udev-base-dir ;

libqmi configure:

./configure --target=mipsel-openwrt-linux --host=mipsel-openwrt-linux --build=x86_64-pc-linux-gnu --program-prefix="" --program-suffix="" --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/lib --sysconfdir=/etc --datadir=/usr/share --localstatedir=/var --mandir=/usr/man --infodir=/usr/info --disable-nls  --disable-static --disable-gtk-doc --disable-gtk-doc-html --disable-gtk-doc-pdf --disable-silent-rules --enable-firmware-update --enable-mbim-qmux --enable-more-warnings=yes --without-udev --without-udev-base-dir ;

modemmanager configure:

./configure --target=mipsel-openwrt-linux --host=mipsel-openwrt-linux --build=x86_64-pc-linux-gnu --program-prefix="" --program-suffix="" --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/lib --sysconfdir=/etc --datadir=/usr/share --localstatedir=/var --mandir=/usr/man --infodir=/usr/info --disable-nls  --without-polkit --without-udev --without-systemdsystemunitdir --disable-rpath --disable-gtk-doc --with-mbim --with-qmi ;

minimal build:

cat <<EOF>diffconfig_hexs_mm
CONFIG_TARGET_ramips=y
CONFIG_TARGET_ramips_mt7621=y
CONFIG_TARGET_ramips_mt7621_DEVICE_mikrotik_routerboard-760igs=y
CONFIG_PACKAGE_modemmanager=y
EOF
target=diffconfig_hexs_mm; cp $target .config; make defconfig; ./scripts/feeds {update,install} -a; cp $target .config; make defconfig; make clean; make -j1 V=s 2>&1 | tee log

Current package versions in packages-21.02:

• libmbim: 1.24.6-1
• libqmi: 1.28.2-1
• modemmanager: 1.16.2-1

From: https://downloads.openwrt.org/releases/packages-21.02/mips_24kc/packages/Packages.manifest

Given the 21.02 modemmanager depends on both libqmi & libmbim, it looks like the packages are built with default config options?

Package versions details

``` Package: libmbim Version: 1.24.6-1 Depends: libc, glib2 Source: feeds/packages/libs/libmbim SourceName: libmbim License: LGPL-2.0-or-later LicenseFiles: COPYING.LIB Section: libs SourceDateEpoch: 1611543923 Maintainer: Nicholas Smith Architecture: mips_24kc Installed-Size: 86689 Filename: libmbim_1.24.6-1_mips_24kc.ipk Size: 87187 SHA256sum: 25dd5080cc469576cdd7299ea7286ecb116e5d9aafa2771edc53ca4cfc302aed Description: Helper library to talk to MBIM enabled modems. Add mbim-utils for extra utilities. ``` ``` Package: libqmi Version: 1.28.2-1 Depends: libc, glib2, libmbim, libqrtr-glib Source: feeds/packages/libs/libqmi SourceName: libqmi License: LGPL-2.0-or-later LicenseFiles: COPYING.LIB Section: libs SourceDateEpoch: 1616969356 Maintainer: Nicholas Smith Architecture: mips_24kc Installed-Size: 505568 Filename: libqmi_1.28.2-1_mips_24kc.ipk Size: 499887 SHA256sum: 7c14c6bd4c30d97d84b8f12bb9774e563502275f8f53ce299a2c84cd03e0460d Description: Helper library talk to QMI enabled modems. Add qmi-utils for extra utilities. ``` ``` Package: modemmanager Version: 1.16.2-1 Depends: libc, glib2, dbus, ppp, libmbim, libqmi Source: feeds/packages/net/modemmanager SourceName: modemmanager License: GPL-2.0-or-later LicenseFiles: COPYING Section: net SourceDateEpoch: 1616969280 Maintainer: Nicholas Smith Architecture: mips_24kc Installed-Size: 915581 Filename: modemmanager_1.16.2-1_mips_24kc.ipk Size: 904140 SHA256sum: d6c186913cbc2088b567f7be1586a35d15d04d523386f364d31d79bbb798c337 Description: ModemManager is a D-Bus-activated service which allows controlling mobile broadband modems. Add kernel modules for your modems as needed. Select Utilities/usb-modeswitch if needed. ```

Defaults:

modemmanager: https://github.com/openwrt/packages/tree/master/net/modemmanager

CONFIGURE_ARGS += \
    --without-polkit \
    --without-udev \
    --without-systemdsystemunitdir \
    --disable-rpath \
    --disable-gtk-doc

# config.in defaults:
  CONFIGURE_ARGS += --with-mbim
  CONFIGURE_ARGS += --with-qmi

libqmi: https://github.com/openwrt/packages/tree/master/libs/libqmi both LIBQMI_WITH default y

CONFIGURE_ARGS += \
    --disable-static \
    --disable-gtk-doc \
    --disable-gtk-doc-html \
    --disable-gtk-doc-pdf \
    --disable-silent-rules \
    --enable-firmware-update \
    --$(if $(LIBQMI_WITH_MBIM_QMUX),en,dis)able-mbim-qmux \
    --$(if $(LIBQMI_WITH_QRTR_GLIB),en,dis)able-qrtr \
    --enable-more-warnings=yes \
    --without-udev \
    --without-udev-base-dir

[feckert]

Closing for now. No porgress since 2 years and owrt-21.02 is EOL. Also a change was made to the ModemManager to address this issue.