Closed xsm1997 closed 2 years ago
And I get kernel panic when starting wifi. It seems that it is related to nss-ecm.
Kernel log:
[ 74.763143] br-lan: port 5(wlan0) entered blocking state
[ 74.768391] br-lan: port 5(wlan0) entered forwarding state
[ 76.555724] Unable to handle kernel read from unreadable memory at virtual address 0000000000000090
[ 76.555774] Mem abort info:
[ 76.563605] ESR = 0x96000005
[ 76.566355] EC = 0x25: DABT (current EL), IL = 32 bits
[ 76.569484] SET = 0, FnV = 0
[ 76.574953] EA = 0, S1PTW = 0
[ 76.577813] FSC = 0x05: level 1 translation fault
[ 76.580854] Data abort info:
[ 76.585717] ISV = 0, ISS = 0x00000005
[ 76.588838] CM = 0, WnR = 0
[ 76.592400] user pgtable: 4k pages, 39-bit VAs, pgdp=0000000044014000
[ 76.595529] [0000000000000090] pgd=0800000049ce0003, p4d=0800000049ce0003, pud=0800000049ce0003, pmd=0000000000000000
[ 76.601963] Internal error: Oops: 96000005 [#1] SMP
[ 76.612540] Modules linked in: ecm ath11k_ahb ath11k nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet mac80211 ipt_REJECT ebtable_nat ebtable_filter ebtable_broute cfg80211 xt_time xt_tcpudp xt_state xt_socket xt_recent xt_quota xt_pkttype xt_physdev xt_owner xt_nat xt_multiport xt_mark xt_mac xt_limit xt_iprange xt_helper xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_cgroup xt_addrtype xt_TPROXY xt_TCPMSS xt_REDIRECT xt_MASQUERADE xt_LOG xt_CT wireguard tcp_bbr qrtr_smd qrtr qmi_helpers ppp_async ns nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject_bridge nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_meta_bridge nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_ct nft_counter nft_compat nft_chain_nat nf_tproxy_ipv6 nf_tproxy_ipv4 nf_tables nf_socket_ipv6 nf_socket_ipv4 nf_reject_ipv4 nf_log_syslog nf_flow_table nf_conntrack_bridge nf_conncount libcrc32c libchacha20poly1305 iptable_raw iptable_nat iptable_mangle
[ 76.612733] iptable_filter ip_tables hwmon ebtables ebt_vlan ebt_stp ebt_redirect ebt_pkttype ebt_mark_m ebt_mark ebt_limit ebt_among ebt_802_3 crc_ccitt compat arptable_filter arpt_mangle arp_tables qca_nss_pppoe pppoe pppox ppp_generic slhc qca_nss_bridge_mgr qca_nss_vlan xt_set ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6table_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip6t_NPT ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 bonding ip6_udp_tunnel udp_tunnel qca_nss_drv tun zram zsmalloc seqiv michael_mic leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom qca_nss_dp qca_ssdk gpio_button_hotplug crc32c_generic
[ 76.762008] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.57 #0
[ 76.784244] Hardware name: Redmi AX6 (DT)
[ 76.790489] pstate: 40400005 (nZcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 76.794401] pc : ecm_nss_ipv4_exit+0x191c/0x2b00 [ecm]
[ 76.801167] lr : ecm_ported_ipv4_process+0x4b8/0x1210 [ecm]
[ 76.806376] sp : ffffffc008003150
[ 76.811842] x29: ffffffc008003170 x28: ffffff8003e0c300 x27: 0000000000000043
[ 76.815322] x26: 0000000000000044 x25: ffffff8009c06000 x24: ffffffc008003588
[ 76.822439] x23: 0000000000000000 x22: ffffff800459fe00 x21: 0000000000000001
[ 76.829558] x20: ffffff8003e0c300 x19: ffffff8003e0c300 x18: 0000000000000000
[ 76.836675] x17: 0000000100000000 x16: 0000000000007765 x15: 0000000000000002
[ 76.843794] x14: ffffffc0010ed200 x13: ffffffc0010ec5d0 x12: 0000000000000000
[ 76.850911] x11: ffffff8002ab0f00 x10: ffffff800459fb00 x9 : 00000000c8a5fbbd
[ 76.858029] x8 : ffffffc0080032b8 x7 : 000000000000003d x6 : ffffffc001104c58
[ 76.865147] x5 : 0000000000000000 x4 : ffffff80028fd100 x3 : 000000000000eb12
[ 76.872266] x2 : 0000000000000000 x1 : ffffffc0080035d0 x0 : 0000000000000000
[ 76.879384] Call trace:
[ 76.886492] ecm_nss_ipv4_exit+0x191c/0x2b00 [ecm]
[ 76.888753] ecm_ported_ipv4_process+0x4b8/0x1210 [ecm]
[ 76.893615] ecm_ipv4_ip_process+0x3c0/0xdd0 [ecm]
[ 76.898736] ecm_ipv4_ip_process+0xb8c/0xdd0 [ecm]
[ 76.903598] nf_hook_slow+0x4c/0xf0
[ 76.908370] br_forward_finish+0x5c/0xb4
[ 76.911755] __br_forward+0xd4/0x14c
[ 76.915921] br_forward+0x130/0x154
[ 76.919480] br_handle_frame_finish+0x358/0x4e4
[ 76.922694] br_handle_frame+0x42c/0x490
[ 76.927207] __netif_receive_skb_core.constprop.0+0x2e0/0xaa4
[ 76.931378] __netif_receive_skb_list_core+0xd8/0x1ec
[ 76.937019] netif_receive_skb_list_internal+0x1c0/0x2b0
[ 76.942054] napi_complete_done+0x68/0x1d0
[ 76.947433] edma_napi+0x2f8/0x454 [qca_nss_dp]
[ 76.951340] __napi_poll+0x38/0x180
[ 76.955764] net_rx_action+0xe8/0x274
[ 76.959237] _stext+0x124/0x29c
[ 76.963055] irq_exit+0xbc/0x100
[ 76.966007] handle_domain_irq+0x64/0x90
[ 76.969481] gic_handle_irq+0x74/0xa0
[ 76.973388] call_on_irq_stack+0x28/0x54
[ 76.976947] do_interrupt_handler+0x54/0x60
[ 76.980940] el1_interrupt+0x30/0x50
[ 76.984845] el1h_64_irq_handler+0x18/0x24
[ 76.988665] el1h_64_irq+0x78/0x7c
[ 76.992570] arch_cpu_idle+0x18/0x2c
[ 76.995956] default_idle_call+0x20/0x70
[ 76.999689] do_idle+0x214/0x240
[ 77.003595] cpu_startup_entry+0x28/0x60
[ 77.006807] rest_init+0xc8/0xe0
[ 77.010712] arch_call_rest_init+0x10/0x1c
[ 77.013925] start_kernel+0x6a8/0x6cc
[ 77.017832] __primary_switched+0xa0/0xa8
[ 77.021569] Code: f90043e3 7941e403 12001c40 f9004be1 (f94048a1)
[ 77.025565] ---[ end trace 8706a52c53c05cb9 ]---
[ 77.031638] Kernel panic - not syncing: Oops: Fatal exception in interrupt
[ 77.036328] SMP: stopping secondary CPUs
[ 77.043012] Kernel Offset: disabled
[ 77.047084] CPU features: 0x00000000,00000802
[ 77.050298] Memory Limit: none
[ 77.254832] Rebooting in 3 seconds..
same issue with qnap
【Pr branch】Indeed, after updating to the latest IPQ807x, it still can't be used, can't get IP, no wireless signal!
I tried the coolsnowwolflede build and it works ... I presume @bitthief there is something wrong with the nss clocks?
I tried the coolsnowwolflede build and it works ... I presume @bitthief there is something wrong with the nss clocks?
Yes, but coolsnowwolf/lede is based on 5.10 kernel and lack support of Xiaomi/Redmi devices for some reasons. Its nss-ecm also has some bugs related to ipv6 and will randomly kernel panic/reboot when using ipv6. By the way, the luci of lede is based on 18.06, and it is a modified version, not openwrt's mainstream.
Previously nss-ecm and nss-firmware in this repo can work perfectly after some hack (delay loading nss-ecm, etc.), but after updating things are broken now.
sure I know just wanted to test nss-ecm...a few hours of experience with this build I don't think nss-ecm improves anything compared with a build without ...
The latest source still can not get IP address also no wifi signal!
Hey guys,
Thanks a lot for testing this and sorry about the issues, I have been working hard trying to fix this and get the same stable builds that I had before, it's so frustrating. I had to add plenty of patches on top of what Robi and Ansuel use, and I literally went through ALL of the code that Qualcomm are publishing in their repos etc. for each module + their 5.4 kernel tree.
As of this morning, I stabilized most of the issues with qca-nss-drv etc., it doesn't seem to kernel panic anymore. I reverted Ansuel's DMA patches which were breaking it and downgraded the nss-firmware to 12.0. That and a few other patches fixed this. The NSS clocks were fine.
We only have 1 outstanding issue, we're back to the ECM notifier registration issues from the past:
[ 20.902096] xt_DNETMAP INIT successfull (version 2)
[ 20.976713] xt_time: kernel timezone is -0000
[ 21.095009] ath10k 5.15 driver, optimized for CT firmware, probing pci device: 0x50.
[ 21.095608] ath10k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[ 21.102445] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[ 22.699058] ath10k_pci 0000:01:00.0: qca9887 hw1.0 target 0x4100016d chip_id 0x004000ff sub 0000:0000
[ 22.699115] ath10k_pci 0000:01:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[ 22.707372] ath10k_pci 0000:01:00.0: firmware ver 10.1-ct-87-__fH-022-ecad3248 api 2 features wmi-10.x,mfp,txstatus-noack,wmi-10.x-CT,ratemask-CT,txrate-CT,get-temp-CT,tx-rc-CT,cust-stats-CT,retry-gt2-CT,txrate2-CT,beacon-cb-CT,wmi-block-ack-CT crc32 c0004f11
[ 22.741314] ath10k_pci 0000:01:00.0: Loading BDF type 0
[ 22.758081] ath10k_pci 0000:01:00.0: board_file api 2 bmi_id N/A crc32 b4ce36fa
[ 23.623136] ath10k_pci 0000:01:00.0: 10.1 wmi init: vdevs: 16 peers: 127 tid: 256
[ 23.630117] ath10k_pci 0000:01:00.0: wmi print 'P 128 V 8 T 410'
[ 23.630171] ath10k_pci 0000:01:00.0: wmi print 'msdu-desc: 1424 sw-crypt: 0 ct-sta: 0'
[ 23.635900] ath10k_pci 0000:01:00.0: wmi print 'alloc rem: 25576 iram: 25068'
[ 23.683309] ath10k_pci 0000:01:00.0: htt-ver 2.2 wmi-op 2 htt-op 2 cal file max-sta 128 raw 0 hwcrypto 1
[ 23.683667] ath10k_pci 0000:01:00.0: NOTE: Firmware DBGLOG output disabled in debug_mask: 0x10000000
[ 23.837316] ath11k c000000.wifi: ipq8074 hw2.0
[ 23.837640] remoteproc remoteproc0: powering up cd00000.q6v5_wcss
[ 23.840869] remoteproc remoteproc0: Booting fw image IPQ8074/q6_fw.mdt, size 668
[ 24.985257] remoteproc remoteproc0: remote processor cd00000.q6v5_wcss is now up
[ 24.986630] ath11k c000000.wifi: qmi ignore invalid mem req type 3
[ 24.987151] kmodloader: done loading kernel modules from /etc/modules.d/*
[ 24.992288] ath11k c000000.wifi: chip_id 0x0 chip_family 0x0 board_id 0x292 soc_id 0xffffffff
[ 25.004591] ath11k c000000.wifi: fw_version 0x250a04a5 fw_build_timestamp 2021-12-20 07:09 fw_build_id WLAN.HK.2.5.0.1-01208-QCAHKSWPL_SILICONZ-1
[ 26.298353] mtdblock: MTD device 'rootfs' is NAND, please consider using UBI block devices instead.
[ 26.393292] ip_local_port_range: prefer different parity for start/end values.
[ 26.789776] mtdblock: MTD device '0:wififw' is NAND, please consider using UBI block devices instead.
[ 26.794273] mtdblock: MTD device 'rsvd0' is NAND, please consider using UBI block devices instead.
[ 26.804809] mtdblock: MTD device 'overlay' is NAND, please consider using UBI block devices instead.
[ 26.809415] mtdblock: MTD device 'rootfs_1' is NAND, please consider using UBI block devices instead.
[ 26.824436] mtdblock: MTD device 'rootfs' is NAND, please consider using UBI block devices instead.
[ 26.827735] mtdblock: MTD device 'crash_syslog' is NAND, please consider using UBI block devices instead.
[ 26.840573] mtdblock: MTD device 'crash' is NAND, please consider using UBI block devices instead.
[ 26.847267] mtdblock: MTD device 'bdata' is NAND, please consider using UBI block devices instead.
[ 26.855125] mtdblock: MTD device '0:art' is NAND, please consider using UBI block devices instead.
[ 26.864406] mtdblock: MTD device '0:appsbl' is NAND, please consider using UBI block devices instead.
[ 26.878335] mtdblock: MTD device '0:appsblenv' is NAND, please consider using UBI block devices instead.
[ 26.882306] mtdblock: MTD device '0:cdt' is NAND, please consider using UBI block devices instead.
[ 26.892020] mtdblock: MTD device '0:rpm' is NAND, please consider using UBI block devices instead.
[ 26.900739] mtdblock: MTD device '0:devcfg' is NAND, please consider using UBI block devices instead.
[ 26.909767] mtdblock: MTD device '0:qsee' is NAND, please consider using UBI block devices instead.
[ 26.923357] mtdblock: MTD device '0:mibib' is NAND, please consider using UBI block devices instead.
[ 26.927729] mtdblock: MTD device '0:sbl1' is NAND, please consider using UBI block devices instead.
[ 34.726332] nss-dp 3a001200.dp2 eth0: Data plane vsi assign failed
[ 37.766332] nss-dp 3a001400.dp3 eth1: Data plane vsi assign failed
[ 40.806325] nss-dp 3a001600.dp4 eth2: Data plane vsi assign failed
[ 43.846333] nss-dp 3a001800.dp5 eth3: Data plane vsi assign failed
[ 44.837278] ath10k_pci 0000:01:00.0: 10.1 wmi init: vdevs: 16 peers: 127 tid: 256
[ 44.884708] ath10k_pci 0000:01:00.0: pdev param 0 not supported by firmware
[ 51.206333] nss-dp 3a001800.dp5 eth3: Data plane vsi assign failed
[ 57.286334] nss-dp 3a001600.dp4 eth2: Data plane vsi assign failed
[ 63.366335] nss-dp 3a001400.dp3 eth1: Data plane vsi assign failed
[ 69.446332] nss-dp 3a001200.dp2 eth0: Data plane vsi assign failed
[ 72.911281] ath10k_pci 0000:01:00.0: 10.1 wmi init: vdevs: 16 peers: 127 tid: 256
[ 72.958732] ath10k_pci 0000:01:00.0: pdev param 0 not supported by firmware
[ 78.407166] mtdblock: MTD device 'rootfs' is NAND, please consider using UBI block devices instead.
[ 80.121374] Unexpected kernel BRK exception at EL1
[ 80.121417] Internal error: BRK handler: f2000800 [#1] SMP
[ 80.125067] Modules linked in: ecm(+) ath11k_ahb ath11k ath10k_pci ath10k_core ath nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet mac80211 lz4 ipt_REJECT ebtable_nat ebtable_filter ebtable_broute cfg80211 xt_u32 xt_time xt_tcpudp xt_tcpmss xt_string xt_statistic xt_state xt_recent xt_quota xt_policy xt_pkttype xt_physdev xt_owner xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length2 xt_length xt_ipv4options xt_iprange xt_iface xt_hl xt_helper xt_hashlimit xt_esp xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connlabel xt_connbytes xt_condition xt_comment xt_cgroup xt_bpf xt_addrtype xt_TPROXY xt_TCPMSS xt_REDIRECT xt_PROTO xt_NFQUEUE xt_NFLOG xt_NETMAP xt_MASQUERADE xt_LOGMARK xt_LOG xt_IPMARK xt_HL xt_FLOWOFFLOAD xt_DSCP xt_DNETMAP xt_DHCPMAC xt_CT xt_CLASSIFY xt_CHECKSUM xt_ACCOUNT xfrm_interface wireguard vhci_hcd usbnet usbip_host usbip_core ums_usbat ums_sddr55 ums_sddr09 ums_karma ums_jumpshot ums_isd200 ums_freecom ums_datafab ums_cypress
[ 80.125294] ums_alauda uinput ts_fsm ts_bm tcp_scalable tcp_hybla tcp_bbr sch_mqprio sch_cake qrtr_tun qrtr_smd qrtr_mhi qrtr qmi_helpers pptp ppp_mppe ppp_async ns nlmon nft_tproxy nft_socket nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject_bridge nft_reject nft_redir nft_quota nft_queue nft_objref nft_numgen nft_nat nft_meta_bridge nft_masq nft_log nft_limit nft_hash nft_fwd_netdev nft_fullcone nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_dup_netdev nft_ct nft_counter nft_compat nft_chain_nat nfnetlink_queue nfnetlink_log nf_tproxy_ipv6 nf_tproxy_ipv4 nf_tables nf_socket_ipv6 nf_socket_ipv4 nf_reject_ipv4 nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_rtsp nf_nat_pptp nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_log_syslog nf_flow_table nf_dup_netdev nf_conntrack_tftp nf_conntrack_snmp nf_conntrack_sip nf_conntrack_rtsp nf_conntrack_pptp nf_conntrack_netlink nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack_broadcast nf_conntrack_bridge
[ 80.195198] ts_kmp nf_conntrack_amanda nf_conncount mhi_pci_generic mhi_net mhi mdio_gpio mdio_bitbang macvlan macremapper lz4_decompress lz4_compress libchacha20poly1305 l2tp_ppp ipvlan iptable_raw iptable_nat iptable_mangle iptable_filter ipt_rpfilter ipt_ah ipt_ECN ip6table_raw ip6t_rpfilter ip_tables hwmon hid_generic ebtables ebt_vlan ebt_stp ebt_snat ebt_redirect ebt_pkttype ebt_nflog ebt_mark_m ebt_mark ebt_log ebt_limit ebt_ip6 ebt_ip ebt_dnat ebt_arpreply ebt_arp ebt_among ebt_802_3 crc_ccitt compat_xtables compat cls_flower cdc_wdm can_dev can arptable_filter arpt_mangle arp_tables act_vlan fuse sch_teql sch_sfq sch_red sch_prio sch_multiq sch_gred sch_fq_pie sch_pie sch_fq sch_dsmark sch_codel em_text em_nbyte em_meta em_cmp act_simple act_police act_pedit act_ipt act_csum em_ipset cls_bpf act_bpf act_ctinfo act_connmark sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact hid evdev
[ 80.281789] qca_nss_crypto qca_nss_pppoe pppoe pppox trelay ledtrig_gpio cryptodev xt_set ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6table_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip6t_NPT ip6t_rt ip6t_mh ip6t_ipv6header ip6t_hbh ip6t_frag ip6t_eui64 ip6t_ah ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 pppoatm ppp_generic slhc bonding ip6_gre ip_gre gre ifb nat46 l2tp_ip6 l2tp_ip l2tp_eth ip6_vti ip_vti sit sctp libcrc32c qca_nss_drv l2tp_netlink l2tp_core ipcomp6 xfrm6_tunnel esp6 ah6 xfrm4_tunnel ipcomp esp4 ah4 ipip ip6_tunnel netlink_diag tunnel6 tunnel4 ip_tunnel veth tun ovpn_dco xfrm_user xfrm_ipcomp af_key xfrm_algo vfat fat udf crc_itu_t cdrom configfs autofs4 dns_resolver
[ 80.368922] br2684 atm raid10 raid1 raid0 md_mod nls_utf8 nls_iso8859_1 nls_cp437 zram zsmalloc vxlan udp_tunnel ip6_udp_tunnel macsec bfq wp512 twofish_generic twofish_common tea serpent_generic khazad cast6_generic cast5_generic cast_common camellia_generic blowfish_generic blowfish_common anubis xcbc crypto_user algif_skcipher algif_rng algif_hash algif_aead af_alg seqiv rmd160 pcbc michael_mic fcrypt echainiv des_generic cts arc4 usb_storage uleds leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom uhci_hcd input_leds input_core qca_nss_dp qca_ssdk gpio_button_hotplug f2fs ext4 mbcache jbd2 exfat dm_mirror dm_region_hash dm_log dm_crypt dm_mod dax mii crc32c_generic crc32_generic encrypted_keys trusted tpm rng_core asn1_encoder
[ 80.521060] CPU: 3 PID: 14509 Comm: modprobe Tainted: G B 5.15.59 #0
[ 80.543298] Hardware name: Xiaomi AX3600 (DT)
[ 80.550931] pstate: 40400005 (nZcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 80.555449] pc : nf_conntrack_register_notifier+0x58/0x60 [nf_conntrack]
[ 80.562221] lr : nf_conntrack_register_notifier+0x2c/0x60 [nf_conntrack]
[ 80.569165] sp : ffffffc00d65ba90
[ 80.575840] x29: ffffffc00d65ba90 x28: ffffffc0080daf10 x27: 0000000000000022
[ 80.579060] x26: 0000000000000100 x25: ffffffc001a95a00 x24: ffffffc001a95a50
[ 80.586179] x23: 0000000000000002 x22: 0000000000000000 x21: ffffffc001a93d98
[ 80.593297] x20: ffffffc008cca180 x19: ffffffc0010c7ae0 x18: 000000070000000a
[ 80.600414] x17: 0000000800000e10 x16: 000000090000012c x15: 0000000a00000708
[ 80.607532] x14: 00000000000001d5 x13: 7265696669746f6e x12: 5f6b636172746e6e
[ 80.614651] x11: 00000000000000c8 x10: 00000000000000c8 x9 : 0000000000000001
[ 80.621768] x8 : 00000000706f7473 x7 : 0000000000000004 x6 : 0000000000000000
[ 80.628887] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000000
[ 80.636005] x2 : ffffff80140c8000 x1 : 0000000000000000 x0 : ffffffc0014d3000
[ 80.643123] Call trace:
**[ 80.650232] nf_conntrack_register_notifier+0x58/0x60 [nf_conntrack]
[ 80.652497] ecm_conntrack_notifier_init+0x54/0x1ec [ecm]
[ 80.659093] init_module+0x164/0x1000 [ecm]
[ 80.664384] do_one_initcall+0x50/0x1b0
[ 80.668377] do_init_module+0x44/0x230
[ 80.672197] load_module+0x1e18/0x2580
[ 80.676015] __do_sys_init_module+0x24c/0x2b0
[ 80.679749] __arm64_sys_init_module+0x1c/0x30
[ 80.684176] invoke_syscall.constprop.0+0x5c/0x104
[ 80.688517] do_el0_svc+0x6c/0x15c
[ 80.693289] el0_svc+0x18/0x54
[ 80.696673] el0t_64_sync_handler+0xe8/0x114
[ 80.699715] el0t_64_sync+0x184/0x188**
[ 80.704145] Code: f94013f5 a8c37bfd d50323bf d65f03c0 (d4210000)
[ 80.707707] ---[ end trace b69636999b99b332 ]---
[ 80.713780] Kernel panic - not syncing: BRK handler: Fatal exception
[ 80.718470] SMP: stopping secondary CPUs
[ 80.724807] Kernel Offset: disabled
[ 80.728706] CPU features: 0x00000000,00000802
[ 80.731920] Memory Limit: none
[ 81.736541] Rebooting in 3 seconds..
I will investigate this further today, I know last time I semi-fixed it by adding a MUTEX before nf conntrack notifier registration and delaying the start-up of ECM a bit. I deleted those patches unfortunately and rebased on Ansuel's WIP patches, which brought the bugs back. I have a few ideas and I know how I worked around it last time, so I'm confident we can get a stable build again. The hard part is done :)
@rmandrad NSS ECM improves many things, performance-wise and stability. When it works :) And, for me, one of the main reasons to get ECM working is the fact that I have A TON of issues with SSL/TLS handshakes when using wired ports over certain speeds. And, ECM can accelerate a bunch of other things using the NSS cores, that's why we bought this device, to make use of its full hardware capabilities.
Glad to hearing that. Good job for solving almost all of the issues. Looking forward to a working nss-ecm.
Glad to hearing that.Has the only remaining problem been solved? I haven't been feeling well these days. Please understand!
thank you @bitthief can report your latest build is stable and not crashing anymore with the vsi assigned errors... note I am using it on the QNAP 301w
ECM starts alongside all of the other QCA drivers.
however I noticed if I look at the interrupts I don't see the nss_queue interrupts ! however the /sys/kernel/debug/ecm is created ...
my mistake I took out from the qnap dts the nss dtsi so it would stop crashing ... just recompiling it with the nss dts now. will report back i presume will have "the data plane vsi" errors
it works - NSS cores booted ; nss queues present on /proc/interrupts and no crashes. Absolutely brilliant @bitthief thank you
it works - NSS cores booted ; nss queues present on /proc/interrupts and no crashes. Absolutely brilliant @bitthief thank you
How do you compile this error?
it works - NSS cores booted ; nss queues present on /proc/interrupts and no crashes. Absolutely brilliant @bitthief thank you
How do you compile this error?
I don't use ppoe nor the nss vlan & bridge
I don't use ppoe nor the nss vlan & bridge
The firmware seems to be stable, but only the system light is on and the Internet light is off. In your case, are both lights on?
yes ... I have wifi and lan... i am planning to move it from the lab to a testing bed where people can then start connecting to ... note that I am not using the ax3600 but the QNAP QORA
ok so running with about 10 clients both on 2g & 5g ... and pretty stable apart from these messages
[ 1676.025561] ath11k c000000.wifi: No VIF found for vdev 1 [ 1676.025610] ath11k c000000.wifi: invalid vdev id 1 in bcn_tx_status [ 1935.152974] ath11k c000000.wifi: failed to flush transmit queue, data pkts pending 1
made a few changes after noticing the qdisc being used if pfifo instead fq_codel .... there was quite a lot of latency using pfifo
also changed the patch target/linux/generic/hack-5.15/660-fq_codel_defaults.patch
from 4 to 32 (note - I am using the qnap that 1GB ram - worth setting it to 16 for the ax3600)
From a6ccb238939b25851474a279b20367fd24a0e816 Mon Sep 17 00:00:00 2001 From: Felix Fietkau nbd@nbd.name Date: Fri, 7 Jul 2017 17:21:53 +0200 Subject: hack: net: fq_codel: tune defaults for small devices
Assume that x86_64 devices always have a big memory and do not need this optimization compared to devices with only 32 MB or 64 MB RAM.
net/sched/sch_fq_codel.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
--- a/net/sched/sch_fq_codel.c +++ b/net/sched/sch_fq_codel.c @@ -469,7 +469,11 @@ static int fq_codel_init(struct Qdisc *s
sch->limit = 10*1024;
q->flows_cnt = 1024;
+#ifdef CONFIG_X86_64 q->memory_limit = 32 << 20; / 32 MBytes / +#else
this helped a lot ... no more latency ... eg setting to fq_codel instead of pfifo
a question @bitthief ran ecm_dump.sh and want to make accel sure is being used all of the ported.accel_mode are set to -2 ... do you know what this means? thx
I haven't switched on offloading ... does anyone know if HW offloading works ? SW doesn't help ...
also changed the patch target/linux/generic/hack-5.15/660-fq_codel_defaults.patch
from 4 to 32 (note - I am using the qnap that 1GB ram - worth setting it to 16 for the ax3600)
From a6ccb238939b25851474a279b20367fd24a0e816 Mon Sep 17 00:00:00 2001 From: Felix Fietkau nbd@nbd.name Date: Fri, 7 Jul 2017 17:21:53 +0200 Subject: hack: net: fq_codel: tune defaults for small devices
Assume that x86_64 devices always have a big memory and do not need this optimization compared to devices with only 32 MB or 64 MB RAM.
Signed-off-by: Felix Fietkau nbd@nbd.name
net/sched/sch_fq_codel.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
--- a/net/sched/sch_fq_codel.c +++ b/net/sched/sch_fq_codel.c @@ -469,7 +469,11 @@ static int fq_codel_init(struct Qdisc *s
sch->limit = 10*1024; q->flows_cnt = 1024;
+#ifdef CONFIG_X86_64 q->memory_limit = 32 << 20; / 32 MBytes / +#else
q->memory_limit = 32 << 20; /* 4 MBytes */
+#endif q->drop_batch_size = 64; q->quantum = psched_mtu(qdisc_dev(sch)); INIT_LIST_HEAD(&q->new_flows);
Does it have to be 32?
no it doesn't need to be 32 ... on the ax3600 I have it with 16
just to report - that by enabling sqm I hit the same error @bitthief with nf_conntrack and ecm_conntrack
here's the log of the kernel crash [ 17.995737] Unexpected kernel BRK exception at EL1 [ 17.999623] Internal error: BRK handler: f2000800 [#1] SMP [ 18.004396] Modules linked in: ecm(+) xt_connlimit pppoe ppp_async nf_conncount iptable_nat ath11k_ahb ath11k xt_state xt_nat xt_helper xt_conntrack xt_connmark xt_connbytes xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD xt_CT pppox ppp_generic nft_redir nft_nat nft_masq nft_fullcone nft_flow_offload nft_fib_inet nft_ct nft_chain_nat nf_nat nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet nf_flow_table nf_conntrack_netlink nf_conntrack mmc_spi mac80211 ipt_REJECT cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_recent xt_quota xt_pkttype xt_physdev xt_owner xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_ecn xt_dscp xt_comment xt_cgroup xt_addrtype xt_TCPMSS xt_LOG xt_HL xt_DSCP xt_CLASSIFY ums_usbat ums_sddr55 ums_sddr09 ums_karma ums_jumpshot ums_isd200 ums_freecom ums_datafab ums_cypress ums_alauda spidev spi_gpio spi_bitbang slhc sch_cake qrtr_tun qrtr_smd qrtr_mhi qrtr qmi_helpers of_mmc_spi ns nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_quota [ 18.004553] nft_objref nft_numgen nft_log nft_limit nft_hash nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_counter nft_compat nf_tables nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 mhi_pci_generic mhi_net mhi mdio_gpio mdio_bitbang macvlan libcrc32c iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables hwmon crc7 crc_itu_t crc_ccitt compat_xtables compat fuse sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact i2c_gpio i2c_algo_bit qca_nss_crypto i2c_mux_gpio i2c_mux cryptodev xt_set ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 bonding ifb qca_nss_drv [ 18.074483] ip_tunnel veth tun autofs4 vxlan udp_tunnel ip6_udp_tunnel seqiv michael_mic usb_storage leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom ar8xxx swconfig qca_nss_dp qca_ssdk gpio_button_hotplug f2fs ext4 mbcache jbd2 exfat gpio_cascade mux_gpio mux_core crc32c_generic crc32_generic [ 18.188064] CPU: 1 PID: 4222 Comm: modprobe Not tainted 5.15.59 #0 [ 18.210298] Hardware name: QNAP 301w (DT) [ 18.216545] pstate: 40400005 (nZcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 18.220630] pc : nf_conntrack_register_notifier+0x58/0x60 [nf_conntrack] [ 18.227400] lr : nf_conntrack_register_notifier+0x2c/0x60 [nf_conntrack] [ 18.234345] sp : ffffffc00d03bac0 [ 18.241021] x29: ffffffc00d03bac0 x28: ffffffc0080d0fe0 x27: 0000000000000022 [ 18.244241] x26: ffffffc00130ba50 x25: 0000000000000022 x24: 0000000000000002 [ 18.251359] x23: 0000000000000002 x22: 0000000000000000 x21: ffffffc001309d98 [ 18.258478] x20: ffffffc008c11400 x19: ffffffc001143a60 x18: 000000070000000a [ 18.265596] x17: 0000000800000e10 x16: 000000090000012c x15: 0000000a00000708 [ 18.272714] x14: 000000000000023c x13: 7265696669746f6e x12: 0000000000000005 [ 18.279832] x11: ffffffc008a28b8c x10: 0000000000000000 x9 : 0000000000000000 [ 18.286950] x8 : 00000000706f7473 x7 : 0000000000000004 x6 : 0000000000000000 [ 18.294068] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000000 [ 18.301186] x2 : ffffff8014b03c00 x1 : 0000000000000000 x0 : ffffffc001158000 [ 18.308304] Call trace: [ 18.315413] nf_conntrack_register_notifier+0x58/0x60 [nf_conntrack] [ 18.317678] ecm_conntrack_notifier_init+0x54/0x1ec [ecm] [ 18.324273] init_module+0x164/0x1000 [ecm] [ 18.329565] do_one_initcall+0x50/0x1b0 [ 18.333557] do_init_module+0x44/0x1d0 [ 18.337377] load_module+0x1c40/0x23d0 [ 18.341197] __do_sys_init_module+0x190/0x250 [ 18.344931] __arm64_sys_init_module+0x1c/0x30 [ 18.349357] invoke_syscall.constprop.0+0x5c/0x104 [ 18.353698] do_el0_svc+0x6c/0x15c [ 18.358469] el0_svc+0x18/0x54 [ 18.361854] el0t_64_sync_handler+0xe8/0x114 [ 18.364895] el0t_64_sync+0x184/0x188 [ 18.369325] Code: f94013f5 a8c37bfd d50323bf d65f03c0 (d4210000) [ 18.372887] ---[ end trace 4edf31a582b14806 ]--- [ 18.378960] Kernel panic - not syncing: BRK handler: Fatal exception [ 18.383650] SMP: stopping secondary CPUs [ 18.389987] Kernel Offset: disabled [ 18.393887] CPU features: 0x00000000,00000802 [ 18.397101] Memory Limit: none
I think this can be closed. Thank you!
Network interface failed to start on Redmi AX6 after adding firmware-nss-ipq8074. If I delete the nss firmware in rootfs, or I don't compile it in .config, the network interface will start normally.
Notice the errors
bad descriptor dump for nss core = 0
andnss-dp 3a001400.dp3 eth1: Data plane vsi assign failed
in the kernel log. Maybe they can help?Kernel log: