search

apernet / OpenGFW

OpenGFW is a flexible, easy-to-use, open source implementation of GFW (Great Firewall of China) on Linux
https://gfw.dev/
Mozilla Public License 2.0
9.59k stars 722 forks source link

2024-03-24T04:05:45Z INFO engine exited {"error": "could not unbind existing handlers (if any): netlink receive: invalid argument"} #110

Closed BQvQB closed 6 months ago

BQvQB commented 6 months ago

我使用了本地编译、直接下载你们最新的可执行文件，但是运行之后出现了以下错误（如图所示），之后使用了poweroff命令关闭软路由系统，准备再次进入软路由系统时，却无法进入（无法正常开机）。

root@iStoreOS:~# chmod a+x OpenGFW
root@iStoreOS:~# ls
OpenGFW      config.yaml  geosite.dat  rules_.yaml
OpenGFW_     geoip.dat    rules.yaml
root@iStoreOS:~# ./OpenGFW -c config.yaml rules_.yaml
2024-03-24T04:05:45Z    INFO    engine started
2024-03-24T04:05:45Z    DEBUG   worker started  {"id": 0}
2024-03-24T04:05:45Z    DEBUG   worker started  {"id": 2}
2024-03-24T04:05:45Z    DEBUG   worker started  {"id": 3}
2024-03-24T04:05:45Z    DEBUG   worker started  {"id": 1}
2024-03-24T04:05:45Z    DEBUG   worker stopped  {"id": 0}
2024-03-24T04:05:45Z    DEBUG   worker stopped  {"id": 1}
2024-03-24T04:05:45Z    INFO    engine exited   {"error": "could not unbind existing handlers (if any): netlink receive: invalid argument"}
2024-03-24T04:05:45Z    DEBUG   worker stopped  {"id": 3}
2024-03-24T04:05:45Z    DEBUG   worker stopped  {"id": 2}
root@iStoreOS:~# exit
Connection to 192.168.100.1 closed.

C:\Users\qvbsk\Downloads\OpenGFW-0.3.0\OpenGFW-0.3.0>ssh root@192.168.100.1
root@192.168.100.1's password:

BusyBox v1.35.0 (2023-07-05 07:27:39 UTC) built-in shell (ash)

   ▀     ▄▄▄▄    ▄                          ▄▄▄▄   ▄▄▄▄
 ▄▄▄    █▀   ▀ ▄▄█▄▄   ▄▄▄    ▄ ▄▄   ▄▄▄   ▄▀  ▀▄ █▀   ▀
   █    ▀█▄▄▄    █    █▀ ▀█   █▀  ▀ █▀  █  █    █ ▀█▄▄▄
   █        ▀█   █    █   █   █     █▀▀▀▀  █    █     ▀█
 ▄▄█▄▄  ▀▄▄▄█▀   ▀▄▄  ▀█▄█▀   █     ▀█▄▄▀   █▄▄█  ▀▄▄▄█▀

                                  Powered by OpenWRT
 -------------------------------------------------------
 iStoreOS 22.03.5, 2023122916
 -------------------------------------------------------
root@iStoreOS:~#

openwrt_error openwrt_error

haruue commented 6 months ago

请确认你已经正确地安装了依赖， 如同 README 中描述的那样。 https://github.com/apernet/OpenGFW/blob/master/README.zh.md#openwrt

如果你的机器上有 nft 这个命令， 请安装 kmod-nft-queue kmod-nf-conntrack-netlink， 否则， 请安装 kmod-ipt-nfqueue iptables-mod-nfqueue kmod-nf-conntrack-netlink。

BQvQB commented 6 months ago

请确认你已经正确地安装了依赖， 如同 README 中描述的那样。 https://github.com/apernet/OpenGFW/blob/master/README.zh.md#openwrt

如果你的机器上有 nft 这个命令， 请安装 kmod-nft-queue kmod-nf-conntrack-netlink， 否则， 请安装 kmod-ipt-nfqueue iptables-mod-nfqueue kmod-nf-conntrack-netlink。

936034ee5267343c302752b1d79e123 ![Uploading 123.png…]()

BQvQB commented 6 months ago

请确认你已经正确地安装了依赖， 如同 README 中描述的那样。 https://github.com/apernet/OpenGFW/blob/master/README.zh.md#openwrt

如果你的机器上有 nft 这个命令， 请安装 kmod-nft-queue kmod-nf-conntrack-netlink， 否则， 请安装 kmod-ipt-nfqueue iptables-mod-nfqueue kmod-nf-conntrack-netlink。

我确实正确安装了依赖，这个版本的openwrt使用的是iptables，我通过您提供的解决方案又尝试了遍，但是这个问题还是没办法解决。

BQvQB commented 6 months ago

请确认你已经正确地安装了依赖， 如同 README 中描述的那样。 https://github.com/apernet/OpenGFW/blob/master/README.zh.md#openwrt 如果你的机器上有 nft 这个命令， 请安装 kmod-nft-queue kmod-nf-conntrack-netlink， 否则， 请安装 kmod-ipt-nfqueue iptables-mod-nfqueue kmod-nf-conntrack-netlink。

我确实正确安装了依赖，这个版本的openwrt使用的是iptables，我通过您提供的解决方案又尝试了遍，但是这个问题还是没办法解决。

我重启设备后，设备再次无法开机，又要重置，头大，我已经尝试好多遍，奈何对内核理解不够，我想我自己无法解决这个问题了。

haruue commented 6 months ago

我感觉这个版本的系统有点问题。。 我下载了一个和你同版本的 istoreos-22.03.5-2023122916-x86-64-squashfs-combined.img.gz， 用 qemu 起了， 给了 4GB 内存， 然而即使只是执行 opkg install kmod-ipt-nfqueue iptables-mod-nfqueue kmod-nf-conntrack-netlink 也会重启。

展开内核堆叠追踪 ``` [ 66.241834] ccp_crypto: Cannot load: there are no available CCPs [ 66.511056] kvm: already loaded the other module [ 66.697297] BUG: kernel NULL pointer dereference, address: 0000000000000148 [ 66.698661] #PF: supervisor read access in kernel mode [ 66.699656] #PF: error_code(0x0000) - not-present page [ 66.700633] PGD 10c2e4067 P4D 10c2e4067 PUD 10eef8067 PMD 0 [ 66.701687] Oops: 0000 [#1] SMP NOPTI [ 66.702528] CPU: 2 PID: 14469 Comm: kmodloader Tainted: G U 5.10.176 #0 [ 66.703838] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.3-1-1 04/01/2014 [ 66.705346] RIP: 0010:_1+0x1899f67/0x189b12f [nfnetlink_queue] [ 66.706463] Code: ea 5b 41 5c 5d e9 79 89 35 e0 66 0f 1f 84 00 00 00 00 00 55 8b 05 41 31 00 00 48 89 e5 41 54 49 89 fc 53 48 8b 97 a8 0e 00 00 <48> 8b 1c c2 e8 f0 e1 86 df 48 8d 7b 10 89 d8 45 31 c9 48 83 e7 f8 [ 66.709733] RSP: 0018:ffffc90007a27bc8 EFLAGS: 00010286 [ 66.710819] RAX: 0000000000000029 RBX: ffffffffa18ab020 RCX: 0000000000000000 [ 66.712098] RDX: 0000000000000000 RSI: 0000000000000086 RDI: ffffffff82304640 [ 66.713388] RBP: ffffc90007a27bd8 R08: 0000000000000001 R09: ffff88813bd20d00 [ 66.714653] R10: 0000000000000000 R11: 0000000000000008 R12: ffffffff82304640 [ 66.715944] R13: ffffffff82304640 R14: 0000000000000029 R15: ffff88810c7eb000 [ 66.717246] FS: 00007fd70bf32b48(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000 [ 66.718627] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 66.719764] CR2: 0000000000000148 CR3: 00000001094ea000 CR4: 0000000000350ee0 [ 66.721058] Call Trace: [ 66.721835] ops_init+0x3d/0x130 [ 66.722706] register_pernet_operations+0xdb/0x1c0 [ 66.723754] register_pernet_subsys+0x24/0x40 [ 66.724724] ? 0xffffffffa000e000 [ 66.725599] _1+0x12/0x1000 [nfnetlink_queue] [ 66.726601] ? 0xffffffffa000e000 [ 66.727468] do_one_initcall+0x4b/0x1b0 [ 66.728401] ? kmem_cache_alloc+0x126/0x260 [ 66.729380] do_init_module+0x48/0x230 [ 66.730288] load_module+0x2350/0x25e0 [ 66.731191] __do_sys_init_module+0xf1/0x130 [ 66.732156] ? __do_sys_init_module+0xf1/0x130 [ 66.733179] __x64_sys_init_module+0x15/0x20 [ 66.734159] do_syscall_64+0x38/0x50 [ 66.735060] entry_SYSCALL_64_after_hwframe+0x61/0xc6 [ 66.736130] RIP: 0033:0x7fd70bef2363 [ 66.737011] Code: 0e 4c 8b 44 24 10 4d 8d 48 08 4c 89 4c 24 10 44 8b 4c 24 08 4d 8b 00 4c 01 c9 41 83 f9 2f 76 05 48 8b 4c 24 10 4c 8b 09 0f 05 <48> 89 c7 e8 55 ee fd ff 48 83 c4 58 c3 31 d2 56 bf 01 00 00 00 be [ 66.740446] RSP: 002b:00007ffee71667f0 EFLAGS: 00000212 ORIG_RAX: 00000000000000af [ 66.741809] RAX: ffffffffffffffda RBX: 0000000000005e80 RCX: 00007fd70bef2363 [ 66.743142] RDX: 00000000004043c5 RSI: 0000000000005e80 RDI: 00007fd70be580f0 [ 66.744427] RBP: 00007fd70be580f0 R08: 0000000000000000 R09: 0000000000000014 [ 66.745717] R10: 0000000000000000 R11: 0000000000000212 R12: 0000000000000000 [ 66.746988] R13: 0000000000406300 R14: 0000000000000004 R15: 00000000004043c5 [ 66.748266] Modules linked in: nfnetlink_queue(+) xt_FULLCONENAT rtw_8822cu rtw_8822ce rtw_8822c rtw_8822bu rtw_8822be rtw_8822b rtw_8821cu rtw_8821ce rtw_8821c rtw_8723du rtw_8723de rtw_8723d rtl8821ae rtl8192se rtl8192de rtl8192cu rtl8192c_common rtl_usb rtl_pci pppoe ppp_async mt76x0u mt76x0_common l2tp_ppp iwlmvm iwldvm cdc_mbim btcoexist ath10k_pci ath10k_core ath wireguard vfio_pci uvcvideo sr9700 snd_usb_audio smsc95xx sierra_net rtw_usb rtw_pci rtw_core rtlwifi rtl8xxxu rtl8812au rndis_host qmi_wwan qcserial pptp pppox ppp_mppe ppp_generic plusb option mwifiex_pcie mwifiex mt7921u mt7921e mt7921_common mt7915e mt76x2u mt76x2_common mt76x02_usb mt76x02_lib mt7663u mt7663_usb_sdio_common mt7615e mt7615_common mt76_usb mt76_connac_lib mt76 mcs7830 mac80211 libchacha20poly1305 kalmia iwlwifi ipw ipt_REJECT huawei_cdc_ncm dm9601 curve25519_x86_64 chacha_x86_64 cfg80211 cdc_subset cdc_ncm cdc_ether cdc_eem ax88179_178a asix aqc111 amdgpu zstd xt_time xt_tcpudp xt_tcpmss xt_string [ 66.748301] xt_statistic xt_state xt_socket xt_recent xt_quota xt_pkttype xt_physdev xt_owner xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_iprange xt_iface xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_cgroup xt_bpf xt_addrtype xt_TPROXY xt_TCPMSS xt_REDIRECT xt_NETMAP xt_MASQUERADE xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY xr_usb_serial_common visor videobuf2_v4l2 videobuf2_common via_velocity via_rhine vfio_virqfd vfio_mdev vfio_iommu_type1 vfio usbnet usblp usbatm usb_wwan usb_serial_simple ums_usbat ums_sddr55 ums_sddr09 ums_karma ums_jumpshot ums_isd200 ums_freecom ums_datafab ums_cypress ums_alauda tulip ts_fsm ts_bm ti_usb_3410_5052 solos_pci snd_usbmidi_lib smsc slhc sky2 skge sis900 sis190 sierra sfc_falcon sfc sch_mqprio sch_cake rtl8150 radeon r8152 r6040 poly1305_x86_64 pegasus pcnet32 oti6858 ntfs3 niu nf_tproxy_ipv6 nf_tproxy_ipv4 nf_socket_ipv6 nf_socket_ipv4 nf_reject_ipv4 nf_nat_tftp [ 66.761900] nf_nat_snmp_basic nf_nat_sip nf_nat_pptp nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_log_ipv6 nf_log_ipv4 nf_log_common nf_flow_table nf_conntrack_tftp nf_conntrack_snmp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_netlink nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack_broadcast ts_kmp nf_conntrack_amanda nf_conncount ne2k_pci mos7840 mos7720 mmc_spi mlx5_core mlx4_en mlx4_core mdio_gpio mdio_bitbang mdev mct_u232 macvlan lzo_rle lzo libcurve25519_generic libchacha kvm_amd kvm keyspan kaweth irqbypass iptable_raw iptable_nat iptable_mangle iptable_filter ipt_ECN ipheth ip_tables io_edgeport iavf i915 hso hid_cp2112 gpu_sched garmin_gps forcedeth ezusb ethoc et131x dmx_usb cypress_m8 crc7 crc_ccitt compat cm109 cls_flower chaoskey cdc_wdm cdc_acm br_netfilter bnx2x bnx2 belkin_sa be2net ax88796b atl2 atl1e atl1c atl1 ark3116 alx 8390 8139too 8139cp fuse sch_teql sch_sfq sch_multiq sch_gred sch_fq sch_dsmark sch_codel em_text em_nbyte em_meta em_cmp [ 66.775957] act_simple act_pedit act_csum em_ipset cls_bpf act_bpf act_ctinfo act_connmark sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact videobuf2_vmalloc videobuf2_memops sg videodev evdev drivetemp i2c_dev ledtrig_usbport trelay spi_ks8995 siit ledtrig_activity xt_set ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ipmac ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink st rtl8366s rtl8366rb rtl8306 ip6table_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip6t_NPT ip17xx b53_mdio b53_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 nfsv4 nfsv3 nfsd nfs nfs_ssc msdos bonding 3c59x ip6_gre ip_gre gre ixgbevf ixgbe r8169 igc igbvf i40e e1000e e1000 [ 66.790608] e100 amd_xgbe ifb dummy nat46 l2tp_ip6 l2tp_ip l2tp_eth sit sctp mdio l2tp_netlink l2tp_core ipcomp6 xfrm6_tunnel esp6 ah6 xfrm4_tunnel ipcomp esp4 ah4 ipip ip6_tunnel netlink_diag tunnel6 tunnel4 ip_tunnel udp_diag tcp_diag raw_diag inet_diag hfcpci hfcmulti rpcsec_gss_krb5 auth_rpcgss oid_registry dm_raid veth tun snd_rawmidi snd_seq_device snd_pcm_oss snd_mixer_oss snd_hwdep snd_compress snd_pcm snd_timer snd soundcore nbd mISDN_dsp l1oip mISDN_core xfrm_user xfrm_ipcomp af_key xfrm_algo vfat fat udf crc_itu_t lockd sunrpc grace minix hfsplus hfs cramfs configfs cifs binfmt_misc autofs4 9p dns_resolver br2684 atm aoe multipath fscache 9pnet_virtio 9pnet raid456 async_raid6_recov async_pq async_xor async_memcpy async_tx raid10 raid1 raid0 linear md_mod nls_utf8 nls_iso8859_1 nls_cp936 nls_cp437 zram zsmalloc natsemi vxlan udp_tunnel ip6_udp_tunnel ena sha512_ssse3 sha512_generic sha1_ssse3 seqiv jitterentropy_rng drbg pcbc md5 md4 kpp rsa_generic mpi asn1_decoder akcipher [ 66.806003] ccp sha1_generic hmac fcrypt echainiv des_generic libdes deflate cts cmac authenc arc4 crypto_acompress xhci_plat_hcd dwc3 dwc2 roles rtl8367b swconfig rtl8366_smi sata_via sata_sil24 sata_sil pata_pdc202xx_old sata_nv pata_artop fsl_mph_dr_of ehci_platform ehci_fsl mvsas mpt3sas raid_class igb xfs reiserfs jfs exfat btrfs zstd_decompress zstd_compress xxhash xor raid6_pq lzo_decom Booting `iStoreOS' Booting `iStoreOS ```

在载入之后， nfnetlink_queue 这个模块就触发了空指针导致 kernel panic， 我认为这就是你重启之后无法开机的原因。

haruue commented 6 months ago

我又下载了官网上最新的 istoreos-22.03.6-2024031514-x86-64-squashfs-combined.img.gz， 无此问题， 然而 OpenGFW 仍然无法正常工作。

在简单地审视了 iptables 规则之后， 我发现了一些问题。。

filter 表的 FORWARD 链， 预设规则如下

-A FORWARD -m comment --comment "!fw3: Custom forwarding rule chain" -j forwarding_rule
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
-A FORWARD -i br-lan -m comment --comment "!fw3" -j zone_lan_forward
-A FORWARD -i eth0 -m comment --comment "!fw3" -j zone_wan_forward
-A FORWARD -i docker0 -m comment --comment "!fw3" -j zone_docker_forward
-A FORWARD -m comment --comment "!fw3" -j reject
-A FORWARD -m connmark --mark 0x3e9 -j ACCEPT
-A FORWARD -m connmark --mark 0x3ea -j DROP
-A FORWARD -j NFQUEUE --queue-num 100 --queue-bypass

考虑到 OpenGFW 主要针对局域网流量， 匹配其中 -i br-lan 规则， 对应的 zone_lan_forward 是

-A zone_lan_forward -m comment --comment "!fw3: Custom lan forwarding rule chain" -j forwarding_lan_rule
-A zone_lan_forward -m comment --comment "!fw3: Zone lan to wan forwarding policy" -j zone_wan_dest_ACCEPT
-A zone_lan_forward -m comment --comment "!fw3: Zone lan to docker forwarding policy" -j zone_docker_dest_ACCEPT
-A zone_lan_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
-A zone_lan_forward -m comment --comment "!fw3" -j zone_lan_dest_ACCEPT

最终的 zone_lan_dest_ACCEPT 是

-A zone_lan_dest_ACCEPT -o br-lan -m comment --comment "!fw3" -j ACCEPT

这实际上短路了 OpenGFW 添加的 NFQUEUE 规则， 因此流量不会被 OpenGFW 处理。

haruue commented 6 months ago

为了解决这个问题， 我认为可以安装 nftables。 在存在 nftables 的情况下， OpenGFW 会优先使用 nftables 来设置 NFQUEUE。 而 nftables 和 iptables 是链状处理关系， 也就是说其中一个的 accept 会继续将包交给另一个进行处理， 不会被短路掉。

我进行了一些测试， 在 istoreos-22.03.6-2024031514-x86-64-squashfs-combined.img.gz 上， 先安装以下包

opkg install nftables kmod-nft-queue kmod-nf-conntrack-netlink

然后启动 OpenGFW， 就能看到已经工作正常了。

haruue commented 6 months ago

综上所述， 以下是为了解决你遇到的问题， 提出的建议：

  1. 升级 iStoreOS 到 istoreos-22.03.6-2024031514， 你可以去 iStoreOS 官方提供的下载点进行下载（x86-64, x86-64-efi）。
  2. 执行这条命令来安装包 opkg install nftables kmod-nft-queue kmod-nf-conntrack-netlink。
  3. 部署并测试 OpenGFW。
BQvQB commented 6 months ago

Awesome, you are awesome! Thank you for your answer. It is indeed available now. This is the cause of the problem you mentioned.

综上所述， 以下是为了解决你遇到的问题， 提出的建议：

  1. 升级 iStoreOS 到 istoreos-22.03.6-2024031514， 你可以去 iStoreOS 官方提供的下载点进行下载（x86-64, x86-64-efi）。
  2. 执行这条命令来安装包 opkg install nftables kmod-nft-queue kmod-nf-conntrack-netlink。
  3. 部署并测试 OpenGFW。

Awesome, you are awesome! Thank you for your answer. It is indeed available now. This is the cause of the problem you mentioned.