search

Chion82 / netfilter-full-cone-nat

A kernel module to turn MASQUERADE into full cone SNAT
GNU General Public License v2.0
430 stars 121 forks source link

Nftables fullcone implementation #42

Open wongsyrone opened 2 years ago

wongsyrone commented 2 years ago

Based on code from @Chion82 and @llccd , I implement fullcone support for nftables.

OpenWrt kernel module Package: Compile and run tested on Linux 5.15.32 OpenWrt master branch， not run tested thoroughly. https://github.com/wongsyrone/nft-fullcone

OpenWrt firewall4, a.k.a. the new nftables-based firewall, patched repo listed below: https://github.com/wongsyrone/openwrt-firewall4-with-fullcone

You have to patch libnftnl and nftables for the new 'fullcone' statement, patched repo listed below: https://github.com/wongsyrone/libnftnl-1.2.1-with-fullcone https://github.com/wongsyrone/nftables-1.0.2-with-fullcone

Sometimes you have to call autoreconf to regenerate configure script.

BONUS: luci-app-turboacc change listed below, I only use its shortcut-fe and fullcone NAT functionality, thus takes what you want.

https://github.com/wongsyrone/lede-1/commit/a15fece6ef570065ec6963bb3604c3b8b01354bc

fraybyl commented 2 years ago

Hi, I get the error root@OpenWrt:~# /etc/init.d/firewall restart nft_try_fullcone: cmd /usr/sbin/nft -c 'add table inet fw4-fullcone-test; add chain inet fw4-fullcone-test dstnat { type nat hook prerouting priority -100; policy accept; fullcone; }; add chain inet fw4-fullcone-test srcnat { type nat hook postrouting priority -100; policy accept; fullcone; }; ' 2>/dev/null nft_try_fullcone failed, disable fullcone globally Section @zone[0] (lan) fullcone is not enabled in default settings, ignore fullcone settings in zone Section @zone[1] (wan) fullcone is not enabled by default, ignore zone fullcone setting

wongsyrone commented 2 years ago

@fraybyl You need to patch nftables and libnftnl as well

fraybyl commented 2 years ago

@wongsyrone

I spent quite a bit of time with this, but if I am not mistaken, I was able to change the repository for nftables and libnftnl. Is there any way to check if nftables and libnftnl are built correctly?

wongsyrone commented 2 years ago

@fraybyl

nft_try_fullcone failed disappears if everything is okay.

fraybyl commented 2 years ago

@wongsyrone I take openwrt/package/libs/libnftnl/Makefile and change the PKG_SOURCE_URL to yours, what am I doing wrong?

wongsyrone commented 2 years ago

@fraybyl Please note the patches folder in libnftnl and nftables directory.

wongsyrone commented 2 years ago

@fraybyl

If you really don't know how to compile these binaries correctly, just copy these folders and replace yours.

https://github.com/wongsyrone/lede-1/tree/master/package/network/utils/nftables https://github.com/wongsyrone/lede-1/tree/master/package/libs/libnftnl https://github.com/wongsyrone/lede-1/tree/master/package/network/config/firewall4

fraybyl commented 2 years ago

@wongsyrone Apparently I am very stupid!) thank you so much for your help and your efforts.

debiansid commented 2 years ago

can I patch it into 5.10.113?

wongsyrone commented 2 years ago

can I patch it into 5.10.113?

Yes.

debiansid commented 2 years ago

can I patch it into 5.10.113?

Yes.

https://github.com/Chion82/netfilter-full-cone-nat#kernel-patch-optional I want to build it into kernel like this link. would you please guide me to get that? thanks

debiansid commented 2 years ago

How to configure it on Linux ?

debiansid commented 2 years ago

Will this work ?

table inet my_nat {
     chain my_masquerade {
     type nat hook postrouting priority srcnat;
     oifname "pppoe0" masquerade fullcone
  }
}
wongsyrone commented 2 years ago

Example from OpenWrt:

table inet fw4 {
    chain dstnat {
        type nat hook prerouting priority dstnat; policy accept;
        iifname "pppoe-wan" jump dstnat_wan comment "!fw4: Handle wan IPv4/IPv6 dstnat traffic"
    }

    chain srcnat {
        type nat hook postrouting priority srcnat; policy accept;
        oifname "pppoe-wan" jump srcnat_wan comment "!fw4: Handle wan IPv4/IPv6 srcnat traffic"
    }

    chain srcnat_wan {
        fullcone comment "!fw4: Handle wan IPv4/IPv6 fullcone NAT traffic"
    }

    chain dstnat_wan {
        fullcone comment "!fw4: Handle wan IPv4/IPv6 fullcone NAT traffic"
    }
}
wongsyrone commented 2 years ago

If you use my OpenWrt firewall4 patch, you get these rules automatically.

wongsyrone commented 2 years ago

https://github.com/Chion82/netfilter-full-cone-nat#kernel-patch-optional I want to build it into kernel like this link. would you please guide me to get that? thanks

You may modify the kernel source based on the link you refer to, I do not maintain a kernel patch.

debiansid commented 2 years ago

I understand it is implementation on openwrt. I want to get it work on Linux. I will keep trying, thanks

paladin4fan commented 2 years ago

Based on code from @Chion82 and @llccd , I implement fullcone support for nftables.

OpenWrt kernel module Package: Compile and run tested on Linux 5.15.32 OpenWrt master branch， not run tested thoroughly. https://github.com/wongsyrone/nft-fullcone

OpenWrt firewall4, a.k.a. the new nftables-based firewall, patched repo listed below: https://github.com/wongsyrone/openwrt-firewall4-with-fullcone

You have to patch libnftnl and nftables for the new 'fullcone' statement, patched repo listed below: https://github.com/wongsyrone/libnftnl-1.2.1-with-fullcone https://github.com/wongsyrone/nftables-1.0.2-with-fullcone

Sometimes you have to call autoreconf to regenerate configure script.

BONUS: luci-app-turboacc change listed below, I only use it's shortcut-fe and fullcone NAT functionality, thus takes what you want.

wongsyrone/lede-1@a15fece

Thanks for provide the solutions. Should I compile these project along? Or is there a way to apply it to the openwrt package? I just compiled an openwrt 22.03.0 build with the kmod-nft-fullcone you gave, but I'm confused how to add the rest 3 project into my build

bitthief commented 2 years ago

Hi,

@wongsyrone thank you very much for this, impressive work!

I was able to integrate all your repos etc. into a nice and clean solution for OpenWRT trees, including the libnftnl and nftables patches. autoreconf was needed, but PKG_FIXUP did the trick.

Please find attached a commit in one of my forks which basically patches everything etc. and you can generate ipkgs. Tested partially on an AX3600 build, no issues. There was an initialization error with existing rules from firewall3 though, which might or might not be related.

https://github.com/bitthief/openwrt/commit/baa231ce92c9a937ae931006796bb6ab2ef77202

wongsyrone commented 2 years ago

There was an initialization error with existing rules from firewall3 though, which might or might not be related.

I should change that to info level instead of warning level.

wongsyrone commented 2 years ago

autoreconf was needed, but PKG_FIXUP did the trick.

Indeed. PKG_FIXUP is OpenWrt-specific, I just want to notify the one using GNU/Linux distros.

skill7899 commented 2 years ago

能出个教程，怎么用openwrt-firewall4-with-fullcone 和 libnftnl-1.2.1-with-fullcone nftables-1.0.2-with-fullcone这三个补丁，我看不像是feeds。

wongsyrone commented 2 years ago

能出个教程，怎么用openwrt-firewall4-with-fullcone 和 libnftnl-1.2.1-with-fullcone nftables-1.0.2-with-fullcone这三个补丁，我看不像是feeds。

不是feeds，需要把带有 'add fullcone' 字样的commit给 git format-patch 然后在openwrt里面就可以用了，打给对应的包即可，如果有不能打的补丁，需要适配新版本

paladin4fan commented 1 year ago

链接里的 libnftnl nftables和fw4补丁要如何使用呢……看上去不是makefile类型的 应该也不是放在patch文件夹里

debiansid commented 1 year ago

什么时候出类似这里的iptables补丁格式，打到内核的

wongsyrone commented 1 year ago

什么时候出类似这里的iptables补丁格式，打到内核的

https://stackoverflow.com/questions/22368264/compiling-out-of-tree-kernel-module-against-any-kernel-source-tree-on-the-filesy

ysc3839 commented 1 year ago

@wongsyrone Is it possible to build this module without changing kernel config? https://github.com/wongsyrone/lede-1/blob/ae833ac9e14a4a874144807b15f01c5d922a54c0/package/external/nft-fullcone/Makefile#L23 I'm trying to build this module that is compatible with official OpenWrt release kernel.

escape0707 commented 1 year ago

从很早以前 netfilter 方面的建议就是不要折腾 NAT，而是直接注重配好 ipv6。

现在中国大陆以外的网络基本上都很容易就能换到有 ipv6 地址的 ISP。在中国大陆的绝大多数城市也是可以通过中国电信等 ISP 的光猫获得 ipv6 地址的。进入网关配置关闭一下 ipv6 的防火墙即可。

感觉这里的开发没有什么必要了。

编辑： 怎么说呢，我今天看到 reddit 上面一个帖子 https://www.reddit.com/r/homelab/comments/z8o8n3/are_we_ipv6_ready/iycj2q3

我觉得我们真的应该想尽办法快速普及 ipv6，甚至于阻挠人们继续使用并停留在 ipv4 上了，以致于我甚至认为在合适的时机 OpenWRT 应该像主流浏览器废除 SSL 等过时的加密通信接口，OpenSSH 废除 RC4 等不安全的加密算法和密钥类型一样加快往 ipv6 的转化。

ipv6 已经 22 年了，而且现在 ipv4 的地址段越来越贵。大公司甚至可以通过高价购买 ipv4 地址段进行垄断而阻挠初创公司购买 ipv4 地址段来运营自己的业务，这样的资本寡头垄断的时代真的不应该到来。

我建议还是不要在完全圆锥上下功夫了。

skill7899 commented 1 year ago

从很早以前 netfilter 方面的建议就是不要折腾 NAT，而是直接注重配好 ipv6。

现在中国大陆以外的网络基本上都很容易就能换到有 ipv6 地址的 ISP。在中国大陆的绝大多数城市也是可以通过中国电信等 ISP 的光猫获得 ipv6 地址的。进入网关配置关闭一下 ipv6 的防火墙即可。

感觉这里的开发没有什么必要了。

编辑： 怎么说呢，我今天看到 reddit 上面一个帖子 https://www.reddit.com/r/homelab/comments/z8o8n3/are_we_ipv6_ready/iycj2q3

我觉得我们真的应该想尽办法快速普及 ipv6，甚至于阻挠人们继续使用并停留在 ipv4 上了，以致于我甚至认为在合适的时机 OpenWRT 应该像主流浏览器废除 SSL 等过时的加密通信接口，OpenSSH 废除 RC4 等不安全的加密算法和密钥类型一样加快往 ipv6 的转化。

ipv6 已经 22 年了，而且现在 ipv4 的地址段越来越贵。大公司甚至可以通过高价购买 ipv4 地址段进行垄断而阻挠初创公司购买 ipv4 地址段来运营自己的业务，这样的资本寡头垄断的时代真的不应该到来。

我建议还是不要在完全圆锥上下功夫了。

不同意，只是V6看起来普及很快，实际上，很多大厂的官网还是V4，只要是V4还是主流，就存在NAT问题。除非国家给政策XX年所有互联网必需支持双栈，否则这条路还长着

ysc3839 commented 1 year ago

@Escape0707 但问题是仍然有大量软件不支持 IPv6，尤其是 P2P 通信的游戏。而且直到现在 github.com 还不支持 IPv6。

escape0707 commented 1 year ago

@Escape0707 但问题是仍然有大量软件不支持 IPv6，尤其是 P2P 通信的游戏。而且直到现在 github.com 还不支持 IPv6。

主要这是一个悖论，就好像现在中国人越来越难摆脱在中国人头上拉屎撒尿的微信一样。当你有越多的 workaround，你的ISP 和水平不够、贪心的游戏公司、软件公司就越喜欢让你继续用 ipv4 并买地址段垄断。另外如果你用的是开源软件为主，那些开源软件才有可能真正的与时俱进用上 ipv6，闭源软件和其公司才时刻想着垄断。

ysc3839 commented 1 year ago

I wrote a script builds nft-fullcone kernel module suitable for OpenWrt official kernel. It's kernel vermagic is same as OpenWrt official kernel. So this module is compatible and can be installed directly in OpenWrt official image.

The build process references https://hamy.io/post/0015/how-to-compile-openwrt-and-still-use-the-official-repository/ The patch is derived from https://github.com/wongsyrone/lede-1

It uses GitHub Actions to automatically build binaries https://github.com/ysc3839/openwrt-official-builds-fullcone/actions

我编写了个编译 nft-fullcone 内核模块的脚本，可编译适用于 OpenWrt 官方内核的模块。 编译出的模块的内核 vermagic 和 OpenWrt 官方内核是一致的，因此可以直接在 OpenWrt 官方镜像中安装。

编译流程参考了 https://hamy.io/post/0015/how-to-compile-openwrt-and-still-use-the-official-repository/ patch 文件修改自 https://github.com/wongsyrone/lede-1

使用 GitHub Actions 自动编译 https://github.com/ysc3839/openwrt-official-builds-fullcone/actions

https://github.com/ysc3839/openwrt-official-builds-fullcone

debiansid commented 1 year ago

来个标准linux内核补丁

debiansid commented 1 year ago

用@wongsyrone的repo 没法在debian编译成功 有空给看看？

vbzc commented 1 year ago

@fraybyl

If you really don't know how to compile these binaries correctly, just copy these folders and replace yours.

https://github.com/wongsyrone/lede-1/tree/master/package/network/utils/nftables https://github.com/wongsyrone/lede-1/tree/master/package/libs/libnftnl https://github.com/wongsyrone/lede-1/tree/master/package/network/config/firewall4

大佬你好，请问 拷贝好文件以后，后续是按照一般情况 来编译吗？其他还需要做什么吗？谢谢!

escape0707 commented 1 year ago

我最近看到一篇tailscale写得非常好非常通俗易懂的（但是是英语）解释udp打洞的文章。

https://tailscale.com/blog/how-nat-traversal-works/

据我所知linux、包括openwrt用的都是endpoint independent mapping。所以从linux方面来说没有技术限制打洞。只是有的闭源应用软件开发做的不好。建议要么换ipv6要么换开源软件。

Ovear commented 1 year ago

所以从linux方面来说没有技术限制打洞

主要问题是Endpoint-Dependent firewall (dest. IP+port)，这样打洞的难度会提高很多。

不过这点我是有点疑问的，用NatTypeTester测下来是 Endpoint-Dependent firewall + Endpoint-Dependent NAT mapping，也就是Symmetric NAT。

根据Conntrack的工作原理来看，这应该是预期行为。

晚点我再翻一翻之前openwrt吵架的那一篇issue和tailscale新旧两篇文章看看是不是误解了什么，感谢推荐。

IPv6的话实际是用下来感觉不太算银弹，就算大陆很普及了，但是实际上其他地区的普及率还是很低，软件兼容性也是一个大问题。问题最大的还是路由和网关（尤其是移动网络）上配置的防火墙行为，如果是Netfilter和Windows defender的话，又要和Endpoint-Dependent firewall (dest. IP+port)缠斗了。

另外插播一则喜报，RouterOS v7.10beta5喜提endpoint-independent-nat，也就是fullcone。现在主流的软路由都能获得fullcone能力了。

最后再提一嘴IPv6吧。。如果在多线接入又没有BGP的支持下，还是得和NAT这永远绕不开的议题打交道（虽然NPT简单很多，但是最终还是引入了这个问题）。

Ovear commented 1 year ago

其实我更多还是比较好奇为什么Openwrt官方会对fullcone如此抗拒；但是反而市面上的商业产品（不管有没有基于openwrt，不管是家用还是商用，甚至运营商级）都不约而同的内置了fullcone。

很有趣的一个现象。

escape0707 commented 1 year ago

主要问题是Endpoint-Dependent firewall (dest. IP+port)，这样打洞的难度会提高很多。

文章中破除的迷信就是这个错误的观念，希望您能仔细阅读一下。

不过这点我是有点疑问的，用NatTypeTester测下来是 Endpoint-Dependent firewall + Endpoint-Independent NAT mapping，也就是Symmetric NAT。

比如您说的这种测试结果，用老名词对应着说的话应该是port restricted，这点和我以前测试的各种openwrt都是对的上的。

但是反而市面上的商业产品（不管有没有基于openwrt，不管是家用还是商用，甚至运营商级）都不约而同的内置了fullcone。

因为他们用自定的系统而不是依赖于linux的防火墙功能。另外cgnat实际上是被另一篇rfc要求达到所谓fullcone的，具体的现在不方便搜（

Ovear commented 1 year ago

比如您说的这种测试结果，用老名词对应着说的话应该是port restricted，这点和我以前测试的各种openwrt都是对的上的。

这个是我打错了，是Endpoint-Dependent firewall + Endpoint-Dependent NAT mapping

文章中破除的迷信就是这个错误的观念，希望您能仔细阅读一下。

NAT traversal文章我还没仔细看，根据之前how tailscale works这篇文章来看，Symmetric NAT是非常难以进行打洞的。

escape0707 commented 1 year ago

您是在什么网络环境下测出来的？

Ovear commented 1 year ago

您是在什么网络环境下测出来的？

感谢提醒，之前测试的时候没注意测试的网络是否下发了公网IP；如果是内网IP的话，测试会受到出口路由的NAT/Filter行为影响。

刚才重新用下列环境确保在公网IP下复测的结果是EndpointIndependent NAT mapping + AddressAndPortDependent firewall。

Openwrt 21.02 + Windows 10 with WDF

如果默认行为能达到这个级别，打洞确实不应该存在显著的困难。之前我只确认了netfilter与CONNTRACK的关系带来的Endpoint-Dependent firewall；重复测试之后也通过复现解除了NAT mapping的误解。

即：netfilter提供的默认行为确实是Endpoint-Independent NAT mapping + Endpoint-Dependent firewall (dest. IP+port)。

另外cgnat实际上是被另一篇rfc要求达到所谓fullcone的，具体的现在不方便搜（

这个RFC我挺好奇的，大佬有空了的时候，方便搜一下发出来吗？

escape0707 commented 1 year ago

呀，记错了

https://www.rfc-editor.org/rfc/rfc6888

REQ-7 但是是推荐。不过其实影响不大，RFC而已，中国都有防火长城了也不会去兢兢业业遵守这个。就比如REQ-9还让CGNAT必须提供PCP呢

哇哦，这有个幸运儿他的cgnat还真支持 https://www.reddit.com/r/HomeNetworking/comments/cx1rl3/

Ovear commented 1 year ago

很神奇的事，刚刚想着用原生Linux复现一下，结果出现了不一致的情况。

测试环境

测试客户端(192.168.0.100)->Openwrt(192.168.0.1/10.0.0.2)->Debian 11(10.0.0.1)

其中Openwrt开启Fullcone

测试客户端采用pystun3(自定义测试用）和NatTypeTester

受限于时间限制，暂时使用了多层NAT，有空了想办法看看单层NAT能否复线。

测试结论

NatTypeTester

RFC5780: EndpointIndependent Mapping + AddressAndPortDependent Filtering
RFC3489: Symmetric

pystun3

RFC3489: Symmetric NAT

结果分析

NatTypeTester中RFC3489按顺序进行了三次测试

Test1: 向目标服务器发送Bind测试（获取到changed address/port） Test2: 向目标服务器发送ChangeIPAndPort请求（此时STUN服务器会用changed address/port回复） Test3: 向changed address/port发送Bind测试

测试结果 Test1收到回复/Test2未收到回复/Test3收到回复 Test1回报的公网地址与Test3回报的公网地址相同，但是回报的端口不同。 程序显示为Symmetric NAT

NatTypeTester中RFC5780按顺序进行了三次测试

Test1: 向目标服务器发送Bind测试（获取到changed address/port） Test2: 向目标服务器发送ChangeIPAndPort请求（此时STUN服务器会用changed address/port回复） Test3: 向目标服务器发送ChangePort请求（此时STUN服务器会用changed port回复） Test4: 向changed address和原始端口发送Bind测试

测试结果 Test1收到回复/Test2未收到回复/Test3未收到回复/Test4收到回复 Test1回报的公网地址和端口与Test4回报的相同 程序显示为EndpointIndependent Mapping + AddressAndPortDependent Filtering

pystun3追加测试

RFC3489

默认情况下，pystun3采用测试的流程与NatTypeTester中RFC3489中的测试一致，但是增加了重复发包来避免丢包的行为，

测试表现、结果与NatTypeTester一致，都存在Test3与Test1回报的公网端口不同问题。

修改版RFC3489测试

变动如下 Test3(修改版): 向changed address和原始端口发送Bind测试 (即RFC5780的Test4)

测试结果 Test1收到回复/Test2未收到回复/Test3收到回复 Test1回报的公网地址和端口与Test3回报的相同 程序显示为Restric Port NAT

修改版RFC3489测试

变动如下 Test3(修改版): 向changed address和原始端口发送Bind测试 (即RFC5780的Test4) Test4(追加): 向changed address/port发送Bind测试 (即原本的Test3)

测试结果 Test1收到回复/Test2未收到回复/Test3收到回复/Test4收到回复 Test1回报的公网地址和端口与Test3回报的相同 Test1/Test3回报的公网地址与Test4回报的公网地址相同，但是端口不同

结论

本次测试中Debian 11的NAT MAPPING行为：

在目标端口一致时表现为Endpoint-Independent NAT mapping 目标端口不一致时表现为Endpoint-Dependent NAT mapping

也就是至少采用三元组匹配 (src_ip,src_port,dst_port)

PS：测试中端口变更均发生在Debian11进行nat的时候，Openwrt侧未发生端口变更；因为时间原因未测试Openwrt关闭Fullcone的情况，有空了看看能不能补充下。

PS2：突然想起来很久以前我测试是在原版上做的（时间有点久远既不太清了），所以才测出之前说的Symmetric NAT的问题；而前面回复的测试也是在打了Fullcone补丁但是关闭选项的Openwrt下测试的；复测时应测原版Openwrt更好，毕竟不能保证补丁关闭后完全无影响。等测完Debian 11看看能不能复测下。

escape0707 commented 1 year ago

说实话没看明白你说的测试环境和步骤，你先是一层修改过netfilter的openwrt做nat，然后又用设置了另一个局域网络然后用一个debian机器做默认网关和第二层nat？你的debian机器是公网IP？你10.0.0.0/24的网络用的什么方法组建的？物理直连？

另外如果你找到linux原生masquerade不能Endpoint Independent Mapping的话你应该去netfilter那边发bug了，这特性netfilter是明确支持的

Ovear commented 1 year ago

说实话没看明白你说的测试环境和步骤

请问哪里没看懂？openwrt和debian都有公网权限，openwrt到debian是走的隧道，目前是二层隧道softether。测试时采用的stun服务器是NatTypeTester提供的服务器。

另外如果你找到linux原生masquerade不能Endpoint Independent Mapping的话你应该去netfilter那边发bug了，这特性netfilter是明确支持的

请问这个有资料吗？

escape0707 commented 1 year ago

openwrt和debian都有公网权限，openwrt到debian是走的隧道，目前是二层隧道softether

都有公网权限你为啥还要要 Debian 那台机器，你openwrt用个原版的、直接在 192.168.0.100 上面测不就好了吗。

请问这个有资料吗？

这个你随便搜一下就有，以前邮件列表里主要维护者已经解答过了：https://marc.info/?l=netfilter&m=125491138128462

Ovear commented 1 year ago

刚刚在Vultr上做的快速测试，结论还是一样

测试环境如下

Node1正常购买+开启VPC，WAN IP为x.x.x.x
Node2购买IPv6 Only机型，或者手动关闭IPv4+开启VPC

Node1开启forwarding，同时iptables设定masquerade
Node2新增默认路由通过VPC（网卡2）

测试结果如下

默认测试

root@vultr:~/pystun3# python3 cli.py -d -H stun.syncthing.net
DEBUG:pystun3:Do Test1
DEBUG:pystun3:sendto: ('stun.syncthing.net', 3478)
DEBUG:pystun3:recvfrom: ('139.59.84.212', 3478)
DEBUG:pystun3:Result: {'Resp': True, 'ExternalIP': 'x.x.x.x', 'ExternalPort': 54320, 'SourceIP': '139.59.84.212', 'SourcePort': 3478, 'ChangedIP': '139.59.49.16', 'ChangedPort': 3479}
DEBUG:pystun3:Do Test2
DEBUG:pystun3:sendto: ('stun.syncthing.net', 3478)
DEBUG:pystun3:sendto: ('stun.syncthing.net', 3478)
DEBUG:pystun3:sendto: ('stun.syncthing.net', 3478)
DEBUG:pystun3:sendto: ('stun.syncthing.net', 3478)
DEBUG:pystun3:Result: {'Resp': False, 'ExternalIP': None, 'ExternalPort': None, 'SourceIP': None, 'SourcePort': None, 'ChangedIP': None, 'ChangedPort': None}
DEBUG:pystun3:Do Test1
DEBUG:pystun3:sendto: ('139.59.49.16', 3479)
DEBUG:pystun3:recvfrom: ('139.59.49.16', 3479)
DEBUG:pystun3:Result: {'Resp': True, 'ExternalIP': 'x.x.x.x', 'ExternalPort': 16172, 'SourceIP': '139.59.49.16', 'SourcePort': 3479, 'ChangedIP': '139.59.84.212', 'ChangedPort': 3478}
NAT Type: Symmetric NAT
External IP: x.x.x.x
External Port: 16172

修改版测试

测试描述 Test1:向服务器发起Bind Test2:向服务器发起ChangeAddressAndPort Test3(修改版): 向changed address和原始端口发送Bind Test4(追加): 向服务器发送changePort Test5(pystun3自带)：向changed address/原始端口发送ChangePort Test6(pystun3自带)：向changed address/port发送Bind

DEBUG:pystun3:Do Test1
DEBUG:pystun3:sendto: ('139.59.84.212', 3478)
DEBUG:pystun3:recvfrom: ('139.59.84.212', 3478)
DEBUG:pystun3:Result: {'Resp': True, 'ExternalIP': 'x.x.x.x', 'ExternalPort': 54320, 'SourceIP': '139.59.84.212', 'SourcePort': 3478, 'ChangedIP': '139.59.49.16', 'ChangedPort': 3479}
DEBUG:pystun3:Do Test2
DEBUG:pystun3:sendto: ('139.59.84.212', 3478)
DEBUG:pystun3:sendto: ('139.59.84.212', 3478)
DEBUG:pystun3:sendto: ('139.59.84.212', 3478)
DEBUG:pystun3:sendto: ('139.59.84.212', 3478)
DEBUG:pystun3:Result: {'Resp': False, 'ExternalIP': None, 'ExternalPort': None, 'SourceIP': None, 'SourcePort': None, 'ChangedIP': None, 'ChangedPort': None}
DEBUG:pystun3:Do Test4
DEBUG:pystun3:sendto: ('139.59.84.212', 3478)
DEBUG:pystun3:sendto: ('139.59.84.212', 3478)
DEBUG:pystun3:sendto: ('139.59.84.212', 3478)
DEBUG:pystun3:sendto: ('139.59.84.212', 3478)
DEBUG:pystun3:Result: {'Resp': False, 'ExternalIP': None, 'ExternalPort': None, 'SourceIP': None, 'SourcePort': None, 'ChangedIP': None, 'ChangedPort': None}
DEBUG:pystun3:Do Test3
DEBUG:pystun3:sendto: ('139.59.49.16', 3478)
DEBUG:pystun3:recvfrom: ('139.59.49.16', 3478)
DEBUG:pystun3:Result: {'Resp': True, 'ExternalIP': 'x.x.x.x', 'ExternalPort': 54320, 'SourceIP': '139.59.49.16', 'SourcePort': 3478, 'ChangedIP': '139.59.84.212', 'ChangedPort': 3479}
DEBUG:pystun3:Do Test5
DEBUG:pystun3:sendto: ('139.59.49.16', 3478)
DEBUG:pystun3:sendto: ('139.59.49.16', 3478)
DEBUG:pystun3:sendto: ('139.59.49.16', 3478)
DEBUG:pystun3:sendto: ('139.59.49.16', 3478)
DEBUG:pystun3:Result: {'Resp': False, 'ExternalIP': None, 'ExternalPort': None, 'SourceIP': None, 'SourcePort': None, 'ChangedIP': None, 'ChangedPort': None}
DEBUG:pystun3:Do Test6
DEBUG:pystun3:sendto: ('139.59.49.16', 3479)
DEBUG:pystun3:recvfrom: ('139.59.49.16', 3479)
DEBUG:pystun3:Result: {'Resp': True, 'ExternalIP': 'x.x.x.x', 'ExternalPort': 19882, 'SourceIP': '139.59.49.16', 'SourcePort': 3479, 'ChangedIP': '139.59.84.212', 'ChangedPort': 3478}
NAT Type: Restric Port NAT
External IP: x.x.x.x
External Port: 54320

配置一览

Node1

root@vultr:~# ip -4 ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    inet x.x.x.x/23 brd x.x.x.x scope global dynamic enp1s0
       valid_lft 84898sec preferred_lft 84898sec
3: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc mq state UP group default qlen 1000
    inet 10.1.96.4/20 brd 10.1.111.255 scope global enp6s0
       valid_lft forever preferred_lft forever

root@vultr:~# ip r
default via 66.135.18.1 dev enp1s0 
10.1.96.0/20 dev enp6s0 proto kernel scope link src 10.1.96.4 
x.x.x.x/23 dev enp1s0 proto kernel scope link src x.x.x.x 
169.254.169.254 via 66.135.18.1 dev enp1s0 
root@vultr:~# 
root@vultr:~# iptables-save 
# Generated by iptables-save v1.8.7 on Wed May 10 20:15:37 2023
*filter
:INPUT ACCEPT [674:37696]
:FORWARD ACCEPT [183:80318]
:OUTPUT ACCEPT [623:36135]
COMMIT
# Completed on Wed May 10 20:15:37 2023
# Generated by iptables-save v1.8.7 on Wed May 10 20:15:37 2023
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -o enp1s0 -j MASQUERADE
COMMIT
# Completed on Wed May 10 20:15:37 2023

root@vultr:~# sysctl -p
net.ipv4.ip_forward = 1

root@vultr:~# iptables --version
iptables v1.8.7 (nf_tables)

root@vultr:~# uname -a
Linux vultr 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux

Node2

root@vultr:~/pystun3# ip -4 ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    inet 100.68.124.159/18 brd 100.68.127.255 scope global dynamic enp1s0
       valid_lft 84949sec preferred_lft 84949sec
3: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc mq state UP group default qlen 1000
    inet 10.1.96.3/20 brd 10.1.111.255 scope global enp6s0
       valid_lft forever preferred_lft forever

root@vultr:~/pystun3# ip r
default via 10.1.96.4 dev enp6s0 
10.1.96.0/20 dev enp6s0 proto kernel scope link src 10.1.96.3 
100.68.64.0/18 dev enp1s0 proto kernel scope link src 100.68.124.159 
169.254.169.254 via 100.68.64.1 dev enp1s0 

root@vultr:~/pystun3# iptables-save 
# Generated by iptables-save v1.8.7 on Wed May 10 20:16:08 2023
*filter
:INPUT ACCEPT [83:73113]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [115:8289]
COMMIT
# Completed on Wed May 10 20:16:08 2023

root@vultr:~/pystun3# iptables --version
iptables v1.8.7 (nf_tables)

root@vultr:~/pystun3# uname -a
Linux vultr 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux
escape0707 commented 1 year ago

According to the terminology of RFC 3489, netfilter implements port restricted cone NAT. If the --random flag is specified to the SNAT/MASQUERADE/... targets, it's better described as a symmetric NAT

你得发一下你防火墙怎么配的。

Ovear commented 1 year ago

没有加这个参数的，已经附在上面了。