search

zerotier / ZeroTierOne

A Smart Ethernet Switch for Earth
https://zerotier.com
Other
14.53k stars 1.7k forks source link

Zerotier creates huges amouts of UPnP port mappings and crashes the router #2038

Open DontBreakAlex opened 1 year ago

DontBreakAlex commented 1 year ago

Zerotier is basically unusable for me. As you can see on the screenshot, it constantly creates UPnP port mappings with result in the router crashing after 1 or 2 hours.

I am using zerotier 1.10.6 on debian 11.

This may or may not be a bug in zerotier but I can't imagine my ISP fixing its router if it is a bug in their UPnP implementation.

Let me know if you need more info.

Screenshot from 2023-07-05 11-50-15

laduke commented 1 year ago

you can disable upnp with the portMappingEnabled option here: https://github.com/zerotier/ZeroTierOne/tree/dev/service

Haven't seen or heard about this from other folks.

DontBreakAlex commented 1 year ago

Thanks, will do. Is there a way to set the external port to some manual value so that entries are not duplicated but UPnP ? What is the performance cost of disabling UPnP ?

laduke commented 1 year ago

Is there a way to set the external port to some manual value so that entries are not duplicated but UPnP ?

I don't think so.

What is the performance cost of disabling UPnP ?

Behind a ISP router, you probably don't need UPnP. UDP hole punching has a good chance of working without it. ipv6 helps with direct connections too.

Tha14 commented 1 year ago

I also believe that zerotier one creates way too many upnp mappings but I am not sure how many are actually needed for proper operation.

laduke commented 1 year ago

only 1!

Tha14 commented 1 year ago

only 1!

I am seeing way more than one. It's either that it creates too many or it creates a new one each time without expiring the older ones. image

EDIT: Using the windows 10 build v1.10.5.

laduke commented 1 year ago

I can't reproduce this on my routers so it's hard to guess what could be happening.

If anyone is able to compile zerotier, give it a try. It looks like you have to un-comment #define ZT_PORTMAPPER_TRACE 1 in PortMapper.cpp

and then make, sudo ./zerotier-one (stop your installed/system zerotier-one first!)

The code is here https://github.com/zerotier/ZeroTierOne/blob/dev/osdep/PortMapper.cpp

Tha14 commented 1 year ago

I can't reproduce this on my routers so it's hard to guess what could be happening.

If anyone is able to compile zerotier, give it a try. It looks like you have to un-comment #define ZT_PORTMAPPER_TRACE 1 in PortMapper.cpp

and then make, sudo ./zerotier-one (stop your installed/system zerotier-one first!)

The code is here https://github.com/zerotier/ZeroTierOne/blob/dev/osdep/PortMapper.cpp

Sorry to bother you about this but I've made the required change and build the service for windows. I am starting the service on the console now and I'm not sure where the trace log is being saved. I saw that there is data being written into metrics.prom but I'm not sure if that would help you. Please advise. Thanks

laduke commented 1 year ago

It should show in the same console where you start zerotier. Hrm

It should look something like this:

PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e613366b      
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e613366a     
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e6133669        
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:Layer3Forwarding:1
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:WANPPPConnection:1
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:WANCommonInterfaceConf
...

if you already disabled portmapping in your local.conf, maybe you won't see it...

Tha14 commented 1 year ago

It should show in the same console where you start zerotier. Hrm

It should look something like this:

PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e613366b      
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e613366a     
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': uuid:d401ecb1-f4e2-4ef1-aabb-67f7e6133669        
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:Layer3Forwarding:1
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:WANPPPConnection:1
PortMapper: found UPnP device at URL 'http://192.168.82.1:33665/rootDesc.xml': urn:schemas-upnp-org:service:WANCommonInterfaceConf
...

if you already disabled portmapping in your local.conf, maybe you won't see it...

I'm on windows. I'm not sure where local.conf is located so I am still using the default configuration. It's definitely not in my programdata folder. EDIT: Ignore my replies. My problem seems to be irrelevant to what the OP has posted.

Nevcairiel commented 1 year ago

I'm also seeing a new port mapping being generated anytime ZT1 starts up, quickly accumulating over time. Its not finding and re-using the old one, or not clearing the old one in some conditions.

I have turned off port mapping in the meantime, as it seems to work just fine as is.

laduke commented 1 year ago

https://docs.zerotier.com/zerotier/zerotier.conf has the locations of local.conf for various operating systems.

Let us know if you're still seeing the port mapping issues on versions 1.12.1 or later.

It's not impossible it's not fixed, but we can't reproduce this on any of our routers, so can't fix it for sure.

suzikuo commented 11 months ago

I also have related issues. After I joined the local area network, the company's WiFi often couldn't connect and the network was disconnected. Since I shut down/uninstalled it, there have been no issues with the company's WiFi for a month now.

DarkArc commented 6 months ago

I'm seeing this on pfSense 24.03. I suspect this might be related to https://github.com/miniupnp/miniupnp/issues/715.

image image

pfSense moved to FreeBSD 14 last year https://docs.netgate.com/pfsense/en/latest/releases/versions.html while miniupnp seemingly has issues starting with FreeBSD 14.

Are others using BSD based routers/miniupnp based setups?

I'm seeing what (I believe to be) non-ZeroTier PMP activity as well with similar characteristics.

image image

I suspect this isn't a ZeroTier bug at all but an (unfortunately increasingly) widespread issue with this common NAT-PMP and UnPnP implementation.

EDIT: See also https://redmine.pfsense.org/issues/15470