Current UPnP integration is completely broken ... despite that it would really simplify deploying Yunohost at home

[alexAubin]

For some random reason, had a look today with ljf :

• the code of firewall_reload calls firewall_upnp which is absolutely not straightforward to understand
• ... and itself call firewall_reload, which is probably among the reasons why firewall_reload execution time is crazy on arm devices
• Kinda reproduced this issue once : tried to enable UPnP once, it did not detect the UPnP interface on the box ... ran it a second time and for some reason it did work ... Detecting the UPnP interface is currently not reliable though it's not clear why.
• Apparently the firewall is blocking relevant traffic for UPnP : https://forum.yunohost.org/t/upnp-bloque-par-le-firewall/2503
• The current behavior of UPnP's implementation is kinda catastrophic, because if it fails to open one port among all the ports that should be opened, it will disable UPnP entirely forever even though it could just be a temporary issue (or the ports are just already taken)
• The documentation or UX about UPnP management in Yunohost is basically nonexistent. Yunohost tries to enable it by default during the postinstall, but there's nothing displayed anywhere about wether it succeeded or not and the user is never invited (in cli or webadmin) to state if UPnP should be working or not - or detect automatically wether it'd be relevant or not (e.g. if we detect that we are on a local home network, and not on a VPS or inside a test LXC)

[decentral1se]

Btw, discovered from #yunohost chat:

The upnp code cannot handle ranges like - 49153:49193:

Action GET /firewall/upnp {"action":"enable"} Traceback Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/moulinette/interfaces/api.py", line 439, in process ret = self.actionsmap.process(arguments, timeout=30, route=_route) File "/usr/lib/python2.7/dist-packages/moulinette/actionsmap.py", line 523, in process return func(**arguments) File "/usr/lib/moulinette/yunohost/firewall.py", line 366, in firewall_upnp if upnpc.getspecificportmapping(port, protocol): TypeError: an integer is required

[llebout]

I wouldnt say the integration is broken. After some testing, the library used seems to throw errors randomly, does not have documentation at all, nor provides explanations for the error it throws. I could see that it seems to be the only UPnP Python library available in Debian Stretch repos. At this point, I would suggest switching to another library because the current one at the version provided by Debian Stretch is certainly unreliable.

[alexAubin]

By the way, the other day I was randomly looking at other Python library for UPnP

i'm randomly working on upnp because i'm again mad about miniupnpc being absolutely unreliable and there seem to be a more robust lib for upnp so wondering if anybody is courageous to try to "pip3 install upnpy" and check what this script returns

import upnpy
import sys

upnp = upnpy.UPnP()

devices = upnp.discover()

if len(devices) >= 1:
    device = devices[0]
else:
    try:
        device = upnp.get_igd()
    except Exception as e:
        print("No UPnP device found")
        sys.exit(-1)

services = device.get_services()

for service in services:
    print("Service found:")
    print(service)
    print("\nAvailable actions:")
    for action in service.get_actions():
        print(action.name)

And Kayou was seeing among other things :

<Service (WANIPConnection) id="WANIPConn">

Available actions:
[...]
GetGenericPortMappingEntry
GetSpecificPortMappingEntry
AddPortMapping
DeletePortMapping
GetExternalIPAddress

which I think it what we're looking for (service supporting Add/Delete PortMapping)

Did not investigate further though ... It's a bit complex for me because my box @ home doesn't support these

[zamentur]

Here is a solution from a user to make it work in its context: https://forum.yunohost.org/t/upnp-bloque-par-le-firewall/2503/8?u=ljf

[freechelmi]

Is there a PR or somebody working on this ? this sounds like a very important feature to me for newbies to selfhost

[alexAubin]

Eh nope I don't think anybody's working on this at the moment :s

[alexAubin]

(Reopening because imho 1109 doesn't fully fix the issue)

[kay0u]

Does https://github.com/YunoHost/yunohost/pull/1154 is ~enough?

[alexAubin]

The recent changes are definitely going in the right direction but imho there's still many issue to be addressed :

• firewall_reload calling firewall_upnp itself calling firewall_reload (essentially "only" a refactoring/performance issue but still worrysome)
• the lib we're using is not reliable ... ? I don't know how the recent changes improve the situation or not (I don't remember if we're using version 2.x now which may help)
• as far as I know Yunohost will still disable upnp forever if it fails once, which is probably not what we want
• overall the current UX / UI is still pretty much catastrophic :/
• personally I still don't have significant knowledge/experience regarding "should UPnP generally work", "how many internet router supports it", or what expectation to have. Like - assuming our UPnP implementation was pretty good - what portion of users would still need to configure port forwarding manually (is it 20% ? 50% ? 80% ?)

[llebout]

@alexAubin

the lib we're using is not reliable ... ? I don't know how the recent changes improve the situation or not (I don't remember if we're using version 2.x now which may help)

I think that's no longer true with current Debian Buster etc. And I am also unsure if my previous tests from a long time ago found out about unreliability in the library or race conditions in Yunohost itself which I may have misdiagnosed.

[dbuscher]

Just to say that the upgrade of the miniupnp library to the latest version seems to have made a difference to the reliability of UPNP on my system - it used to drop out and have to be re-enabled manually about once every week or two. Since the upgrade I have not had a drop out (yet). I may now even be tempted to try and fix the other problems with it, which I thought might have to wait for a transition to Python 3 to use better upnp libraries.

[alexAubin]

Thanks for the feedback! Note that the current unstable (or more precisely dev) branch (to become version 4.2) is now using python 3

[oiseauroch]

We noticed that upnp seems to not be enabled on boot. upnp is enabled on postinstall but change seems not persistent. To get it work I wrote a simple systemd service and enabled it :

[Unit]
Description=Upnp service
Require=yunohost-firewall.service
After=yunohost-firewall.service

[Service]
Type=oneshot
ExecStart=/usr/bin/yunohost firewall upnp enable
ExecStop=/usr/bin/yunohost firewall upnp disable
ExecReload=/usr/bin/yunohost firewall upnp reload
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

[MX10-AC2N]

Hello, I am posting my problem here because it is also a problem with the Upnp ports blocked by the firewall .. I am under the latest version of Yunohost and when I try to activate the Upnp service in the webadmin or in CLI my answer is impossible to open the ports .. When I try to open the UDP 1900 ports, my answer is port 1900 is already closed in IPv6 whereas if I look at the firewall settings port 1900 is open in TCP for IPv4 and IPv6 ..? If I deactivate the yunohost firewall I can finally see my device on my Upnp network => it is therefore the yunohost firewall that is blocking me .. Have some people tried to deactivate the yunohost firewall and manage with a firewall directly installed under debian (ufw)? It's curious all the same the behavior of the yunohost firewall ..! Upnp is not really integrated despite an option to activate it in the parameters (but which does not work ...) Thank you.

[MX10-AC2N]

It works !! Just make the CLI request < sudo yunohost firewall allow UDP 1900 > and Jellyfin finally sees my upnp renderer and stream on it, on the other hand if we reboot we will have to type this command again otherwise we end up with the UDP port 1900 blocked by the firewall ... Mystery but also beginning of resolution ..

[dbuscher]

Thanks for the feedback! Note that the current unstable (or more precisely dev) branch (to become version 4.2) is now using python 3

OK, I have now established that the upgrade of the miniupnpc module fixes some of the key problems I had had when previously trying to fix the reliability problems with upnp, so although the upgrade to python 3 is welcome, it is not strictly needed in this case. I am looking at fixing the remaining problems with dropping out and not starting automatically, plus also the port 1900 conflict mentioned in the most recent message above. If anyone else is also working on this, let me know so that we don't make conflicting changes...

[MX10-AC2N]

Hello @dbuscher , how can we test your solution

[dbuscher]

I think the only way to test is to install this version on a system which is on a network with a UPnP router/gateway, and check that when you open ports on UPNP via the yunohost admin interface/command line that they are visible to the internet (e.g. yunohost diagnosis shows that your web server and mail server are visible from outside your local network). If that test is passed then you can check that the forwarding is restored if you reboot the router and the system (doing so simultaneously simulates a power cut, not so infrequent at my house!). Some routers will also show you which ports are opened under UPNP, and this is a quick way to verify operation.

[dbuscher]

By the way the way I installed my test version was to install yunohost v4.2 and then overwrite the main changed file (firewall.py). This is not entirely kosher, but I think it is valid and much easier than a full build and install.

[hovancik]

Might be related, I am not able to figure out how to turn on UPNP feature: https://forum.yunohost.org/t/not-able-to-turn-on-upnp-after-moving-new-internet-provider/21494