Gateway info is not valid

[rugal0123]

Greeting, i tried to use your libraries to test on my esp32 and somehow end up with this error. I've tried with 2 different router, 1 is my ASUS TM- AC 1900 and the other is my Samsung S7 Mobile HotSpot and Tethering. I've double checked on the require "filled_in" info and tried with different listen port, also tried to change the LEASE_DURATION. Please help i'm getting lost...

[ofekp]

No device on your network responded to the M-SEARCH message.

Can you confirm that your internal network is 192.168.29.0? do the routers have the same address format (except for the last number)? Can you please check that you router has UPnP enabled?

I do not know Samsung S7 Mobile HotSpot but I am guessing it is cellular network? I am not sure this can work since cellular networks to not give you much control, so I am not sure you will be able to check what I asked on this router.

[rugal0123]

Hi ofekp, thanks for the response. I'm able to setup port forwarding manually and verify it's working with my router by connecting to my esp32 from outside network. But i have no luck with trying to implement the libraries. I also set the UpnP feature enable at first. Yes, the Samsung S7 Hotspot is just the cellular network as a 2nd test case for me. I was hoping it should work with my Asus router as first.

[ofekp]

This seems to be related to issue https://github.com/ofekp/TinyUPnP/issues/35 I have merged the code few week ago, but it is not deployed to the Arduino Library Manager yet.

Can you take the code from master branch and try again? Simply use the code as you see it GitHub rather than use the library from the Library Manager.

Let me know how it went please.

[rugal0123]

I just deleted the libraries and used the latest from GitHub as you recommended but still no luck. Same result again, my router completely ignored the M-SEARCH message... I've checked the router log to verify the Upnp feature is enabled

[ofekp]

why is it listening on 5351? Is that the UPnP? should be 1900... Do you have access to 'upnpc' bash program? If so please show me the output for upnpc -l

Make sure it looks like this:

I did see a post about issues with UPnP specifically with your router: https://community.t-mobile.com/thread/140179

Have you tried that firmware update?

[rugal0123]

I just bought a new asus router RT-AX58U to try but still got the same result after update the firmware to latest...

I'm not sure about the 'upnpc' bash program but this is the log report from the router web page after i disabled and re-enabled the UPnP service. No idea why all Asus router keep listening on port 5351 by default... I guess Asus routers doesn't work at all..

[ofekp]

I am not yet sure it actually means they listen on that port, it might mean something else.

Can you please read this short post and perform the actions described in it? https://zapek.com/blog/how-to-fix-upnp-igd-not-working-on-asus-rt-ac87u/

[rugal0123]

Yes i've tried that also but not working either. For the RT-AX58U router, look like they have the "IGMP snooping" feature enabled by default.

[ofekp]

Can you change this line to the 5351 port: https://github.com/ofekp/TinyUPnP/blob/master/src/TinyUPnP.h#L16

If that does not work, please restore 1900 port.

Then, we'll need to set you up with upnpc tool. Do you have a linux env somewhere? Old computer, Raspberry Pi? Anything?

[rugal0123]

Hi ofekp, i did try to mess with the port number from your source code. I just read that Asus router doesn't allow Upnp feature to the WAN, LAN devices only... I guess that's why no matter if i turn on or off Upnp feature there's no response at all... Here's the discussion: https://www.snbforums.com/threads/new-upnp-exploit-affecting-most-asus-routers-upnproxy-blackhat-proxies-via-nat-injections.46011/ Maybe i will swap out with another router with different brand tomorrow to see how's the library works ...

[ofekp]

That looks unfortunate... But since you are contacting the router from the LAN I still don't think it should be a problem. The router should respond to M-SEARCH messages coming from the LAN, wifi or not...

I am willing to keep looking for the issue, but we will need to execute upnpc commands from within the LAN.

Do you have any other device in your network that is using UPnP and puts rules dynamically in your router?

[rugal0123]

yes, you are right. I just tried with a Netgear router and still no different... Currently, i don't have any device that using UPnP in my network.

[ofekp]

The plot thickens! Please describe your network to me. Nice and easy, I want to know how many routers you have specifically. How many switches, how many devices. Anything you can think of please.

I found a workaround for upnpc command in windows environment:

• go to http://miniupnp.free.fr/files/
• look for "latest files"
• in the table, find the one file that is called "upnpc-exe" (upnpc-exe-win32-20150918)
• download the folder and extract it somewhere (no installation needed)
• open win terminal (command prompt)
• enter the following command (change the folder path as needed): C:\Users\user-name>C:\Users\user-name\Downloads\upnpc-exe-win32-20150918\upnpc-shared.exe -l

Then please copy paste the output here.

I still need to know the layout of your network too.

[rugal0123]

I use Comcast cable internet and have only 1 modem and 1 router. Currently running with 1 raspberry Pi that i can connect remotely with tightvnc service, 2 smart plug devices that i can control from anywhere, 1 PS4 and several mobile devices (iphones) and Smart TVs... very standard i think.

[ofekp]

OK, sounds standard, please try the upnpc for windows and and let's see what we get.

[rugal0123]

Hi ofekp, I just ran the tool you suggested and here's the output...

[ofekp]

Made a PR for you, if you feel adventurous (I have yet to test on my device) please give it a shot: https://github.com/ofekp/TinyUPnP/pull/53

Updated the PR. Please make sure to use commit with hash 94dc6cb, thanks.

[rugal0123]

I've compiled and use the new version #94dc6cb you provided but same error...

[ofekp]

Darn, I made a mistake that caused it to behave the same... new hash is b375e92

But please note that the output indicates that you're using the library from Arduino Library Manager and not from the branch I created.

You'll need to go to the folder where the package is and override the code there in both .h file and .cpp file. Folder should look something like this: D:\arduino-1.8.9\libraries\TinyUPnP\src There you'll find the files that you need to change.

[rugal0123]

I've deleted the old library from Arduino Library Manager and use the branch that you created. The output displayed different but still no luck...

[ofekp]

Still timeout? Can you please attach a text file of the output in following posts?

Seems like your Netgear router is also prone to UPnP issues: https://community.netgear.com/t5/General-WiFi-Routers-Non/UPnP-still-broken-on-R6400/m-p/1639401

The claim that: "UPnP is fixed by R6400 firmware version 1.0.1.46."

Can you please check what firmware you're running there?

[rugal0123]

Here's the output file output.txt The version i got is newer 1.0.4.84.

[ofekp]

so no "timeout" message after the M-SEARCH is sent? Did you wait long enough?

[rugal0123]

Yes, i didn't capture all of them in the text file. My bad. This is the full log. output.txt

[ofekp]

Added another device type, please give the PR another try.

[rugal0123]

Just did. Same result. Here's the log. output2.txt

[ofekp]

Can you type in the path to the xml file in your browser that was printed by the upnpc command that you attached in this comment and attach the result here please?

[rugal0123]

Hi ofekp, this is what i got after entered the info you asked logfile.txt

[ofekp]

Now this please: http://192.168.1.1:5000/Public_UPNP_WANIPConn.xml

And this: http://192.168.1.1:5000/Public_UPNP_WANPPPConn.xml

And this: http://192.168.1.1:5000/Public_UPNP_Layer3F.xml

😄

[rugal0123]

Here you go ! logfile_WAN.txt logfile_WANPP.txt UPNP_layer3F.txt

[ofekp]

Thank you :) Can you now make sure the starting address for DHCP protocol that your router will distribute is 10 and above? Meaning 192.168.1.10 will be the first address that any device on your network will receive. Please also make sure your network mask is 255.255.255.0. After this, I would like you to verify that the ESP's IP (from the serial log) is listed as one of the devices in your router. This can be in either the DHCP section which lists devices or under WiFi section. Next, please run the upnpc command again and see if you get the xml file listed again (BTW, that means the router responded to M-SEARH for that particular device), send me the output too please. Then from the same device please ping the ESP device using its IP. Thank you!

[rugal0123]

"Can you now make sure the starting address for DHCP protocol that your router will distribute is 10 and above". Done

"Please also make sure your network mask is 255.255.255.0". Done "Verify that the ESP's IP (from the serial log) is listed as one of the devices in your router". Done. Device ip now is 192.168.1.22. Here's the xml that i found: Gatedesc.txt UPNP_WANPPP.txt UPNP_WANIPConn.txt UPNP_layer3F002.txt Pinged the ESP succesfully:

[ofekp]

Very good.

Can you please check serial log on the ESP after loading the code from master branch?

I am guessing you checked the PR code when you got the new 22 IP and it was still failing.

Please attach the text file.

[rugal0123]

You are right. It's still failing with the code from master branch. Here's the log file. Log_master_branch_error.txt

[ofekp]

Hey, I pushed another debug line and added another header (USER_AGENT) to the PR, please give the same PR a try (hash 71ad57d).

Can you also please run the upnpc command again and send me the output. Thanks.

[rugal0123]

Hi ofekp, here's the new result ( still failed though). Log_error003.txt UPNP_WANIPConn003.txt Gatedesc003.txt UPNP_WANPPP003.txt UPNP_layer3F003.txt

[ofekp]

This is starting to get rather depressing LOL You said you have a RPi, can you ssh to it and run upnpc -l I want to see that the router is replying to ssdp protocol over WiFi too. You may be required to install upnpc, in that case, use this command: sudo apt-get install miniupnpc. Can you attach the output here? Not the xmls but the output of the command. Thank! Don't give up! We'll find it!!

[rugal0123]

Hi ofekp, i have installed the upnpc and here's the response.

[ofekp]

Thanks!

The RPi is connected with WiFi right?

Can you increase the number Advertisement Time To Live (in hops) in your UPnP section in the router?

I see it is set to 4 in this comment Let's change it to 30 (to be on the safe side).

After you change the value, reset the router and wait for things to get back to order. Then, please run the code with both the PR version and then the master version.

At this point, I am pretty sure this is something to do with your network configuration. I hope we will be able to find it.

I compared the UDP packet I am sending with that sent by upnpc tool (which does get a reply from the router) and it is identical down to the character.

OK, let's start with that please.

[rugal0123]

Yes, the RPi is connect to Wifi. I did increase the Advertisement Time To Live (in hops) to 30 as you suggested but still no luck. Maybe you are right this is something to do with network configuration but we have tried two different brands of router. I'm not sure if it's on the router side anymore...

[ofekp]

Yeah, I agree it does not seem like a router issue, rather the setup itself. would you mind disconnecting everything, I mean everything except for the router, modem and ESP?

Few more questions:

1. Do you have repeaters in your setup?
2. Does the modem function as a router too? Any chance it is a fancy modem that also acts as a router?
3. Do you use VPN on some device?

[rugal0123]

1. I only have 1 router to cover the Wifi area, no repeater.
2. The modem is a different device, i can swap out the router easily and leave the modem alone.
3. I don't think i ever used the VPN, only VNC to remote connect to the RPi.

[ofekp]

any other change in the router's configuration you may have made?

Can you look at the router's log while the esp is sending the M-SEARCH message? actually, first with upnpc and then with the esp.

[rugal0123]

I'm not sure there is extra configuration that i set. They are all default setting at first. Here's the log when esp is connecting There's no response or record in router's log when upnpc or esp sending the M-SEARCH message. Also if you look below, there are multiple errors when i ran the upnpc tool

[ofekp]

Yeah, I have seen the errors, been trying to look for them but came up with nothing. That said, the router does respond to M-SEARCH message from upnpc, you know that because it responded with the xml file.

Two options here:

1. The errors are related to the problem with the ESP for some reason
2. They are not related, in which case we will have to deal with these issues after we solve the M-SEARCH issue in the ESP.

I am guessing your windows machine might block UPnP with a firewall which should not be the case with the RPi.

Please run the following command from your windows machine, which should try to open a port in your router and let's see if it made it (I will still want to see the output for every command of course): upnpc -e 'try1' -r 17998 TCP (windows machine)

Please run this command from RPi: upnpc -e 'try2' -r 17999 TCP (rpi)

In addition, please run this command (from your RPi): ip route | head -1 | awk '{print $3}' (rpi)

[rugal0123]

Here's the result for the first command from window machine From the RPi:

[ofekp]

If upnpc has trouble adding a port mapping (please check in the router that it actually failed) then I don't think we should be debugging TinyUPnP for now. I suggest a hard reset for the router, and then let me know exactly what settings you changed. Also give the command a try until you see the upnpc is succeeding. I also suggest disconnecting every device, wired or not wired except for the pc. I can also suggest (and help with) opening an issue to miniupnpc repo. They may be very helpful in this case. First start by looking into similar issues in the repo. I will sign off for now, but may take a look myself tomorrow. For now, do as much of this as you can please.

[rugal0123]

Hi ofekp, good news. I checked the Upnp menu in the router and look like the table has been mapped. I guess the upnpc tool is working somehow...

[ofekp]

Good thing we checked that :) Please try to flash the code from this new PR Please attach the output as text, and wait at least 4 cycles of M-SEARCH message. Thanks

[rugal0123]

Here's the output and define setting, in case you are interested. Same result ... log_3139bf0.txt