mrhaav / openwrt

18 stars 4 forks source link

Connection fails at certain intervals, I get "Unknown error (REGISTRATION_FAILED)". #5

Open Peacock365 opened 1 year ago

Peacock365 commented 1 year ago

@mrhaav

Dear developer,

I am currently facing an issue with my TP-Link MR6400 V5.2 with OpenWRT 22.03.2 installed.

I have followed the instructions of @fsufyan here and got LTE working in my router:

https://forum.openwrt.org/t/tp-link-tl-mr6400-v5-2-lte-configuration/123603/2

Speed and everything seems to be OK, except reliability. I repeatedly get "Unknown error (REGISTRATION_FAILED)" with both RX / TX going to zero. Any browser then fails to load any website. A restart of the QMI Cellular interface (that is wwan0 here) or a reboot of the router fixes the issue for about 24 hours, after which the error is there again. The error also seems to occur always around the same timeframe within the day, it happens periodically / in intervals. Does the provider change "something" there and the router fails to respond to it everyday? It seems like the router is not able to "fetch" something from the outside.

Please help me kind developer, I have come so far as to get it working (minus the error) despite being a complete noob as far as OpenWRT is concerned, I really want to use this firmware because I do not trust the TP-Link firmware and the TP-Link firmware does not seem to be well-supported at all (no updates after a year, and sparse updates in between).

mrhaav commented 1 year ago

Hi What version of uqmi are you using? It looks like your service provider disconnect you every 24 hour. The latest version, uqmi_2022-09-13-0.9_mipsel_24kc.ipk, has a daemon that check the connectivity and re-connect if you have been disconnected.

Peacock365 commented 1 year ago

So far, I've been using version 22.03.0 of uqmi_2022-09-13-0.9_mipsel_24kc.ipk, because it seemed to be the closest match to the firmware on the router (OpenWRT 22.03.2). Since you have created a new version, I will update to that and see if it fixes anything.

If this helps you identify the cause: Upon the router's reboot or the restarting of the interface, I seem to have a new IP address. Could it be that the old IP address expires periodically and the router fails to fetch a new one? This has been my pet theory, but could be unrelated of course.

Thank you for your support so far & all you have been doing here. I will report back to you on whether or not the update fixed anything.

Peacock365 commented 1 year ago

@mrhaav

I could not update uqmi_2022-09-13-0.9_mipsel_24kc.ipk to version 22.03.2 because the device is telling me that the package is already up to date at root, seems like version 22.03.0 and version 22.03.2 of that file are identical. Do you have any other suggestions?

mrhaav commented 1 year ago

You can re-install the same version with: opkg install uqmi_2022-09-13-0.9_mipsel_24kc.ipk --force-reinstall

Do you still have the syslogs from the time you lost the connection? Can you paste those?

Peacock365 commented 1 year ago

You can re-install the same version with: opkg install uqmi_2022-09-13-0.9_mipsel_24kc.ipk --force-reinstall

I have completely re-flashed my router and was able to install the most recent version of uqmi_2022-09-13-0.9_mipsel_24kc.ipk. Thank you for the hint anyway.

Do you still have the syslogs from the time you lost the connection? Can you paste those?

No, because of the re-flash. However, there are two options here as of now: Either it works now with your new version or I will be able to post system logs when it fails again. If it's the latter, I will be able to provide logs tomorrow.

I have found these links, possibly related:

https://forum.openwrt.org/t/how-to-configure-sim-on-openwrt/45279

https://forum.openwrt.org/t/solved-lte-qmi-troubles/28379

https://forum.openwrt.org/t/troubleshooting-qmi-connection-to-a-lte-modem-does-not-get-ip-address/14926

When you search the following two pages for the term registration_failed, this could explain it too:

https://patchwork.ozlabs.org/project/openwrt/patch/20181104234553.GA6732@makrotopia.org/

https://www.mail-archive.com/openwrt-devel@lists.openwrt.org/msg59235.html

Peacock365 commented 1 year ago

@mrhaav

So, it happened again. I have the log now, what do you make of this? Here it is:

Wed Nov 16 16:11:32 2022 user.notice uqmi_d: Modem disconnected Wed Nov 16 16:11:32 2022 daemon.notice netifd: Interface '4G' has lost the connection Wed Nov 16 16:11:32 2022 daemon.warn dnsmasq[1]: no servers found in /tmp/resolv.conf.d/resolv.conf.auto, will retry Wed Nov 16 16:11:32 2022 daemon.notice netifd: Network device 'wwan0' link is down Wed Nov 16 16:11:32 2022 daemon.info dnsmasq[1]: read /etc/hosts - 4 addresses Wed Nov 16 16:11:32 2022 daemon.info dnsmasq[1]: read /tmp/hosts/dhcp.cfg01411c - 2 addresses Wed Nov 16 16:11:32 2022 daemon.info dnsmasq[1]: read /tmp/hosts/odhcpd - 1 addresses Wed Nov 16 16:11:32 2022 daemon.info dnsmasq-dhcp[1]: read /etc/ethers - 0 addresses Wed Nov 16 16:11:32 2022 user.notice uqmi_d: Unable to re-connect IPv4 - Interface restarted Wed Nov 16 16:11:32 2022 daemon.notice netifd: 4G (18168): Stopping network 4G Wed Nov 16 16:11:33 2022 daemon.notice netifd: 4G (18168): Command failed: ubus call network.interface notify_proto { "action": 0, "link-up": false, "keep": false, "interface": "4G" } (Permission denied) Wed Nov 16 16:11:33 2022 daemon.notice netifd: Interface '4G' is now down Wed Nov 16 16:11:33 2022 daemon.notice netifd: Interface '4G' is setting up now Wed Nov 16 16:11:43 2022 daemon.notice netifd: 4G (18257): PINcode disabled Wed Nov 16 16:11:43 2022 daemon.notice netifd: 4G (18257): Data format set to raw-ip Wed Nov 16 16:11:44 2022 daemon.notice netifd: 4G (18257): Default profile: 1 Wed Nov 16 16:11:44 2022 daemon.notice netifd: 4G (18257):  searching on 26203 Wed Nov 16 16:11:46 2022 daemon.notice netifd: 4G (18257):  searching on 26203 Wed Nov 16 16:11:48 2022 daemon.notice netifd: 4G (18257):  searching on 26203 Wed Nov 16 16:11:50 2022 daemon.notice netifd: 4G (18257):  searching on 26203 Wed Nov 16 16:11:53 2022 daemon.notice netifd: 4G (18257):  searching on 26203 Wed Nov 16 16:11:57 2022 daemon.notice netifd: 4G (18257):  searching on 26203 Wed Nov 16 16:12:00 2022 daemon.notice netifd: 4G (18257):  searching on 26203 Wed Nov 16 16:12:03 2022 daemon.notice netifd: 4G (18257):  searching on 26203 Wed Nov 16 16:12:09 2022 daemon.notice netifd: 4G (18257):  searching on 26203 Wed Nov 16 16:12:15 2022 daemon.notice netifd: 4G (18257):  searching on 26203 Wed Nov 16 16:12:15 2022 daemon.notice netifd: 4G (18257): Unable to register to o2 - de on LTE Wed Nov 16 16:12:15 2022 daemon.notice netifd: 4G (18257): Check subscription or APN settings Wed Nov 16 16:12:15 2022 daemon.notice netifd: 4G (18405): Stopping network 4G Wed Nov 16 16:12:16 2022 daemon.notice netifd: 4G (18405): Command failed: ubus call network.interface notify_proto { "action": 0, "link-up": false, "keep": false, "interface": "4G" } (Permission denied) Wed Nov 16 16:12:16 2022 daemon.notice netifd: Interface '4G' is now down

If you need more, please tell me.

mrhaav commented 1 year ago

For some reason it take more then 35s to register. I have seen this before, but I don´t understand why. You can let the modem search forever with switch:

uci set network.4G.abort_search=false
uci commit network

That may help.

Peacock365 commented 1 year ago

Noob alert: Where can I input those commands?

I only know how to handle LuCi, I need further instructions on how to input that. As I said, OpenWrt noob here.

mrhaav commented 1 year ago

You need to access your router with an ssh client, like PuTTY, and your router IP-address. image Login in with user: root and your password. You can print your modem configuration with: uci show network.4G and then run

uci set network.4G.abort_search=false
uci commit network
Peacock365 commented 1 year ago

@mrhaav

So far it seems stable, I will test for a few days and report back to you if the error occurs again. Thank you for your suggestion.

I've opted for luci-app-commands to input your commands, because that makes it easier for me to add further commands as needed on the fly.

mrhaav commented 1 year ago

Hi I´m just little curious how the re-connections work. Can you paste the re-connection from syslog?

Peacock365 commented 1 year ago

I am currently away, I will post the re-connection log later today or tomorrow. So far so good, your lines have worked for me. Do they survive a restart, or do the lines have to be input every time?

mrhaav commented 1 year ago

Ok, great to hear. Yes,

uci set network.4G.abort_search=false
uci commit network

will survive a restart.

Peacock365 commented 1 year ago

Router is up for 2 days 1 hour now without interruptions. I am getting this log:

Tue Nov 22 13:48:37 2022 user.notice uqmi_d: Modem disconnected Tue Nov 22 13:48:37 2022 daemon.notice netifd: Interface '4G' has lost the connection Tue Nov 22 13:48:37 2022 daemon.warn dnsmasq[1]: no servers found in /tmp/resolv.conf.d/resolv.conf.auto, will retry Tue Nov 22 13:48:37 2022 daemon.notice netifd: Network device 'wwan0' link is down Tue Nov 22 13:48:38 2022 user.notice uqmi_d: IPv4 re-connected Tue Nov 22 13:48:38 2022 daemon.notice netifd: Interface '4G' is now up Tue Nov 22 13:48:38 2022 daemon.notice netifd: Network device 'wwan0' link is up Tue Nov 22 13:48:38 2022 daemon.info dnsmasq[1]: reading /tmp/resolv.conf.d/resolv.conf.auto Tue Nov 22 13:48:38 2022 user.notice uqmi_d: IPv6 re-connected

mrhaav commented 1 year ago

Looks good! Do you know if any of your disconnections caused our modem to do a new network search?

Peacock365 commented 1 year ago

Router is up and running for 8 days now without interruption, seems like you fixed my issue with your suggestion, many thanks!

Do you know if any of your disconnections caused our modem to do a new network search?

What am I looking for in the logs? I am getting the last log I've posted here sometimes, and otherwise only IP exchanges between my devices and the router. I am getting nothing like my first protocol / log anymore. I got something similar to my first protocol / log days ago, when I restarted the router (after having applied your lines of code), later I didn't get that anymore.

Peacock365 commented 1 year ago

@mrhaav

You've updated your custom version of uqmi 3 days ago, did that do anything? I've updated to the new version.

mrhaav commented 1 year ago

In uqmi_2022-11-29-0.10 i have done corrections for long plmn_descriptions, in --get-serving-system, and when you run IPv6 only. It should not affect your setup,

Is your router still connected?

Peacock365 commented 1 year ago

OK, thanks for the info.

Is your router still connected?

Yes. I think your commands addressing the timeout did fix the issue. I will observe for 1 - 2 weeks more and then close the issue I think. Many thanks to you so far.

Peacock365 commented 1 year ago

@mrhaav

The router has been stable and connected for 30+ days now, I think we can close the issue now. I am very thankful to you for your fix, and everything else you have done so far to make this router model work! If I can do something for you, just let me know. Resolved fixed!

Peacock365 commented 1 year ago

Sorry, spoke too soon. I had an extended period of approx. 55 days where nothing happened, but now the daily disconnect is back. However, now the router seems to be stuck with...

daemon.notice netifd: 4G (18257): searching on 26203

...forever. This appears again and again in the log, but no reconnect happens, ever. No error code either, I am not getting _Unknown error (REGISTRATIONFAILED) anymore, presumably due to the searching never ending.

Restarting the QMI Cellular interface does nothing anymore, it just resumes the searching. A restart of the router makes the router reconnect, until it disconnects again 24 hours later. Do you have any other proposals, any idea what else could it be?

mrhaav commented 1 year ago

Sorry to hear. I have seen others with the same problem that the modem start to search for network after disconnections. But, I don´t understand why.

We could try one thing: Can you add uqmi -d "$device" --set-device-operating-mode low_power in /usr/bin/uqmi_d.sh at the IF statement with "Called failed"? This will put the modem in Airplane mode and a new attach will be performed when the interface is brought up.

# IPv4
.......
            if [ "$pdh_4" = '"Call failed"' ]
            then
                logger -t uqmi_d 'Unable to re-connect IPv4 - Interface restarted'
                uqmi -d "$device" --set-device-operating-mode low_power
                ifup $interface
                /etc/init.d/uqmi_d stop
            else
                logger -t uqmi_d IPv4 re-connected
            fi

# IPv6
........
            if [ "$pdh_6" = '"Call failed"' ]
            then
                logger -t uqmi_d 'Unable to re-connect IPv6 - Interface restarted'
                uqmi -d "$device" --set-device-operating-mode low_power
                ifup $interface
                /etc/init.d/uqmi_d stop
            else
                logger -t uqmi_d IPv6 re-connected
            fi
Peacock365 commented 1 year ago

Can you add uqmi -d "$device" --set-device-operating-mode low_power in /usr/bin/uqmi_d.sh at the IF statement with "Called failed"?

How do I do that?

I've noticed one further thing: The router seems to reconnect by itself after 6 - 7 minutes. Now, of course I can live without Internet for 6 - 7 minutes, it is just incredibly annoying when you download a game for example and that fails. Further, if you can't find the reason for the issue, would it be possible to report this issue to the device maintainer? Or is the problem uqmi-related to a degree where that doesn't make sense?

mrhaav commented 1 year ago

Hi I'm on holiday trip and will back mid next week. When I'm back I can create a new package with the modifications.

Peacock365 commented 1 year ago

When I'm back I can create a new package with the modifications.

That's great to hear, many thanks. Enjoy your holidays!

mrhaav commented 1 year ago

Hi I have made a new package with the modified uqmi_d.sh for test. It´s available for download at:

wget https://github.com/mrhaav/openwrt/raw/master/uqmi_test/uqmi_2022-11-22-0.11_mipsel_24kc.ipk
opkg install uqmi_2022-11-22-0.11_mipsel_24kc.ipk
Peacock365 commented 1 year ago

OK, I have updated your custom uqmi to the new version. The router has reverted back to normal for a few days before that, so no interruption / disconnect after 24 hours anymore. It was working again. But the intermittent failure to work that I've reported recently still seems to make this new fix necessary. All in all, I think we were already 75% there even before the new fix. Thank you for the new package! I am testing now.

mrhaav commented 1 year ago

Hi Can you run logread | grep 'uqmi\|4G'? I´m just curious if my modification made any difference.

noMaaam commented 1 year ago

Hello,

i had the exact same problem with the reconnect after 24h. My ISP is same as yours, its O2 Germany. I tried a lot of things but the only one that worked for me so far was forcing the plmn.

if you want to try just add this option to etc/config/network

config interface '4G' option plmn '26203' <-

Peacock365 commented 1 year ago

@noMaaam

A fellow O2 Germany customer, with the same router + OpenWRT too! Can't believe it!

I appreciate your advice, but you are overlooking that I am a noob as far as OpenWRT is concerned. I was capable of getting it installed and getting the needed packages and that's it. I don't know how I can edit etc/config/network! Can you explain this to me? Also, if this is something that @mrhaav can potentially include in the package by default, I would appreciate that.

mrhaav commented 1 year ago

@noMaaam Do you mean that your disconnections disappeared or that your re-connections worked? What uqmi version do you use?

@Peacock365 Is there any difference in behavior in the new package?

You can add the forced PLMN settingw with:

uci set network.4G.plmn=26203
uci commit network
Peacock365 commented 1 year ago

@mrhaav

Thank you for telling me how to apply it.

Is there any difference in behavior in the new package?

Unsure. I got one disconnect with the updated package so far. On the other hand, before that and after that I have had a stable network again, in both cases for days. If I had to answer, I would say the new package was neither good nor bad.

You wanted me to run logread | grep 'uqmi|4G' - I don't know how that's done.

mrhaav commented 1 year ago

Your disconnections will not disappear. The change in the new package will try to prevent that you where stuck in searching on 26203.

Can you paste the re-connection part from your syslog?

Peacock365 commented 1 year ago

Your disconnections will not disappear. The change in the new package will try to prevent that you where stuck in searching on 26203.

I know this. I understand that our goal here is a disconnect with immediate reconnect. What I tried to say was that I got a permanent disconnect once with the new package.

Can you paste the re-connection part from your syslog?

Sure, I can. From very recently (note that I've recently disabled IPv6):

Thu Feb 23 21:02:36 2023 user.notice uqmi_d: Modem disconnected Thu Feb 23 21:02:36 2023 daemon.notice netifd: Interface '4G' has lost the connection Thu Feb 23 21:02:36 2023 daemon.warn dnsmasq[1]: no servers found in /tmp/resolv.conf.d/resolv.conf.auto, will retry Thu Feb 23 21:02:36 2023 daemon.notice netifd: Network device 'wwan0' link is down Thu Feb 23 21:02:36 2023 user.notice uqmi_d: IPv4 re-connected Thu Feb 23 21:02:36 2023 daemon.notice netifd: Interface '4G' is now up Thu Feb 23 21:02:36 2023 daemon.info dnsmasq[1]: reading /tmp/resolv.conf.d/resolv.conf.auto Thu Feb 23 21:02:37 2023 user.notice firewall: Reloading firewall due to ifup of 4G (wwan0)

mrhaav commented 1 year ago

Do you have syslog from the permanent disconnection?

noMaaam commented 1 year ago

Do you mean that your disconnections disappeared or that your re-connections worked? What uqmi version do you use?

it´s still disconnecting after 24h but uqmi can get a new IP adress without scanning the network, i think.

im using version uqmi_2022-11-29-0.10_mipsel_24kc.ipk

Peacock365 commented 1 year ago

Do you have syslog from the permanent disconnection?

Unfortunately not. I can wait for another permanent disconnect in order to "catch it", so to speak.

Isn't the solution of @noMaaam the fix we are looking for?

mrhaav commented 1 year ago

Yes, try @noMaaam´s solution.

Peacock365 commented 1 year ago

OK, I will try this fix and report back to this thread after some time. I would also report back in case of another permanent disconnect!

Peacock365 commented 1 year ago

@mrhaav

Quick update, it works flawlessly for two weeks now with your new package and the forced plmn suggested by @noMaaam. Will continue to observe it, but seems good so far.

mrhaav commented 1 year ago

Great to hear that @noMaaam solution is working.

Peacock365 commented 1 year ago

@mrhaav

So far, no issues anymore with your test version of uqmi and the forced plmn. I think I'll close this issue now unless you need further information from me. Thank your for helping me, thank you also to @noMaaam!

Peacock365 commented 9 months ago

Hi @mrhaav

sadly I have another problem, this device truly is cursed it seems. I am geting frequent connection losses over the day, but the connection to my devices recovers within a minute. Before the connection loss, the system log always shows:

Sat Oct 21 11:20:15 2023 daemon.info hostapd: wlan0: STA XX:XX:XX:XX:XX:XX IEEE 802.11: disassociated Sat Oct 21 11:20:16 2023 daemon.info hostapd: wlan0: STA XX:XX:XX:XX:XX:XX IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)

(Redacted MAC address)

Solutions I have already tried:

uci set wireless.@wifi-iface[0].disassoc_low_ack=0 uci commit wireless wifi

This is suggested here, for this error:

https://openwrt.org/faq/deauthenticated_due_to_inactivity

No joy, it still happens. Another solution has been suggested at the bottom of this page:

https://github.com/openwrt/mt76/issues/169

Perhaps my attempted fix does not work due to this:

https://reddit.com/r/openwrt/comments/lgsqsn/newifid2_deauthenticated_due_to_inactivity/gmtglvl/

How do I do this? Thank you in advance.

mrhaav commented 9 months ago

Sorry to hear that you have issues with your router. I don´t use wireless on my MR200v4, so I have no experiences about the wireless part.

I didn´t understand what part you had problem to test. Setting option disassoc_low_ack '0', or?

Have you raised a question on the Openwrt forum?

Peacock365 commented 9 months ago

I didn´t understand what part you had problem to test. Setting option disassoc_low_ack '0', or?

No, I have successfully set this setting, however, apparently, it doesn't work as long as "option country" exists in the wireless config, see this comment and the ones immediately preceding it:

https://github.com/openwrt/mt76/issues/169#issuecomment-731795422

The comment below that suggests yet another solution, which I don't know how to set. I would need the fitting commands for both.

Have you raised a question on the Openwrt forum?

No, and I will tell you why: Apparently, the chipset in this router is only supported in a very mediocre manner (not even WiFi works error-free) by OpenWRT, and no one seems interested in fixing the issues even though the exact same issue I have has already been raised there, multiple times. So I see no point in adding my voice to the choir, I have also not yet tried all workarounds (see above).

With your help and the help of @noMaaam, I was able to fix the issue where the router disconnected whenever a new IP address was assigned. This is resolved fixed for me. But apparently, there is an issue with the WiFi driver here that still leads to disconnects, unrelated to the LTE connection.

noMaaam commented 9 months ago

Hi,

i dont have this issue with the MR6400 v5, but i have 3 RTSP streams with wifi running 24/7. Maybe if you keep your wifi "busy" with traffic it doesnt drop the connection. Have you tried 23.05.0 ? There is a lot of MT76 new Drivers and Patches comming with this version. But be aware, uqmi version 22.03.5 cant be installed on 23.05.0 because some package dependencies are newer it seems. For 4G reconnect i use watchcat until a new version is being released.

mrhaav commented 9 months ago

Hi

I have compiled uqmi for 23.05.0 now. I have not jet tested it.

noMaaam commented 9 months ago

Hi @mrhaav thank you for your work with uqmi. Installation of the new version was fine, now the daemon is back and i dont have to use watchcat for reconnecting.

noMaaam commented 8 months ago

Hi @mrhaav ,

there is a problem with uqmi and 23.05.2. The deamon seems to not do anything when connection is down. I have to restart the interface by hand to renew IP adress after disconnect.

Tue Nov 14 14:39:38 2023 user.notice uqmi_d: Daemon started Sun Nov 19 13:59:16 2023 user.notice uqmi_d: Daemon stopped, interface down Sun Nov 19 13:59:32 2023 user.notice uqmi_d: Daemon started

mrhaav commented 8 months ago

Hi @noMaaam

Please, try the 23.05.2 version. It´s now uploaded.