search

gawindx / WinNUT-Client

This is a NUT windows client for monitoring your ups hooked up to your favorite linux server.
GNU General Public License v3.0
395 stars 70 forks source link

WinNUT client disconnects each night #57

Closed SeeBer61 closed 3 years ago

SeeBer61 commented 3 years ago

I'm running WinNUT-Client for several days now but did not manage to keep it up and running for more than 24hrs. The Client disconnects from the server, process dies (or vice versa - can't tell so far).

Log says "Force Disconnect" - maybe... but that's not me (system itself does not go into hibernate or sleep state and I did NOT force the client to disconnect):

22.01.2021 16:41:24 44184 UPS_Network : Apply Fallback Value when retrieving output.voltage 22.01.2021 16:41:24 44184 UPS_Network : Apply Fallback Value when retrieving battery.capacity 22.01.2021 16:41:24 44184 UPS_Network : Apply Fallback Value when retrieving input.frequency 22.01.2021 16:41:24 44184 UPS_Network : Apply Fallback Value when retrieving output.voltage 22.01.2021 16:41:24 44184 UPS_Network : Deconnect from Nut Host 22.01.2021 16:41:24 44184 UPS_Network : Force Disconnect 22.01.2021 16:41:24 44184 UPS_Network : Deconnect from Nut Host 22.01.2021 16:41:24 44184 UPS_Network : Force Disconnect 23.01.2021 16:43:06 47408 UPS_Network : Connection to Nut Host Established 23.01.2021 16:43:06 47408 UPS_Network : Apply Fallback Value when retrieving battery.capacity

I have re-established connection manually by restarting the client. I have checkmarked "re-establish connection" in the options (just below the "credentials") but this doesn't work whick is an indicator that the process died before the connection was lost.

You can also see that timestamps in the log file do NOT change whilde the client is operating - timestamp show time of startup in every record (it's just by chance that the manual startup time today is the same as yesterday - just a quarter to five approx...)

For WinNUT-client is a nice tool to look into my (remote) UPS but at the time being I can't use it for serious monitoring purposes or emergency shutdown on powerfails. Have to lookout for something else in the meantime but would be more than happy if you could solve (or at least answer) this issue

best - Bernhard

gawindx commented 3 years ago

Hello, Sorry for this late response.

So, there is indeed a bug concerning the logs, but it is not a Timestamp problem, the value you see (44184 or 47408) corresponds to the Pid of the process. However, the time of the event should have been different. This is fixed for v2.0.5

Regarding the disconnection or the crash of the app, I have not yet encountered this problem but I admit that I have not tested it for 24 hours or more either. This is not originally the purpose of this app but I don't see what it wouldn't work for.

Have you looked in the windows logs to see if a particular event matches (or could match) when WinNUT crashes? The same for your NUT server?

I will also explore on my own as it could be a connection terminated problem without it being correctly detected by WinNUT.

Look in your logs and for my part I will come back to you in a few days with a specific version that will incorporate better management of the disconnection of the stream from the NUT server.

SeeBer61 commented 3 years ago

Sorry for being late myself now...

Unfortunately the identical "timestamps" during normal activity of WINnut client do not allow a real tracking - I do not really know the time it crashed (ok - with some further investigation I could checkout the crashdump but I assume this will not be necessary): My router disconnects from the internet for less than a second (this is by design) each morning at 5:53 and I strongly believe that this is also the time when WINnut crashes (however I do not see a good reason for this - my network itself keeps operating - just the router gets a new external IP adress from the ISP)

I'm a bit confused on your statement that it's not the original purpose to permanently monitor the UPS and take preventive action on a powerfail alarm. What other purpose could be more important ??

Meanwhile I', using another solution which does the monitoring and alert job, does not crash overnight, produces exact timestamp resulting in valueable logfiles - but has no Monitoring GUI at all. Seems I have to decide waht is more important for my network: automatic shutdown on powerfail or a nice GUI to look into my UPS. Decided for the first one ....

gawindx commented 3 years ago

I understand your point of view even if I feel a touch of annoyance behind.

Regarding the problem you are encountering, I actually think that there is a link between the behavior of the router and the crash encountered, without having any explanation to provide you.

As for the fact that, at the base, WinNut is not intended to monitor an inverter, it is good that I explain better what I mean.

First, it is the Nut server which monitors the inverter, WinNut is only used to provide a simple visualization tool.

Second, I am not at the origin of this project, I only brought it back to life at a time when I was looking to have a view of the state of my inverter from windows without having to connect to it. a web page or query Nut directly on my server. I added some features and made the sources available for the good of the community.

Personally, it suited me very well, but since it was developed under AutoIt and many people complained that it was identified as unhealthy by antiviruses (because of Autoit), I redefined it under visual basic so as not to be polluted by these regular complaints.

Since then, this project seems to be gaining momentum and, although it is very far from being perfect, I continue to develop it and others are starting to contribute to it as well.

A new version will be available soon and I would be interested to have your feedback when it is released to know if it turns out to be more stable than the previous one (some problems which have been fixed concern connection / reconnection problems).

Over time, it is certain that WinNut will evolve to become a fairly reliable monitoring solution but by my own admission, we are still far from the mark.

SeeBer61 commented 3 years ago

Thanks for your detailed answer.

I’m not annoyed in any way – how could I? Just have some ideas how a SW like this should and could work

I really appreciate that you are doing all the work (I can’t). Honestly I had some experience in SW engineering but that’s way in the past.

I fully understand that WinNUT does not monitor the UPS itself – it just monitors the linux based NUT server which has been built into many NAS products such as synology diskstation’s OS DSM 6.x. So you could say that WinNUT monitors the monitor. Would you agree with this?

In addition WinNUT allows you to set some parameters for automatic shutdown. Do you agree with this as well? This however would not make any sense if WinNUT is not up and running continuously – say: as a background service. Therefor it shouldn’t crash under no circumstances.

The process might lose connection to the NAS if for some reason there is a short interrupt in the network (by the way: none of my SMB connections in my private network ever broke over night) but the process itself should not die: in case of a lost connection it should try to re-establish it.

Just my opinion – just my 2cents …

Last but not least: I know and fully understand that NUT – the Network UPS Tools project – is not yours, you are just contributing to this very interesting and important project.

Thanks again

Bernhard

Von: Decaux Nicolas [mailto:notifications@github.com] Gesendet: Montag, 1. Februar 2021 23:55 An: gawindx/WinNUT-Client WinNUT-Client@noreply.github.com Cc: SeeBer61 info@bbusley.de; Author author@noreply.github.com Betreff: Re: [gawindx/WinNUT-Client] WinNUT client disconnects each night (#57)

I understand your point of view even if I feel a touch of annoyance behind.

Regarding the problem you are encountering, I actually think that there is a link between the behavior of the router and the crash encountered, without having any explanation to provide you.

As for the fact that, at the base, WinNut is not intended to monitor an inverter, it is good that I explain better what I mean.

First, it is the Nut server which monitors the inverter, WinNut is only used to provide a simple visualization tool.

Second, I am not at the origin of this project, I only brought it back to life at a time when I was looking to have a view of the state of my inverter from windows without having to connect to it. a web page or query Nut directly on my server. I added some features and made the sources available for the good of the community.

Personally, it suited me very well, but since it was developed under AutoIt and many people complained that it was identified as unhealthy by antiviruses (because of Autoit), I redefined it under visual basic so as not to be polluted by these regular complaints.

Since then, this project seems to be gaining momentum and, although it is very far from being perfect, I continue to develop it and others are starting to contribute to it as well.

A new version will be available soon and I would be interested to have your feedback when it is released to know if it turns out to be more stable than the previous one (some problems which have been fixed concern connection / reconnection problems).

Over time, it is certain that WinNut will evolve to become a fairly reliable monitoring solution but by my own admission, we are still far from the mark.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gawindx/WinNUT-Client/issues/57#issuecomment-771217564 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ASP6JPYKRC23LK44JEVASZLS44WMLANCNFSM4WPZAQEQ . https://github.com/notifications/beacon/ASP6JP7HR2GLDDOF2YFCWGTS44WMLA5CNFSM4WPZAQE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOFX35RHA.gif

gawindx commented 3 years ago

I had the impression that it annoyed you, but my impression on my part, probably due to the language barrier.

For the points you state, I totally agree and the fact that WinNUT crashes is obviously not the behavior it should have.

Some bugs, as well as cases of figure not thought or encountered during the development are the cause. Recent corrections should address this problem, hoping that none remains.

Small precision, when I mentioned that I revived this project, I was not talking about NUT itself (that would be very pretentious on my part) but of the old WinNUT project which is, it seems to me, still available on sourceforge but not supported for a long time and which is not mine originally.

For the upcoming version, it should be available within a week if all goes well.

gawindx commented 3 years ago

The new version is available, just start the update manually from winNUT and everything should be fine.

If you can test and tell me if you still have the same problem.

SeeBer61 commented 3 years ago

Just installed latest version from github – looks good so far (including correct timestamp / logging)

Gonna keep you up informed on how things are going here …

Best rgds

Bernhard

Von: Decaux Nicolas [mailto:notifications@github.com] Gesendet: Dienstag, 9. Februar 2021 20:11 An: gawindx/WinNUT-Client WinNUT-Client@noreply.github.com Cc: SeeBer61 info@bbusley.de; Author author@noreply.github.com Betreff: Re: [gawindx/WinNUT-Client] WinNUT client disconnects each night (#57)

The new version is available, just start the update manually from winNUT and everything should be fine.

If you can test and tell me if you still have the same problem.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gawindx/WinNUT-Client/issues/57#issuecomment-776173217 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ASP6JP4STOZG7FCCTNDN6UTS6GCFNANCNFSM4WPZAQEQ . https://github.com/notifications/beacon/ASP6JP7FUCUDPZNEZ7EUN3DS6GCFNA5CNFSM4WPZAQE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOFZBXNII.gif

SeeBer61 commented 3 years ago

Hi Nicolas,

I’ve run the latest version now and it still crashes when my router initiates a very short interrupt each morning @5:40 – I do not know why this should affect components in my network (the interrupt is only for a second or so – and results in a new PUBLIC ipv4 address)

NAS (Syno DS200+) and PC are connected via Switch “behind” the router, I do not see a reason why the DHCP Server in the router should not work during this 1 second interrupt … anyway: WinNUT-client always “dies” at the same time my router receives initiates the interrupt from DSL

Unfortunately WinNUT-client does not automatically restart although I’ve marked the checkbox in Connection Settings. Therefor I can’t use it to watch/monitor my systems and automatically shut down in case of a powerfail alarm initiated by the UPS.

Here my logs….

From WinNUT-client log:

11.02.2021 05:41:03Pid: 10908 UPS_Network : Deconnect from Nut Host

11.02.2021 05:41:03Pid: 10908 UPS_Network : Force Disconnect

11.02.2021 05:41:03Pid: 10908 UPS_Network : Deconnect from Nut Host

11.02.2021 05:41:03Pid: 10908 UPS_Network : Force Disconnect

11.02.2021 09:51:00Pid: 18096 UPS_Network : Connection to Nut Host Established

From Router (Overview):

verbunden seit 11.02.2021, 05:40 Uhr

From Router (Event Log):

11.02.21 05:40:17

Verwendetes IPv6-Präfix: 2002:2e54:xxx

IPv6-Präfix used: 2002:2e54:xxx

11.02.21 05:40:17

Internetverbindung IPv6 (Tunnel) wurde erfolgreich hergestellt.

Internet connection IPv6 (tunnel) was successfully established.

11.02.21 05:40:17

Internetverbindung wurde erfolgreich hergestellt. IP-Adresse: 46.84.95.xx, DNS-Server: 217.237.151.115 und 217.237.148.102, Gateway: 62.155.242.156, Breitband-PoP: MUNJ32

Internet connection has been successfully established. IP-Adress: 46.84.95.xx, DNS-Server: 217.237.151.115 and 217.237.148.102, Gateway: 62.155.242.156, Broadband-PoP: MUNJ32

11.02.21 05:40:16

Internetverbindung wurde getrennt.

Internet connection was disconnected

11.02.21 05:40:16

Internetverbindung IPv6 wurde getrennt, Präfix nicht mehr gültig.

Internet connection IPv6 was disconnected, prefix no longer valid.

11.02.21 05:40:16

Die Internetverbindung wird kurz unterbrochen, um der Zwangstrennung durch den Anbieter zuvorzukommen.

The Internet connection is briefly interrupted to pre-empt the forced disconnection by the provider

Best regards from Munich

Bernhard

Von: Decaux Nicolas [mailto:notifications@github.com] Gesendet: Dienstag, 9. Februar 2021 20:11 An: gawindx/WinNUT-Client WinNUT-Client@noreply.github.com Cc: SeeBer61 info@bbusley.de; Author author@noreply.github.com Betreff: Re: [gawindx/WinNUT-Client] WinNUT client disconnects each night (#57)

The new version is available, just start the update manually from winNUT and everything should be fine.

If you can test and tell me if you still have the same problem.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gawindx/WinNUT-Client/issues/57#issuecomment-776173217 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ASP6JP4STOZG7FCCTNDN6UTS6GCFNANCNFSM4WPZAQEQ . https://github.com/notifications/beacon/ASP6JP7FUCUDPZNEZ7EUN3DS6GCFNA5CNFSM4WPZAQE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOFZBXNII.gif

gawindx commented 3 years ago

Well,

it is clear that there is a link, and in any case it gives me clues to try to reproduce the problem.

subsidiary questions:

on the other hand it does not explain that Winnut crashes ...

SeeBer61 commented 3 years ago

Hi,

to answer your questions first:

Maybe I can give you another “clue” – the log from another WinNUT client solution (“WinNUT Configuration Tool”) which seems to be very robust but does not provide a dashboard to look inside the UPS:

Level ALERT 02/11/2021 22:45:18 WinNUTUpsMon Service is starting to monitor UPS

Level ERROR 02/12/2021 05:39:48 Poll UPS [ups@192.168.225.11] failed - Server disconnected

Level NOTICE 02/12/2021 05:39:48 Communications with UPS ups@192.168.225.11 lost

Level ERROR 02/12/2021 05:39:54 UPS [ups@192.168.225.11]: connect failed: Connection failure: Unknown error

Level NOTICE 02/12/2021 05:39:59 Communications with UPS ups@192.168.225.11 established

As you can see the Monitoring Service (could also be configured as a process) lost connection to NUT server [WHY???] but manages to re-establish it. I had this solution running for two weeks without crash

BR

Von: Decaux Nicolas [mailto:notifications@github.com] Gesendet: Donnerstag, 11. Februar 2021 23:36 An: gawindx/WinNUT-Client WinNUT-Client@noreply.github.com Cc: SeeBer61 info@bbusley.de; Author author@noreply.github.com Betreff: Re: [gawindx/WinNUT-Client] WinNUT client disconnects each night (#57)

Well,

it is clear that there is a link, and in any case it gives me clues to try to reproduce the problem.

subsidiary questions:

on the other hand it does not explain that Winnut crashes ...

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gawindx/WinNUT-Client/issues/57#issuecomment-777838552 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ASP6JP2UBGX2VRVZOUPUXRDS6RLWTANCNFSM4WPZAQEQ . https://github.com/notifications/beacon/ASP6JPZNTJQJNWOBTKWIA2DS6RLWTA5CNFSM4WPZAQE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOFZON7WA.gif

gawindx commented 3 years ago

I must admit that I do not understand either why the reconnection fails, so quickly, and that it causes a crash.

When the connection is lost, a process is initiated to make 30 successive attempts at the rate of one every 30 seconds. This leaves a total of 15 minutes before considering the failure as being serious enough to avoid persisting.

For my part, I do not have exactly the same configuration as you:

My server provides me with DHCP on the network. In the containerized services, I have a DNS server (Bind9).

I tried to reproduce by doing the following: 1st test: Network cut by disconnecting the cable connecting my server to the switch (to simulate what can happen when your router restarts Result: no problem, the reconnection process is carried out correctly and the connection is established on the attempt following the reinstatement of the network link.

2nd test: Stopping the DNS and emptying the dns cache of the PC No worries on this side

3rd test DNS and DHCP server cut Always ok

4th test The same as the previous one but in addition I cut the network card of my PC Always ok

So I don't really see where it gets stuck and the logs are not very meaningful given that the app is crashing.

gawindx commented 3 years ago

Small reflection from my side:

Have you checked what happens on your pc when the router restarts?

More precisely the behavior of your network card?

Also on the DiskStation side? do you see illogical behavior at this time?

I suspect you're not going to stay up at 5:40 a.m. just to see it, but do you have the means to change the schedule to see with your own eyes the behavior of each device and see if an interesting element for the future could not come out. ?

SeeBer61 commented 3 years ago

Actually the Router (AVM fritz!box which is extremely popular in Germany) does not restart – it just interrupts and restarts connection to the internet. I can change the time and I can switch this scheduled interrupt OFF (in that case my ISV which is Deutsche Telekom could interrupt my DSL connection once a day – this is a policy to generate a new public/external IP address once a day).

I’m sure that I would not even notice to be OFFLINE for less than one second (in fact, most of my PCs / Servers / even Laptops are running 24/7) – I would rather try to find some abnormalities in a PCs log file. Found only the following critical event (every day at 5.39):

Sorry that I do not have any more clues and hints at the time being….

rgds

Von: Decaux Nicolas [mailto:notifications@github.com] Gesendet: Freitag, 12. Februar 2021 20:26 An: gawindx/WinNUT-Client WinNUT-Client@noreply.github.com Cc: SeeBer61 info@bbusley.de; Author author@noreply.github.com Betreff: Re: [gawindx/WinNUT-Client] WinNUT client disconnects each night (#57)

Small reflection from my side:

Have you checked what happens on your pc when the router restarts?

More precisely the behavior of your network card?

Also on the DiskStation side? do you see illogical behavior at this time?

I suspect you're not going to stay up at 5:40 a.m. just to see it, but do you have the means to change the schedule to see with your own eyes the behavior of each device and see if an interesting element for the future could not come out. ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gawindx/WinNUT-Client/issues/57#issuecomment-778402995 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ASP6JP6SREGL7DP5OYRMLGLS6V6DVANCNFSM4WPZAQEQ . https://github.com/notifications/beacon/ASP6JPZHA5QYI4RWATVAYQ3S6V6DVA5CNFSM4WPZAQE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOFZSXZMY.gif

gawindx commented 3 years ago

I understand well, but you are the only one to tell me such a problem and I try to reproduce what you meet I can't.

So any element that can help me understand the sequence that led to this crash is welcome.

How is the restart action carried out on your router? is it planned in the router? a scheduled power cut? if it's in the router, does it perform a complete restart or only the ADSL part?

if it is a complete restart (by power supply or software), it therefore cuts off the DNS, DHCP, its LAN side network card, its routing table must therefore also be reinitialized and surely also other internal service.

As you have an interlayer switch, you should therefore keep the communication between your pc and your NAS, except this is not the case and it is not only WinNUT impacted since your other software also loses communication with your NAS.

It is this process that I would like to understand in order to be able to reproduce it (if possible) and especially to be able to prevent the exception or the error which leads to the crash.

Could you also provide the 3 logs (router, WinNUT and your other software) for the same night and, not in the form of extraction but rather to deposit in a gists the files (say at least for the periods of 5:00 to 6:00) just to see the exact timeline of what happens to everyone before and after the problem.

SeeBer61 commented 3 years ago

OK – this might be due to the fact, that this “forced disconnect” is no longer needed in German Telekoms VDSL networks. Until a few years ago the DSL line was automatically disconnected by the provider every 24hrs to save public IPv4 addresses. To make sure this forced disconnect does not interrupt your work there is an option in every Fritz!box to initiate this dis-/re-connect in a 1hr timeslot the user could define (I defined a window from 5 to 6 in the morning).

Meanwhile this forced disconnect seems to be no longer necessary in German Telekoms Networks so I disabled this feature in my router now.

Therefore I think WinNUT-client could run smooth and stable from mow on ….

(unfortunately this means we will never find out what’s happening in an environment where router and/or ISP still use forced disconnect. Bit for the moment I think we have better things to do)

THANK YOU !!

Bernhard

Von: Decaux Nicolas [mailto:notifications@github.com] Gesendet: Samstag, 13. Februar 2021 16:04 An: gawindx/WinNUT-Client WinNUT-Client@noreply.github.com Cc: SeeBer61 info@bbusley.de; Author author@noreply.github.com Betreff: Re: [gawindx/WinNUT-Client] WinNUT client disconnects each night (#57)

I understand well, but you are the only one to tell me such a problem and I try to reproduce what you meet I can't.

So any element that can help me understand the sequence that led to this crash is welcome.

How is the restart action carried out on your router? is it planned in the router? a scheduled power cut? if it's in the router, does it perform a complete restart or only the ADSL part?

if it is a complete restart (by power supply or software), it therefore cuts off the DNS, DHCP, its LAN side network card, its routing table must therefore also be reinitialized and surely also other internal service.

As you have an interlayer switch, you should therefore keep the communication between your pc and your NAS, except this is not the case and it is not only WinNUT impacted since your other software also loses communication with your NAS.

It is this process that I would like to understand in order to be able to reproduce it (if possible) and especially to be able to prevent the exception or the error which leads to the crash.

Could you also provide the 3 logs (router, WinNUT and your other software) for the same night and, not in the form of extraction but rather to deposit in a gists the files (say at least for the periods of 5:00 to 6:00) just to see the exact timeline of what happens to everyone before and after the problem.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gawindx/WinNUT-Client/issues/57#issuecomment-778631152 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ASP6JP2JYOZK4BU4DMVIVTTS62IE5ANCNFSM4WPZAQEQ . https://github.com/notifications/beacon/ASP6JP6QMME2BB26XCG3H6TS62IE5A5CNFSM4WPZAQE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOFZUPP4A.gif

gawindx commented 3 years ago

I take good note of it and I close the issue.

If in the future this problem reappears, please provide the elements requested previously to be able to understand what is causing the crash, because without this understanding, I don't really see how to reproduce and fix it.

gawindx commented 3 years ago

The new version is available: Winnut v 2.0.7721

This version incorporates a new functionality in the event of a critical error and offers the possibility of retrieving the various information relating to the error encountered.

If you have the possibility to try again with your system, automatic restart of the router enabled, and to report the info to me that could be useful.

There is no rush but it could be a plus for WinNUT to understand in which functions it gets stuck.