search

lumapu / ahoy

Various tools, examples, and documentation for communicating with Hoymiles microinverters
https://ahoydtu.de
Other
947 stars 222 forks source link

Time synchronisation issue - SYNC NTP not working? #642

Closed E1nste1n-AUT closed 1 year ago

E1nste1n-AUT commented 1 year ago

Hardware

nRF24L01+ Module

Antenna:

Power Stabilization:

no additional capacitor used (so far)

Connection diagram I used:

nRF24L01+ Pin ESP8266 GPIO
Pin 1 GND [*] GND
Pin 2 +3.3V +3.3V
Pin 3 CE GPIO2 CE D4
Pin 4 CSN GPIO15 CS D8
Pin 5 SCK GPIO14 SCLK D5
Pin 6 MOSI GPIO13 MOSI D7
Pin 7 MISO GPIO12 MISO D6
Pin 8 IRQ GPIO0 IRQ D3

Software

Version / Git SHA:

Version: 0.5.78 Github Hash: 8457f46

Settings

NTP Server IP: at.pool.ntp.org NTP Port: 123

Coordinates for Sunrise&Sunset entered (with 'dot' as decimal separator) Disable night communication NOT checked

MQTT configured (works) ###################################################

I have used the latest release 0.5.66 and switched to the current dev version 0.5.78, but I still have problems with time synchronisation.

After every reset the ESP-Time is not synchronised. The ahoyDTU start page shows "NTP timeserver unreachable". The only way to fix this is by clicking the "SYNC FROM BROWSER" button. The synchronysation buttons in the "NTP Server" settings menu "FROM BROWSER" and "SYNC NTP" do not work?! And I can confirm the "Polling inverter stop at...." being displayed although the night communication is not disabled (but that has already been addressed by somebody else)

I want to use a NTP Server but over night it seems that the time synchronisation is lost and every morning I'd have to re-sync the ESP-Time by clicking the SYNC FROM BROWSER button - I hope this is not intended? ;)

The problem is without correct ESP-Time my Grafana dashboard won't display any values. Therefore I hoped I could use a NTP server for synchronisation and the ahoyDTU will be up & running for days, weeks, months,...

Is there the possibility to implement a time-sync on a regular basis? Or do you have other suggestions for me, to avoid every day a manual re-sync?

DanielR92 commented 1 year ago

please make sure your firewall have allowed to get the time from the timeserver. Alternative, when its okay for you... get it from your gateway-router.

E1nste1n-AUT commented 1 year ago

Good point, but I'm pretty sure, that the firewall won't block the connections via port 123. (pinging the used ntp server works) But I will try to use an internal NTP server, either on my QNAP NAS or via the gateway router.

I will comeback on Monday and report which solution worked out and which not.

beegee3 commented 1 year ago

is at.pool.ntp.org correct? please try pool.ntp.org (without at. in front) as recommended by https://www.pool.ntp.org/zone/at :

Austria ā€” at.pool.ntp.org
To use this specific pool zone, add the following to your ntp.conf file:

       server 0.at.pool.ntp.org
       server 1.at.pool.ntp.org
       server 2.at.pool.ntp.org
       server 3.at.pool.ntp.org
In most cases it's best to use pool.ntp.org to find an NTP server (or 0.pool.ntp.org, 1.pool.ntp.org, etc if you need multiple server names). The system will try finding the closest available servers for you. If you distribute software or equipment that uses NTP, please see our [information for vendors](https://www.pool.ntp.org/vendors.html).
E1nste1n-AUT commented 1 year ago

Thanks for the hint! It didn't really work in the first place with the default ntp server (pool.ntp.org) therefore I changed it, but obviously I made it even worse...

But is it only me or is it intended, that ahoyDTU is not reachable (21:30 CET)? I had to restart the ESP8266 to change the ntp server. Nevertheless, I tried to ping the ntp.pool.org which refers to IP address 195.168.4.100 and it was successful. So I entered this IP in the ahoyDTU NTP Server settings and clicked "SYNC NTP" and got a "command executed".

I'll keep you updated, if it's now running stable also over night without any time sync issues.

beegee3 commented 1 year ago

to my opinion it's better to use the name of the ntp server not the IP address (which may change). And as said above try again the default pool.ntp.org instead of ntp.pool.org. Normally there is no problem. I prefer fritz.box with ntp enabled in the fritzbox settings, because it's always available. Maybe your router has this feature too. Finally, sometimes ahoyDTU seems to be frozen (very rare), but that's one of the points why the developers are working on it. It's getting better with each new version šŸ˜ƒ

lumapu commented 1 year ago

@E1nste1n-AUT is it solved? Or is there any task for us?

E1nste1n-AUT commented 1 year ago

Eventually, I still need to click at least once a day (in the morning) on the ahoyDTU start page the "SYNC FROM BROWSER" button. Otherwise I would not be able to see all the data in my Grafana dashboard, which is annoying.

So, I think the sync feature with an NTP server works, if the ESP-time is available/valid. But for whatever reason every day in the morning the ESP-time is not valid.

For me personally also a workaround would be fine, like periodically running the "SYNC FROM BROWSER" function automatically, either after every reboot or after a defined period (6h,12h,24h?)

Would this be possible or do you have any other idea, how to solve my issue?

Of course I know ahoyDTU is still under development and I cannot expect it to be working flawlessly at the moment. But is this "instability" issue is known, that wenn time sync is lost ahoyDTU shows no live data and needs a reboot (followed by synchronisation of the ESP-time)?

E1nste1n-AUT commented 1 year ago

Unfortunately, I still have to click "SYNC FROM BROWSER" every morning, although I use a NTP Server and have clicked on "SYNC NTP" in the settings menu. Btw I'm still using 0.5.78.

Is there a possible improvement in any newer version regarding this issue? Because then I would simply upgrade and test it.

Otherwise could you include that the ESP just uses the browser time directly after startup? Or would that cause maybe some other problems?

beegee3 commented 1 year ago

at the moment you are the only one having such a problem. I think there are some faults in your settings. Does your router allow your ESP to connect to an address outside your local network? What is your current NTP server setting? It would help if you could provide an output of the serial debug at boot time to see what happens when th ESP tries to get the NTP time.

E1nste1n-AUT commented 1 year ago

OK, I tried to capture the serial output during reboot:

`I: Rebooting...

ets Jan 8 2013,rst cause:2, boot mode:(3,7)

load 0x4010f000, len 3460, room 16 tail 4 chksum 0xcc load 0x3fff20b8, len 40, room 4 tail 4 chksum 0xc9 csum 0xc9 v00080d20 ~ld I: Initializing FS .. I: .. done I: Settings valid: true I: RF24 Amp Pwr: RF24_PA_I: MAX I: Radio Config: SPI Frequency = 1 Mhz Channel = 3 (~ 2403 MHz) Model = nRF24L01+ RF Data Rate = 250 KBPS RF Power Amplifier = PA_MAX RF Low Noise Amplifier = Enabled CRC Length = 16 bits Address Length = 5 bytes Static Payload Length = 32 bytes Auto Retry Delay = 250 microseconds Auto Retry Attempts = 0 maximum Packets lost on current channel = 0 Retry attempts made for last transmission = 15 Multicast = Disabled Custom ACK Payload = Disabled Dynamic Payloads = Enabled Auto Acknowledgment = Disabled Primary Mode = RX TX address = 0xdeadbeef01 pipe 0 (closed) bound = 0xdeadbeef01 pipe 1 ( open ) bound = 0x1106738001 pipe 2 (closed) bound = 0xc3 pipe 3 (closed) bound = 0xc4 pipe 4 (closed) bound = 0xc5 pipe 5 (closed) bound = 0xc6

AhoyDTU Info: Version: 0.5.78 Github Hash: 8457f46

AP MODE SSID: AHOY-DTU PWD: esp_8266 IP Address: http://192.168.4.1

connect to network '' ... I: added inverter 116183062185 I: resetPayload: id: 0 I: resetPayload: id: 1 I: resetPayload: id: 2 I: resetPayload: id: 3 I: resetPayload: id: 4 I: resetPayload: id: 5 I: resetPayload: id: 6 I: resetPayload: id: 7 I: resetPayload: id: 8 I: resetPayload: id: 9 reconnect in 9 seconds I: Iv: 0 I: reconnect in 8 seconds I: Iv: 0 I: reconnect in 7 seconds I: Iv: 0 I: reconnect in 6 seconds I: Iv: 0 I: reconnect in 5 seconds I: Iv: 0 I: reconnect in 4 seconds I: Iv: 0 I: reconnect in 3 seconds I: Iv: 0 I: reconnect in 2 seconds I: Iv: 0 I: reconnect in 1 seconds I: Iv: 0 I: scanning APs with SSID I: Iv: 0 I: reconnect in -1 seconds I: Iv: 0 I: reconnect in -2 seconds I: Iv: 0 I: reconnect in -3 seconds BSSID 0: 24 5a 4c 79 ed a BSSID 1: 24 5a 4c 79 bf 8 BSSID 2: 78 8a 20 5a bb b1 I: Iv: 0 I: reconnect in -4 seconds I: Iv: 0 I: reconnect in -5 seconds I: Iv: 0 I: reconnect in -6 seconds I: Iv: 0 I: reconnect in -7 seconds I: Iv: 0 I: reconnect in -8 seconds I: Iv: 0 I: reconnect in -9 seconds I: Iv: 0 I: reconnect in -10 seconds try to connect to AP with BSSID: 24 5a 4c 79 ed a I: Iv: 0 I: reconnect in 9 seconds I: Iv: 0 I: reconnect in 8 seconds I: Iv: 0 I:

[WiFi] Connected

Welcome to AHOY!

point your browser to http://192.168.0.222 (Station) to configure your device

[WiFi] AP disabled I: Iv: 0 I: I: Iv: 0 I: I: Iv: 0 I: I: Iv: 0 I: I: MQTT connected I: resetPayload: id: 0 I: (#0) Requesting Inv SN I: (#0) enqueuedCmd: 0x1 I: (#0) enqueuedCmd: 0xb I: (#0) sendTimePacket I: TX 27B Ch40 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 42 00 00 00 00 00 00 00 00 73 9C 30 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0x1 I: TX 27B Ch61 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 42 00 00 00 00 00 00 00 00 73 9C 30 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0x1 I: TX 27B Ch75 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 42 00 00 00 00 00 00 00 00 73 9C 30 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0x1 I: TX 27B Ch3 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 42 00 00 00 00 00 00 00 00 73 9C 30 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0x1 I: TX 27B Ch23 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 42 00 00 00 00 00 00 00 00 73 9C 30 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0x1 I: TX 27B Ch40 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 42 00 00 00 00 00 00 00 00 73 9C 30 I: enqueued cmd failed/timeout I: (#0) I: no Payload received! (retransmits: 5) I: resetPayload: id: 0 I: (#0) Requesting Inv SN I: (#0) sendTimePacket I: TX 27B Ch61 | 15 83 06 21 85 80 73 06 11 80 0B 00 63 E9 46 60 00 00 00 00 00 00 00 00 D8 16 39 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0xb I: TX 27B Ch75 | 15 83 06 21 85 80 73 06 11 80 0B 00 63 E9 46 60 00 00 00 00 00 00 00 00 D8 16 39 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0xb I: TX 27B Ch3 | 15 83 06 21 85 80 73 06 11 80 0B 00 63 E9 46 60 00 00 00 00 00 00 00 00 D8 16 39 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0xb I: TX 27B Ch23 | 15 83 06 21 85 80 73 06 11 80 0B 00 63 E9 46 60 00 00 00 00 00 00 00 00 D8 16 39 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0xb I: TX 27B Ch40 | 15 83 06 21 85 80 73 06 11 80 0B 00 63 E9 46 60 00 00 00 00 00 00 00 00 D8 16 39 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0xb I: TX 27B Ch61 | 15 83 06 21 85 80 73 06 11 80 0B 00 63 E9 46 60 00 00 00 00 00 00 00 00 D8 16 39 I: enqueued cmd failed/timeout I: (#0) I: no Payload received! (retransmits: 5) I: resetPayload: id: 0 I: (#0) Requesting Inv SN I: (#0) enqueuedCmd: 0x1 I: (#0) enqueuedCmd: 0xb I: (#0) sendTimePacket I: TX 27B Ch75 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 7B 00 00 00 00 00 00 00 00 22 A2 66 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0x1 I: TX 27B Ch3 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 7B 00 00 00 00 00 00 00 00 22 A2 66 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0x1 I: TX 27B Ch23 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 7B 00 00 00 00 00 00 00 00 22 A2 66 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0x1 I: TX 27B Ch40 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 7B 00 00 00 00 00 00 00 00 22 A2 66 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0x1 I: TX 27B Ch61 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 7B 00 00 00 00 00 00 00 00 22 A2 66 W: nothing received: Request Complete Retransmit I: (#0) sendTimePacket 0x1 I: TX 27B Ch75 | 15 83 06 21 85 80 73 06 11 80 01 00 63 E9 46 7B 00 00 00 00 00 00 00 00 22 A2 66`

initially ESP time was not set during reboot. I clicked "SYNC FROM BROWSER" right before the message "MQTT connected" was send on serial output. But when I clicked on "SYNC NTP" I didn't notice any response in the serial output.

beegee3 commented 1 year ago

if you were able to get a connection to the NTP server named in your AHOY settings, there would be an Information like 'I: [NTP]: 2023-02-12 21:21:07 UTC' right after the '[WiFi] AP disabled' message, or in case that the server doesn't give a valid time 'I: [NTP]: getNtpTime failed', but you get no information about NTP at all. That is the NTP server name cannot be resolved as an IP address. Either the NTP server name is wrong or AHOY is not allowed to connect to the internet. In the first case try a different name like pool.ntp.org. I guess the BSSID Mac '24 5a 4c 79 ed a' belongs to an AccessPoint, so in the latter case look into the settings of your router and your AccessPoint whether one of them (or both) disallow AHOY an internet connection and try to fix it. If it's impossible, has your router an own NTP service? If so, try the router name or IP as the NTP server name in your AHOY settings.

lumapu commented 1 year ago

it also can be a local DNS problem, maybe the DHCP provides a wrong DNS information to the network. Another problem could be a network fort like PiHole

E1nste1n-AUT commented 1 year ago

ahh good to know! thanks for pointing this out, I will have a closer look and get back to you as soon as possible. (PiHole ist not in use at the moment, but it really could be due to some local DNS problem or an issue with DHCP)

E1nste1n-AUT commented 1 year ago

Finally, good news! It was indeed a configuration problem (combination of wrong ntp settings and my home network configuration). I decided to use the NTP service of my QNAP NAS and configured the local IP address of my NAS as NTP server. With this setting, I got the expected output with [NTP].... (see below):

`I: .. done I: Settings valid: true I: RF24 Amp Pwr: RF24_PA_I: LOW I: Radio Config: SPI Frequency = 1 Mhz Channel = 3 (~ 2403 MHz) Model = nRF24L01+ RF Data Rate = 250 KBPS RF Power Amplifier = PA_LOW RF Low Noise Amplifier = Enabled CRC Length = 16 bits Address Length = 5 bytes Static Payload Length = 32 bytes Auto Retry Delay = 250 microseconds Auto Retry Attempts = 0 maximum Packets lost on current channel = 0 Retry attempts made for last transmission = 15 Multicast = Disabled Custom ACK Payload = Disabled Dynamic Payloads = Enabled Auto Acknowledgment = Disabled Primary Mode = RX TX address = 0xdeadbeef01 pipe 0 (closed) bound = 0xdeadbeef01 pipe 1 ( open ) bound = 0x1106738001 pipe 2 (closed) bound = 0xc3 pipe 3 (closed) bound = 0xc4 pipe 4 (closed) bound = 0xc5 pipe 5 (closed) bound = 0xc6

AhoyDTU Info: Version: 0.5.78 Github Hash: 8457f46

AP MODE SSID: AHOY-DTU PWD: esp_8266 IP Address: http://192.168.4.1

connect to network '' ... I: added inverter I: resetPayload: id: 0 I: resetPayload: id: 1 I: resetPayload: id: 2 I: resetPayload: id: 3 I: resetPayload: id: 4 I: resetPayload: id: 5 I: resetPayload: id: 6 I: resetPayload: id: 7 I: resetPayload: id: 8 I: resetPayload: id: 9 reconnect in 9 seconds reconnect in 8 seconds reconnect in 7 seconds reconnect in 6 seconds reconnect in 5 seconds reconnect in 4 seconds reconnect in 3 seconds reconnect in 2 seconds reconnect in 1 seconds I: Iv: 0 I: scanning APs with SSID reconnect in -1 seconds reconnect in -2 seconds reconnect in -3 seconds BSSID 0: 24 5a 4c 79 ed a BSSID 1: 24 5a 4c 79 bf 8 BSSID 2: 78 8a 20 5a bb b1 reconnect in -4 seconds reconnect in -5 seconds reconnect in -6 seconds reconnect in -7 seconds reconnect in -8 seconds reconnect in -9 seconds I: Iv: 0 I: reconnect in -10 seconds try to connect to AP with BSSID: 24 5a 4c 79 ed a reconnect in 9 seconds reconnect in 8 seconds

[WiFi] Connected

Welcome to AHOY!

point your browser to http://192.168.0.222 (Station) to configure your device

[WiFi] AP disabled I: [NTP]: getNtpTime failed I: [NTP]: getNtpTime failed I: Iv: 0 I: I: [NTP]: getNtpTime failed I: [NTP]: 2023-02-15 05:45:52 UTC I: MQTT connected I: resetPayload: id: 0 I: (#0) Requesting Inv SN I: (#0) enqueuedCmd: 0x1 I: (#0) enqueuedCmd: 0xb I: (#0) sendTimePacket I: TX 27B Ch40 | 15 83 06 21 85 80 73 06 11 80 01 00 63 EC 71 96 00 00 00 00 00 00 00 00 9F 93 35`

I still experience some breakdowns of the ahoyDTU webpage being not accessible right aber reboot or after clicking onto LIVE or SETTINGS. But this is a different issue so I think this NTP sync issue could be closed ;)

beegee3 commented 1 year ago

šŸ‘ finally

I still experience some breakdowns ...

you are on 0.5.78. Please try the latest development 0.5.88 or, if the problems remain, the stable version 0.5.66