Feature request: allow to set NTP server

[MathiasVDA]

I run a secured internal network with multipel VLANs. I initially placed my SmartEVSE on my NoT network that is not allowed access to the network. However, the clock is then not properly set. I discovered that the NTP server is fixed in the firmware. This feature request is to make the NTP server either configurable and/or inherited from the DHCP server. (My DHCP server also publishes the address of the local NTP server)

[dingo35]

I understand your question; for some reason it seems to be hard to get the ntp-server from dhcp on arduino-esp32, so I'm first going the configuration road. The library we are using has a setting called USE_ESP_WIFIMANAGER_NTP . When I set this to true stuff happens with the TimeZone , that is input for the call to the setting of the NTP server.

So first step is to get the TimeZone stuff working; I removed all hardcoded timezone stuff and now the portal detects my timezone and time is set ok (on the oldfashioned, hardcoded ntp server). Not sure if this works without using the portal, so could you please test, if after reflashing, but NOT using the portal, you are still getting the correct time on your SmartEVSE? If not, does reconfiguring your wifi through the portal solve this? ntp.zip

[dingo35]

By the way what is your location? I still have to hardcode Europe as location , because of memory limitations of the ESP32....

[dingo35]

Ok found an undocumented feature: without even using dhcp, the ESP32 can give out an ntp request on 0.0.0.0, and any ntp server on the lan then can answer.

So this version should work for you if you have any ntp-server accessible on the vlan of your SmartEVSE: ntp2.zip

[MathiasVDA]

Ok found an undocumented feature: without even using dhcp, the ESP32 can give out an ntp request on 0.0.0.0, and any ntp server on the lan then can answer.

So this version should work for you if you have any ntp-server accessible on the vlan of your SmartEVSE: ntp2.zip

Thanks dingo35 for your work today! I was amazed to find a response so fast after mentioning the issue. I'm in timezone Europe/Brussels.

I have flashed your ntp2.zip firmware & spiff onto my smartevse. I had the smartevse on my IoT network where it does have internet access so I moved it back onto my NoT network. I have the impression that the time is now saved on the smartevse and that I can't see a difference on the portal. I assume the time is saved into a RTC?

Anyway, I looked for another way to check if the smartevse has a connection to the NTP server.

I did a tcpdump on my access point to see what smartevse does. I only see the following line related to this topic: 16:15:34.647781 IP 192.168.20.133.61791 > 0.0.0.0.123: NTPv4, Client, length 48 I also kept an eye on what's going on at the NTP server but I couldn't see any incoming connections from the smartevse. Normally there should also be a response from the ntp server. Like in this extract: (192.168.2.9 runs the ntp server) I checked my firewall settings and it's open from any IP to any IP in the network that uses destination port 123. So I guess the undocumented feature doesn't work?

[MathiasVDA]

Not sure if this works without using the portal, so could you please test, if after reflashing, but NOT using the portal, you are still getting the correct time on your SmartEVSE?

I'm not exactly sure what you would like me to test here. I have flashed the ntp.zip firmware onto my smartevse and see the same tcp behavior:

[dingo35]

Just flash ntp2.zip; then power off for 5 minutes; that makes the SmartEVSE lose its time. After 5 minutes, power on, and immediately look at the webserver, date should be 1970; within 60 seconds time should be corrected to the actual time IF an ntp server can bw found on your lan; if not it will try to go on the internet after 60s . The SmartEVSE has no RTC so loses its time completely after losing power for 5 minutes.

So see if it functions on your isolated vlan....

Happy testing!

[MathiasVDA]

I have flashed ntp2 on the smartevse again, powered it off for more then 5 minutes and restarted. Time was indeed back to the past. 1970.

After 10min, it still shows the 1970 date. Similar tcp behavior.

Are you sure that sending a tcp packet to 0.0.0.0 should respond in a respons from a local NTP server? 0.0.0.0 is more meant to refer to the host itself. I've looked up this information on wikipedia: https://en.wikipedia.org/wiki/0.0.0.0

I've taken a look at Tasmota, a ESP32 firmware for iot. I run a couple of NodeMCUs for different sensors. But I forgot that I always just configure the NTP server on each client. So it's not using some kind of discovery process.

FYI: During the DHCP request/response process, the time server is also sent as an option:

[dingo35]

I know it is strange, that is,why I call the 0.0.0.0 an "undocumented" feature; I found it works on my lan by accident and Im not sure if/ why it works, that is why I asked you to test it on your setup.

My setup is no-vlan, fritzbox gateway on 10.0.0.1 that has ntp-server enabled. So the ESP32 might find the ntp server by looking at the gateway.... It doesnt matter if I announce the ntp-server-address by dhcp or not.

So is your ntp server running on your gateway?

I know Tasmota, they didnt know how to get this ntp option from dhcp either :-) We use a library to configure wifi / ip, so it is not easy to add an extra ntp configure option....

[MathiasVDA]

I think it works for you because the NTP server runs on your gateway and the ESP asks your gateway for the 0.0.0.0 address which is interpreted as local for the gateway. In my network, the NTP server is hosted on a raspberry pi that is not the gateway. 0.0.0.0 is not routed to the pi so it ends at the gateway. I could maybe create a route from my gateway to the pi for all requests on port 123 but that is probably not a solution you would want to build into the smartevse firmware if it doesn't work for everyone.

If the people from Tasmota haven't found a way to get the NTP server from the DHCP package then I think the best option for smartevse firmware is to allow to set the ntp by the user, just like Tasmota has solved it.

What library is used for DHCP? I see WiFi.h in the source but I'm not sure I'm reading it right. Maybe we could ask the maintainers to implement an enhancement in their library for this use case (and that could benefit Tasmota as well).

Another option I see is to write some custom code that sends a TCP message requesting DHCP information and decode the NTP server from the response. I'll see if I can do this at some point, could be fun :). But the easiest is probably to allow the server to be configured.

[dingo35]

The library used is ESPAsync_Wifimanager.

Just for test, could you see if creating a route so your ntp-server appears to be on your gateway, so I can be sure that that is actually working on any other network than mine?

[MathiasVDA]

Just for test, could you see if creating a route so your ntp-server appears to be on your gateway, so I can be sure that that is actually working on any other network than mine?

I've been trying to do this but the NTP redirect is not working. I'm not sure if this is because of the undocumented feature or because of my incompetence to add a NAT rule :/ (first time I'm doing this, I thought it was easily doable with the UI but it needed to be configured on the router by CLI)..

[dingo35]

Ok I found a sparsely documented function, so this version uses the dhcp-announced ntp server as first choice, and a public internet ntp server as a fallback. Please test. ntp3.zip

[dingo35]

Fixed in 1.7.0

[MathiasVDA]

I can confirm that it works. Just tested by upgrading (to ntp3.zip) and powering off the smartevse for 5minutes and back on. Time is set correctly, here is a tcpdump extract as reference:

Thank you very much!

[dingo35]

I had to disable this solution because of a change in the underlying libraries, in v3.5.0 only public ntp server will be polled... See https://github.com/dingo35/SmartEVSE-3.5/commit/5707a8f5f0237050ef20e5641e9123581a6082c6