DietPi-Set_Software | Add ability to allow local time server as time sync mirror

[MichaIng]

I just recently recognized, that my router serves as time sync mirror within my local network, while itself syncs regularly with configured external time server, e.g. preconfigured by ISP. In such setups it should be beneficial for DietPi to sync with the local server instead of external as well.

When entering the local network address into dietpi-config ntpd mirror, it is added the wrong way, e.g. (0-3).192.168.178.1 which are of course not available (?). If you want this, just 192.168.178.1 needs to be added.

We could add a separate menu entry/option to add a local IP/hostname, which is then added the correct way, with e.g. local gateway (DHCP server) as default preconfigured entry, which will in most cases be correct. Another possibility would be to just allow it via "custom" entry, by e.g. adding "local" in front of IP/hostname. Then we would not need to add new menu items, just grep for "local" and in case add the following string without (0-3)... to the ntp/timesyncd config file.

Affected code lines: https://github.com/Fourdee/DietPi/blob/testing/dietpi/func/dietpi-set_software#L183-L222

Any ideas/suggestions about this?

[k-plan]

Hi @MichaIng ,

In such setups it should be beneficial for DietPi to sync with the local server instead of external as well.

Richtig. 😃 Und ich mag die Idee, einen lokalen NTP Server zu nutzen und konfigurieren zu können.

Another possibility would be to just allow it via "custom" entry, by e.g. adding "local" in front of IP/hostname.

Puh ... das wird komplex bei der Endnutzer Konfiguration. Weiss nicht, ob das etwas für unbedarfte Quereinstiger ist. Klingt mir mehr nach einer "Entwickler"-Lösung. Vorteil den ich noch sehe, man muss nichts im dietpi.txt Handling ändern und in dietpi-globals anpassen. Ob ich mich damit anfreunden könnte? Jein.

We could add a separate menu entry/option to add a local IP/hostname, which is then added the correct way, ...

Sagt mehr eher zu. Wobei ich für eine Untermenü in Custom Manually enter NTPD mirror plädieren würde.

• 1.) Custom Manually enter NTPD mirror pool name (eg: de.pool.ntp.org or uk.pool.ntp.org)

  
  Please enter a new NTPD Mirror

• eg: debian.pool.ntp.org

• 2.) Custom Manually enter NTPD server name or addesses (eg: 192.168.0.1, ntp.lan, 62.116.162.126, ntp.web.de )

  
  Please enter a new NTP server address  or name 
   - 4 maximum, seperated by a space. 
   - eg: 192.168.0.1 ntp.lan 62.116.162.126 ntp.web.de

Das bedarf dann aber einiger Anpassungen in `whiptail menu` , im Handling von `dietpi.txt`  und in `dietpi-globals`.

----------------

> I just recently recognized, that my router serves as time sync mirror within my local network, ....

Das machen aber nicht alle Router. Bei einigen "Billig-Plastik-Router" geht das gar nicht, bei einigen muss man es erst Konfigurieren (z.B. Mikrotik) und einige machen es einfach ohne Konfigurationsmöglichkeiten (z.B. Fritz!Box) wenn sie einen NTP Server via DHCP oder TR-069 auf der WAN Schnittstelle beziehen. Einige haben auch einfach ein paar NTP Server Adressen fix ins Firmware einkompiliert.
![180408-0002](https://user-images.githubusercontent.com/17237601/38469205-2f36aaf6-3b51-11e8-9be1-baab41ee4f25.gif)
Aber lassen das vorerst einmal unberücksichtigt. 

**DHCP in Verbindung mit NTP:**

Hier kann man sehen, wie der DHCP Server konfiguriert ist:

![180408-0001](https://user-images.githubusercontent.com/17237601/38469219-69ba57b8-3b51-11e8-8b4a-be11469da1ba.gif)

Das ist auch so ein Ding.
Im Moment läuft das mit den damit verbundenen DHCP Clients  noch nicht so richtig "standardkonform" bei einigen Debian based Images.

Ist auf einem Gerät der IP Adressbezug mittels dhcp client aktiviert und propagiert der [DHCP Server einen NTP Server](http://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml) via `dhcp option 42` sollte der Client diesen auch benutzen. Das macht der Client im Moment aber nicht.

Bezieht der Client die IP-Adresse via DHCP , dann sollte er die Konfiguration des Zeitservers nicht aus der Datei `/etc/ntp.conf`, sondern aus `/var/lib/ntp/ntp.conf.dhcp`einlesen. Diese Datei wir aber nicht angelegt.
Konfiguriert man den NTP Server per Hand, muss `/var/lib/ntp/ntp.conf.dhcp` gelöscht werden, Netzwerk und Zeitdienste müssen neu gestartet werden. Dann wird `/etc/ntp.conf` eingelesen und verwendet.

Soweit ich das bisher verstanden habe, hängt das mit `/etc/dhcp/dhclient.conf` , `/etc/dhcp/dhclient-exit-hooks.d/ntp` , `/var/lib/ntp/ntp.conf.dhcp` zusammen.
Da hapert es irgendwo. Ob bei allen Images, kann ich nicht sagen.
Wenn deine bridged Dietpi VM via DHCP eine Adresse von der fritz.box bezieht, müsste diese auch der primäre NTP Server deiner VM werden.

https://serverfault.com/questions/329596/how-to-override-the-ntp-information-sent-by-dhcp-in-debian

http://www.raspberry-pi-geek.de/Magazin/2014/03/RasPi-als-DHCP-und-DNS-Server/(offset)/2

https://forum.ubuntuusers.de/topic/ntpd-ignoriert-serverliste-in-etc-ntp-conf/

Sorry, das ist auch wieder so eine Mammut Ding. Hätte ich gerne ausgespart. Aber wenn man es anfasst, sollte man es auch gemäß den Standard tun.

[Fourdee]

@MichaIng @k-plan

Maybe a simple solution, could we not just add the current IP gateway (eg: router) to the server list automatically?

server 192.168.0.1
server 0.europe.pool.ntp.org iburst
server 1.europe.pool.ntp.org iburst
server 2.europe.pool.ntp.org iburst
server 3.europe.pool.ntp.org iburst

My router doesnt support NTP, however, it still passed via external pool server.

EDIT:

We could also have a dietpi.txt entry with:

#Change the local NTP mirror, gateway=default, eg: router, or use a different ip address etc
CONFIG_NTP_LOCAL_MIRROR=gateway

Then additional dietpi-config option to change it?

[k-plan]

@Fourdee

Maybe a simple solution, could we not just add the current IP gateway (eg: router) to the server list automatically?

😃 - yes it's simple. But I think, @MichaIng want to avoid traffic and to pester public ntp server with his LAN devices ...

But I have here more when one SNTP server for highly available in local area network. What should I do? And if I want to ask my local ISP for NTP, because he response quicker when pool.ntp.org server, what should I do?

My router doesnt support NTP, ...

Most network devices only support SNTP server, not completely implement NTP server ...

...however, it still passed via external pool server.

Yes, cheap-plastic-router from Shenzhen market ... 😄 Should I sent you a real one?

We could also have a dietpi.txt entry with:

Nice! 👍 Can we have more than one entry, eg. three or four? And can we have the option to disable the NTP pool server, if CONFIG_NTP_LOCAL_MIRROR is set?

[MichaIng]

@k-plan @Fourdee Jep, the idea was to avoid traffic and perhaps reduce sync time by allowing to just sync with local NTP server. I am not sure how it is handled, if the local IP/host is just added to the servers list.

As dietpi.txt design currently always has all settings set, it would need three settings all together then:

• CONFIG_NTP_MIRROR_MODE=local/pool
• CONFIG_NTP_LOCAL_MIRROR=gateway
• CONFIG_NTP_POOL_MIRROR=debian.pool.ntp.org
• Then an additional G_WHIP_YESNO, when selecting NTP mirror to choose between pool and local, or just another entry to the existing G_WHIP_MENU, something like Custom (local) or make a clear separation between server (locally or external) and pool entries.
• For local server it should be painless to allow several different entries then as well.

But hmm to keep it simple... Isn't actually this 0. / 1. only available for the ntp.org pool servers, thus we could just grep for pool.ntp.org within mirror string to decide if we add it as single server or 4 times with prefix? On the other hand, who knows if here and there it is adopted by local NTP servers as well...

[MichaIng]

https://manpages.debian.org/testing/systemd/timesyncd.conf.5.en.html

Idea: Always set local gateway as as NTP server and pool servers as FallbackNTP to reduce unnecessary NTP pool server load?

[MichaIng]

Okay, following solution, I would go for:

• By default use local gateway as NTP server and debian.pool.ntp.org as FallbackNTP, which will only be called, if the servers, set at NTP= are not reachable.
  • Test needed how long the sync attempts take, until FallbackNTP is used, possibly remove local NTP server automatically, if sync fails?
• 2 settings within dietpi-config:
  • Local time sync server: Add up to 4 time servers, separated by white space, e.g. 192.168.0.1, leave empty to directly use public pool server
  • Public ntp.org pool server: Used as fallback, if local time server is left empty or unreachable, e.g. uk.pool.ntp.org
  • Some explaining words, link to pool.ntp.org maybe. Also the whip menu with continental pools respectively local gateway, and custom text box could be used (similar as currently).
• Accordingly 2 settings within dietpi.txt: CONFIG_NTP_LOCAL_MIRROR, CONFIG_NTP_POOL_MIRROR
  • If CONFIG_NTP_LOCAL_MIRROR=gateway (default), autoconfigure local gateway, if left empty or set to =none (and/or =pool?), then CONFIG_NTP_POOL_MIRROR will be moved from FallbackNTP to NTP to be main sync server.

@Fourdee @k-plan What do you think?

[Fourdee]

@MichaIng

Apologies for the delay, been a bit manic recently lol.

By default use local gateway as NTP server and debian.pool.ntp.orgas FallbackNTP

Yes, love this idea and the additional two options. Worth implementing 👍

[MichaIng]

@Fourdee Okay great, leave this to me, will create PR later.

[MichaIng]

🈺 if NTP= configured local server fails, the daemon does not fallback to the FallbackNTP= servers automatically. FallbackNTP= is only used, if no NTP= is defined and no per-network NTP server within /etc/systemd/network/* configuration: https://manpages.debian.org/stretch/systemd/timesyncd.conf.5.en.html

Thus, we need to let user choose between local NTP server, optional automated gateway test, and ntp.pool.org servers.

[MichaIng]

Testing

Jessie

• By default, without any settings, [0-3].debian.pool.ntp.org is used.
• No timesyncd.conf documentation: https://manpages.debian.org/stretch/systemd/timesyncd.conf.5.en.html
• No hint for config file on service man page: https://manpages.debian.org/jessie/systemd/systemd-timesyncd.service.8.en.html
• 🈯️ Undocumented Servers= setting works
• 🈴 NTP= and FallbackNTP= cause Unknown lvalue error message, falling back to [0-3].debian.pool.ntp.org
• Active DHCP lease has no influence.
• 🈴 /etc/systemd/timesyncd.conf.d/*.conf is not used

Stretch

• Man page: https://manpages.debian.org/stretch/systemd/timesyncd.conf.5.en.html
• 🈯️ Undocumented Servers= and NTP= work the same way, if both is defined, the first value is used.
• 🈴 FallbackNTP= does not work at all, with this and without any setting/config file fallen back to [0-3].debian.pool.ntp.org
• Active DHCP lease leads to gateway is used as DHCP server by default (perhaps answering local NTP server?). Servers= and NTP= overwrite this.
• 🈯️ /etc/systemd/timesyncd.conf.d/*.conf is used

Buster

• Man page: https://manpages.debian.org/testing/systemd/timesyncd.conf.5.en.html
• 🈯️ Undocumented Servers= and NTP= work the same way, if both is defined, the first value is used.
• 🈯️ FallbackNTP= works and is used if no Servers= and NTP= is defined.
• Active DHCP lease leads to gateway is used as DHCP server by default (perhaps answering local NTP server?). Servers= and NTP= overwrite this.
• 🈴 /etc/systemd/timesyncd.conf.d/*.conf is not used 🤔 very confusing, but tested it up and down...

  ---

  Results

• We can't use /etc/systemd/timesyncd.conf.d/dietpi.conf, since it is not supported on Jessie and Buster.
• We can't use FallbackNTP=, since it is not supported on Jessie and Stretch.
• Since NTP= is not supported on Jessie, but undocumented Servers= has the same effect on all Distros, we should stay using it.
• On Stretch and Buster, when using DHCP, local NTP server is used automatically. But this does not work with static IP and not at all on Jessie.

Implementation

• Within menu allow "system default", which removes/comments all settings to fallback to system default.
• This should be new default (DietPi-PREP+config.txt), which leads to [0-3].debian.pool.ntp.org is used (same as now) as fallback, but at least on Stretch with DHCP, local gateway is used, if it serves NTP, which should be preferred.
• As well allow to choose "local gateway (recommended)", detect (ip r) and add network gateway(s) as NTP server(s).
• Within custom inputbox grep manual NTP server input for [[ $G_WHIP_RETURNED_VALUE =~ 'pool.ntp.org' ]], in case add it via [0-3].$G_WHIP_RETURNED_VALUE, otherwise add only input value, to allow local NTP server.
• After choice was done, do run_ntpd status and let user re-choose, if time sync fails.
• Allow to set NTP setting in dietpi.txt to "default" or "gateway".

[MichaIng]

Yeyyy, finally done: https://github.com/Fourdee/DietPi/pull/1974

This even allows to enter multiple custom mirrors (space-separated), and *.pool.ntp.org entries will be added with leading [0-3]. automatically, the others will be left without. Automated local gateway detection, theoretically allowing multiple default gateways as well (if this can be even the case?), reworked explanations and recommendation.

[Fourdee]

🈯️ https://github.com/Fourdee/DietPi/commit/7b5af9ee300534a774b2089c912b6b765e8493fb We need to force check ntp sync during dietpi-config, eg: gateway

root@DietPi:~# systemctl status systemd-timesyncd -l
● systemd-timesyncd.service - Network Time Synchronization
   Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; disabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/systemd-timesyncd.service.d
           └─disable-with-time-daemon.conf
   Active: active (running) since Tue 2018-07-31 22:55:40 BST; 14s ago
     Docs: man:systemd-timesyncd.service(8)
 Main PID: 14182 (systemd-timesyn)
   Status: "Idle."
   CGroup: /system.slice/systemd-timesyncd.service
           └─14182 /lib/systemd/systemd-timesyncd

Jul 31 22:55:40 DietPi systemd[1]: Starting Network Time Synchronization...
Jul 31 22:55:40 DietPi systemd[1]: Started Network Time Synchronization.
Jul 31 22:55:50 DietPi systemd-timesyncd[14182]: Timed out waiting for reply from 192.168.0.1:123 (192.168.0.1).

🈯️ Fixed below commit | I may of broke dietpi-config menu for this, working on it now.

[Fourdee]

@MichaIng

Great work, seems to work a treat 👍

Although I couldnt get gateway to work on my network (https://www.tp-link.com/uk/products/details/cat-9_Archer-C2.html), lacks local NTP server by looks of it.

[MichaIng]

Finished, will mark this as closed.