define ntpd daemon inside the core role

[sarcasticadmin]

Description

Previously in the ansible code base we had defined ntpd: https://github.com/socallinuxexpo/scale-network/blob/ffda5117e0fd7757dbf4d43f190f13f5092b0254/ansible/roles/chrony/tasks/main.yml

We should define this again for the core role so that we can offer it as a service to the rest of the fleet (servers, aps, switches, etc.)

Acceptance Criteria

• core role has ntpd service accessible
• nix tests added to confirm ntpd is working as expected

[nixinator]

might be nice to enable authentication on this. but it means loading a key onto the ntp clients and the servers. which is PITA. A age encrypted secret might suffice.

i can confirm that chronyd is in nixpkgs. https://github.com/NixOS/nixpkgs/blob/nixos-22.11/nixos/modules/services/networking/ntp/chrony.nix

there is a test, https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/chrony-ptp.nix

it's very simple and just checks if the service can start .

so probably requires a more sophisticated one between client and server. however how do you 'test' time?

[davidelang]

I've had bad experience with chronyd, please no.

David Lang

On Fri, 24 Mar 2023, Lee Hughes wrote:

might be nice to enable authentication on this. but it means loading a key onto the ntp clients and the servers. which is PITA. A age encrypted secret might suffice.

i can confirm that chronyd is in nixpkgs.

https://github.com/NixOS/nixpkgs/blob/nixos-22.11/nixos/modules/services/networking/ntp/chrony.nix

[nixinator]

Tell us more! What bad experience did you have...

[davidelang]

This was a couple jobs ago, and I don't remember all the details (I was a dedicated splunk admin there), but it was a combination of not generating useful logs for troubleshooting issues, and not reliably setting the time correctly (causing us to need to try and troubleshoot things)

David Lang

On Fri, 24 Mar 2023, Lee Hughes wrote:

Tell us more! What bad experience did you have...

[nixinator]

We i will generate a couple of configurations for different time servers..

let the best time server win! I've seen time server go badly wrong when running under some kinds of virtualisation.. it as unexpected side effects, but that was many moons ago. Software always gets better, where hardware fail eventually.

Are you sure you were not under a MITM or a WITM attack? People love to do that !

[davidelang]

very sure, we had an on-prem GPS server we were referencing against.

David Lang

On Fri, 24 Mar 2023, Lee Hughes wrote:

Are you sure you were not under a MITM or a WITM attack? People love to do that !

[nixinator]

@owendelong , if you want to have experiment with https://en.wikipedia.org/wiki/Precision_Time_Protocol .

However it looks pretty interesting, maybe a bit overkill . It also needs hardware support, can the Juniper platform do it?

I also went down a time sync research rabbit hole, I guess a time rabbit hole, which could be described as black hole, which was a bit of a 'time sink', rather than 'time sync'.

I found this is being used for synchronising clocks at layer one by encoding the time into the layer 1 encoding! Blimey...

funny enough it's called the white rabbit network. This is probably useful it we plan to run a nuclear fusion reactor at scale40x

https://events.geant.org/event/1207/contributions/1177/attachments/776/1112/GEANT2022-WR_Intro_Lipinski_v1.pdf

Interesting stuff.

[davidelang]

I'm very sure that nothing we run has the hardware to support PTP (I've looked at it a little bit in the past)

as you say, PTP is drastic overkill for what we are doing, simple NTP is all we need (and even there, we need the clocks to be synced to each other far more than we need them syned to 'true' time)

David Lang

On Sat, 25 Mar 2023, Lee Hughes wrote:

@owendelong , if you want to have experiment with https://en.wikipedia.org/wiki/Precision_Time_Protocol .

However it looks pretty interesting, maybe a bit overkill . It also needs hardware support, can the Juniper platform do it?

I also went down a time sync research rabbit hole, I guess a time rabbit hole, which could be described as black hole, which was a bit of a 'time sink', rather than 'time sync'.

I found this is being used for synchronising clocks at layer one by encoding the time into the layer 1 encoding! Blimey...

funny enough it's called the white rabbit network. This is probably useful it we plan to run a nuclear fusion reactor at scale40x

https://events.geant.org/event/1207/contributions/1177/attachments/776/1112/GEANT2022-WR_Intro_Lipinski_v1.pdf

Interesting stuff.

[owendelong]

Let's stick with ntpd and just make it work for this year.

[owendelong]

Marking this as resolved by #665.