search

NearNodeFlash / NearNodeFlash.github.io

View this document https://nearnodeflash.github.io/
Apache License 2.0
3 stars 3 forks source link

nnf-controler-manager uses UTC timestamps #36

Open behlendorf opened 1 year ago

behlendorf commented 1 year ago

When viewing the nnf-controller-manager container logs they use a UTC based timestamp. e.g. 2023-02-27T19:06:31.751Z (note the trailing Z)

2023-02-27T19:06:31.751Z    INFO    controllers.NnfWorkflow    Start    {"Workflow": "default/fluxjob-187278588523840512", "state": "PreRun", "index": 0, "directive": "#DW jobdw capacity=1TiB type=xfs name=test1"}                                                   │
2023-02-27T19:06:31.751Z    INFO    controllers.NnfWorkflow    Start done    {"Workflow": "default/fluxjob-187278588523840512", "state": "PreRun", "index": 0}

Other logs like those from the nnf-nodemanager use the local time.

2023-02-27T10:35:58.314-0800    INFO    controllers.NnfNodeStorage    Created filesystem    {"NnfNodeStorage": "hetchy44/default-fluxjob-187285641715778560-0-xfs-0", "Id": "default-fluxjob-187285641715778560-0-xfs-0-0"}                                             │

Can the nnf-controller-manager be configured or updated to use local time to make it easier to correlate events between log files.

ajfloeder commented 1 year ago

@behlendorf are all of your systems (k8s-master and the Rabbits) configured to use the same time zone and NTP settings?

behlendorf commented 1 year ago

Yes, the configuration is the same on all the nodes running k8s:

# pdsh -w hetchy[4-6,9,44] timedatectl | dshbak -c
----------------
hetchy[4-6,9,44]
----------------
               Local time: Wed 2023-03-01 13:34:12 PST
           Universal time: Wed 2023-03-01 21:34:12 UTC
                 RTC time: Wed 2023-03-01 21:34:12
                Time zone: US/Pacific (PST, -0800)
System clock synchronized: yes
              NTP service: n/a
          RTC in local TZ: no