"Setting oauth clockskew" messages on booting node; slows down boot

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1311700

Launchpad details

affected_projects = ['maas']
assignee = None
assignee_name = None
date_closed = None
date_created = 2014-04-23T14:29:24.949995+00:00
date_fix_committed = None
date_fix_released = None
id = 1311700
importance = low
is_complete = False
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1311700
milestone = None
owner = julian-edwards
owner_name = Julian Edwards
private = False
status = triaged
submitter = rodsmith
submitter_name = Rod Smith
tags = []
duplicates = []

Launchpad user Rod Smith(rodsmith) wrote on 2014-04-23T14:29:24.949995+00:00

On booting nodes for enlistment, commissioning, and starting, I sometimes see repeated errors like this:

2014-04-23 14:15:50,622 - DataSourceMAAS.py[WARNING]: Setting oauth clockskew to -2

(See attached screen shot, too.) These scroll by, once every second or so, for about a minute before the node continues booting. Although a minute here or there isn't all THAT big of a deal, it would be nice to see this delay go away, particularly for occasions when I have to repeatedly test a node.

Here's my version information:

$ dpkg -l 'maas'|cat Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-=====================================================-===================================================-============-=============================================================================== ii maas 1.5+bzr2252-0ubuntu1 all MAAS server all-in-one metapackage ii maas-cli 1.5+bzr2252-0ubuntu1 all MAAS command line API tool ii maas-cluster-controller 1.5+bzr2252-0ubuntu1 all MAAS server cluster controller ii maas-common 1.5+bzr2252-0ubuntu1 all MAAS server common files ii maas-dhcp 1.5+bzr2252-0ubuntu1 all MAAS DHCP server ii maas-dns 1.5+bzr2252-0ubuntu1 all MAAS DNS server ii maas-region-controller 1.5+bzr2252-0ubuntu1 all MAAS server complete region controller ii maas-region-controller-min 1.5+bzr2252-0ubuntu1 all MAAS Server minimum region controller ii python-django-maas 1.5+bzr2252-0ubuntu1 all MAAS server Django web framework ii python-maas-client 1.5+bzr2252-0ubuntu1 all MAAS python API client ii python-maas-provisioningserver 1.5+bzr2252-0ubuntu1 all MAAS server provisioning libraries

[ubuntu-server-builder]

Launchpad user Rod Smith(rodsmith) wrote on 2014-04-23T14:29:24.949995+00:00

Launchpad attachments: Screen shot taken from a test node

[ubuntu-server-builder]

Launchpad user Rod Smith(rodsmith) wrote on 2014-04-23T14:29:54.518187+00:00

Launchpad attachments: Log files from the MAAS server

[ubuntu-server-builder]

Launchpad user Julian Edwards(julian-edwards) wrote on 2014-04-24T00:17:25.890850+00:00

What makes you think this is slowing down the boot?

[ubuntu-server-builder]

Launchpad user Rod Smith(rodsmith) wrote on 2014-04-24T12:54:00.013791+00:00

Because it sits there for a minute without doing anything but print those error messages. If that's normal, then the message should be changed to notify the user that a pause in the boot process is happening, and printed just once; as it is, it looks like an error.

[ubuntu-server-builder]

Launchpad user Julian Edwards(julian-edwards) wrote on 2014-04-27T23:52:05.751880+00:00

Ok that message comes out of cloud-init. I've added a bugtask for Scott's comments to see if anything can be done in MAAS.

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2014-08-25T17:08:34.100595+00:00

I can't come up with any reason that these messages should actually delay boot at all. They're simply output of the following:

• cloud-init tries to reach an OATH protected url (with timestamp from system clock)
• cloud-init is given 403 with a timestamp indicating the servers time
• cloud-init re-tries with timestamp from server (and tells you that it is doing this as it indicates your clock is potentially more than 5 minutes off).

I do not think it delays boot.

Updating the hardware clock during boot is somewhat problematic. As is currently implemented the above procedure occurs each time cloud-init makes a request. We could fix that so that cloud-init remembered the offset across subsequent invocations. We could also look into having it attempt to set the hardware clock to what it views as "right". The issue with doing htat is that anything that has a timer set may be confused. ie: program runs and invokes: sleep 30 cloud-init change clock back 3 days 3 days and 30 seconds later, program returns from 'sleep 30'.

[ubuntu-server-builder]

Launchpad user LaMont Jones(lamont) wrote on 2015-09-08T14:35:58.572059+00:00

This is a cloud-init bug.