Once a while I notice datastore lock timeouts. This must be due to the test that acquired the lock somehow died after acquiring it or failed to communicate to S3 to release the lock.
The lock file in the S3 has been created 14 hours ago.
2017-03-26 11:26:30 INFO Traceback (most recent call last):
2017-03-26 11:26:30 INFO File "/home/ubuntu/cloud-weather-report/cloudweatherreport/run.py", line 338, in <module>
2017-03-26 11:26:30 INFO entry_point()
2017-03-26 11:26:30 INFO File "/home/ubuntu/cloud-weather-report/cloudweatherreport/run.py", line 332, in entry_point
2017-03-26 11:26:30 INFO exitcode = Runner(args.controllers[0], False, args).run()
2017-03-26 11:26:30 INFO File "/home/ubuntu/cloud-weather-report/cloudweatherreport/run.py", line 109, in run
2017-03-26 11:26:30 INFO if not self.run_plan(test_plan):
2017-03-26 11:26:30 INFO File "/home/ubuntu/cloud-weather-report/cloudweatherreport/run.py", line 185, in run_plan
2017-03-26 11:26:30 INFO with datastore.lock():
2017-03-26 11:26:30 INFO File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2017-03-26 11:26:30 INFO return self.gen.next()
2017-03-26 11:26:30 INFO File "/home/ubuntu/cloud-weather-report/cloudweatherreport/datastore.py", line 101, in lock
2017-03-26 11:26:30 INFO raise TimeoutError('Timed out waiting for lock')
2017-03-26 11:26:30 INFO cloudweatherreport.datastore.TimeoutError: Timed out waiting for lock
2017-03-26 11:26:30 INFO ERROR: run_command failed: None
2017-03-26 11:26:30 INFO Gathering remote logs.
2017-03-26 11:26:30 INFO Executing: ['/usr/bin/juju-2.0', 'status', '--format', 'yaml', '-m', 'cwr-gce:cwr-gce']
Once a while I notice datastore lock timeouts. This must be due to the test that acquired the lock somehow died after acquiring it or failed to communicate to S3 to release the lock.
The lock file in the S3 has been created 14 hours ago.