Clear terraform instance cache when not running on leader

[kaufers]

• If the tf plugin is not running on the leader then the cached instances may not be valid. We need to clear the cache whenever we detect that we are not running on the leader.
• Also fixes the locking when clearing the cache after the initial apply (the write lock is needed when clearing).
• Update tf log message for file deltas outside window.
• Update UT to prevent the terraform apply polling loop from running since it's asynchronous nature creates race conditions in the tests.

[GordonTheTurtle]

Please sign your commits following these rules: https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work The easiest way to do this is to amend the last commit:

$ git clone -b "tf-standby-cache" git@github.com:kaufers/infrakit.git somewhere
$ cd somewhere
$ git rebase -i HEAD~842354386096
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

[codecov[bot]]

Codecov Report

Merging #862 into master will decrease coverage by 0.66%. The diff coverage is 37.5%.

@@            Coverage Diff             @@
##           master     #862      +/-   ##
==========================================
- Coverage   48.92%   48.25%   -0.67%     
==========================================
  Files          91       91              
  Lines        8206     8210       +4     
==========================================
- Hits         4015     3962      -53     
- Misses       3822     3881      +59     
+ Partials      369      367       -2

Impacted Files	Coverage Δ
pkg/provider/terraform/instance/plugin.go	86.95% <100%> (-0.19%)	:arrow_down:
pkg/provider/terraform/instance/apply.go	54.25% <16.66%> (-14.85%)	:arrow_down:
pkg/spi/instance/description.go	70.73% <0%> (-9.76%)	:arrow_down:
pkg/rpc/mux/server.go	42.7% <0%> (-5.21%)	:arrow_down:
pkg/types/spec.go	60.17% <0%> (+1.76%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 2c9fc77...3e9a5d2. Read the comment docs.