New ACL Integration Testing

[mkeeler]

• [x] Integration Tests
  • [x] Legacy Vault Integration (?)
  • [x] Vault configured
    • [x] consul as a secrets engine
    • [x] consul as a storage backend
  • [x] Nomad configured to use ACLs for service registration, discovery
  • [x] Legacy UI/Agent with new servers
  • [x] Single Datacenter Tests
    • [x] New Cluster - all nodes running new code
    • [x] Upgraded Servers - Servers w/ new ACLs, clients running 1.3.0
    • [x] Upgraded Clients - Clients w/ new ACLs, servers running 1.3.0
    • [x] All Upgraded - previously ACLs were bootstrapped with a 1.3.0
    • [x] Partial upgrade new leader - some servers running with new ACLs including the leader with other legacy ACL servers there as well (should use legacy ACLs all around)
    • [x] Partial Upgrade old leader - some servers running with new ACLs with others, including the leader, using legacy ACLs
  • [x] Multi Datacenter ACLs - all tests should be run with new ACLs in secondary datacenters with policy-only and policy+token replication.
    • [x] Primary DC Legacy + Secondary DC Upgraded
      • [x] Verify legacy ACL replication within new system still works. It should still replicate the old way and it should replicate tokens that are usable locally.
      • [x] The system shouldn't transition into new-ACL mode until all servers in the primary DC have been upgraded. There's now an ACL flag in the consul info/agent/self command that shows the state. structs/acl.go has all the constants for that.
    • [x] Primary DC Upgrade + Secondary DC Legacy. Should still replicate legacy tokens to the secondary legacy DC from the upgraded primary DC.
    • [x] Primary DC Upgrade + Secondary DC Upgraded
      • [x] Secondary DC legacy clients
      • [x] Replication works? (with #4885)
    • [x] Local Tokens within a secondary DC do not show up in the primary. There's a unit test for this now.
  • [x] Backwards Compatibility
    • [x] Legacy tokens still useable
    • [x] Legacy tokens visible via the new APIs

[mkeeler]

@johncowen Looks like the new UI when running in a cluster where all servers are 1.3.0 doesn't work when ACLs are enabled. It does't let you enter your token anywhere so you remain locked out of everything.

[banks]

For nomad I:

• Verified new Nomad cluster talking to 1.4.0 consul agent (dev mode) with ACLs bootstrapped under new system could start and register it's own services. Also checked it could register services for jobs running.
• Span up a 1.3.0 cluster with ACL bootstrapped
• Started Nomad pointing at the 1.3.0 cluster with a management ACL
• Verified it could register and unregister services
• Upgraded the whole cluster to 1.4.0
• Verified that once Consul was backup, the same nomad agent process could reconnect and continue to register/deregister services using the same ACL token (now a legacy one).