search

canonical / lxd

Powerful system container and virtual machine manager
https://canonical.com/lxd
GNU Affero General Public License v3.0
4.38k stars 931 forks source link

To enable LXD/MAAS integration on controllers, LXD should should look at nodes, rather than machines, when determining maas.machine #6276

Closed ThinGuy closed 5 years ago

ThinGuy commented 5 years ago

In a MAAS HA environment (n+2), it's very useful to create LXD cluster to run containers that provide services like mirrors, keyservers, simplestreams server, snap-proxy, etc.

However, the current MAAS integration only looks at machines, or node_type=0. Controllers can be a node_type of 2, 3, or 4.

If MAAS integration is performed at server level (and server/cluster members are also maas controllers), the same general error occurs with each operation:

$ sudo lxc config show
config:
  cluster.https_address: 172.27.20.1:8443
  core.https_address: 172.27.20.1:8443
  core.trust_password: true
  maas.api.key: Ax625PGbVyA5v5fv8A:WLDFH7KYmdRSPedwnH:dU6FhmAH2WYTxYctzn38fMcuWEkUvqad
  maas.api.url: http://172.27.20.1:5240/MAAS
  maas.machine: OrangeBox20.orangebox.me
$ lxc delete infra4 --force
Error: Can't perform the operation because MAAS is currently unavailable
$ lxc launch ubuntu:b snap-proxy -p maas-profile
Creating snap-proxy
Error: Create container: Create LXC container: Can't perform the operation because MAAS is currently unavailable

Note that looking at MAAS machines only shows node_types that equal 0. In order to see controllers, you have to look at "nodes"

$ maas admin machines read|jq -r '.[]|"\(.hostname) \(.system_id) \(.node_type_name) \(.node_type)"'
node00vm0ob20 smmfp7 Machine 0
node00vm1ob20 tnmfk4 Machine 0
node00vm2ob20 c3fbts Machine 0
node01ob20 hhebp3 Machine 0
node02ob20 ndhfpe Machine 0
node03ob20 sccbsy Machine 0
node06ob20 h4nyw4 Machine 0
node07ob20 baet4s Machine 0
node05ob20 drs8ym Machine 0
node10ob20 attk67 Machine 0
node04ob20 qpfn4q Machine 0
node08ob20 7yegbt Machine 0
node09ob20 xr8mxm Machine 0
$ maas admin nodes read|jq -r '.[]|"\(.hostname) \(.system_id) \(.node_type_name) \(.node_type)"'
node00vm0ob20 smmfp7 Machine 0
node00vm1ob20 tnmfk4 Machine 0
node00vm2ob20 c3fbts Machine 0
node01ob20 hhebp3 Machine 0
node02ob20 ndhfpe Machine 0
node03ob20 sccbsy Machine 0
node06ob20 h4nyw4 Machine 0
node07ob20 baet4s Machine 0
node05ob20 drs8ym Machine 0
node10ob20 attk67 Machine 0
node04ob20 qpfn4q Machine 0
node08ob20 7yegbt Machine 0
node09ob20 xr8mxm Machine 0
OrangeBox20 bds3ke Region and rack controller 4
maas-rack xyn6gk Rack controller 2
maas-region mw3mgk Region controller 3

Just doing MAAS integration at the container/profile level results in the containers showing up in MAAS DNS, but the assigned IP is from the MAAS commissioning range if container is set to use DHCP

If containers are configured for static IP addresses, then container does not register itself with the MAAS DNS server, requiring manual steps to add A records for each container in order for machines deployed by MAAS to use their services (e.g. snap-proxy, repositories, etc)

stgraber commented 5 years ago

We certainly would be happy with having containers be tied to a MAAS controller rather than an independent MAAS machine, but the API we're provided by MAAS through gomaasapi doesn't allow us to do this.

The concept of nodes in that Go package is pretty lacking, with the majority of API calls being tied to machines. It'd be great if you could file a bug against that project here: https://bugs.launchpad.net/gomaasapi

Then comment in this issue so that anyone else hitting this will know where to look.

Ideally for us, the best fix would be if MachineArgs (https://godoc.org/github.com/juju/gomaasapi#MachinesArgs) would be extended to support querying of controllers as well, in which case LXD would just always request for that, saving us from having to support two different code paths for everything. Alternatively, if we had function coverage in the package for Nodes, covering the same things we do with Machines currently, we could also switch over to that instead.

Closing as this may not actually need any changes to LXD (just a rebuild once sorted in gomaasapi) or if the approach taken in that project requires additional changes, letting us know in this closed issue will have us re-open it then so we can sort it out.