resourceSet option for vm resource

[dgmcdona]

We're considering the possibility of distributing terraform configs to students, and it would be great if they could create VMs inside of their designated resource sets. I currently get a permissions error when attempting to create a VM via terraform for resource set users.

[ddelnano]

@dgmcdona I wasn't aware of resource sets until you brought up this but that sounds like a great thing to add support for.

My main focus right is getting more functionality for vm disk and network configurations but this shouldn't be too difficult to implement. I will look into this after I'm done with the other work I have planned.

[ddelnano]

To make sure I understand how the feature works I did the following test:

1. Create non admin user
2. Create a resourceSet with cpu, memory and disk limits. Add the user from the previous step as a "subscriber" and add a storage repository, network and VM-template IDs as "objects"

$ xo-cli resourceSet.getAll
[
  {
    "id": "_g4tDg2z7dI",
    "ipPools": [],
    "limits": {
      "cpus": {
        "available": 4,
        "total": 4
      },
      "disk": {
        "available": 107374182400,
        "total": 107374182400
      },
      "memory": {
        "available": 12884901888,
        "total": 12884901888
      }
    },
    "name": "ddelnano-test",
    "objects": [
      "86a9757d-9c05-9fe0-e79a-8243cb1f37f3",
      "9b0fd2ac-c89a-93b4-5103-8506391395cd",
      "54dcb8d8-0af0-7739-61c5-144875958b1e"
    ],
    "subjects": [
      "60f3d51f-43f2-495d-80e1-8939633c468d"
    ]
  }
]

3. Run the vm.create command specifying the same storage repository, network ID and VM template as the correct user account.

$ xo-cli vm.create bootAfterCreate=true name_label="ddelnano testing resourceSets" template=9b0fd2ac-c89a-93b4-5103-8506391395cd coreOs=false CPUs=1 memoryMax=3221225472 VIFs=json:'[{"network": "54dcb8d8-0af0-7739-61c5-144875958b1e"}]' existingDisks=json:'{"0": {"$SR": "86a9757d-9c05-9fe0-e79a-8243cb1f37f3"}}' resourceSet=_g4tDg2z7dI
d512ea18-ebcc-7a06-00d3-70bfc06e0b62

4. Check that the resourceSet has 1 less CPU, 3221225472 less memory, etc

   $ xo-cli resourceSet.getAll
   [
   {
   "id": "_g4tDg2z7dI",
   "ipPools": [],
   "limits": {
     "cpus": {
       "available": 3,
       "total": 4
     },
     "disk": {
       "available": 103079215104,
       "total": 107374182400
     },
     "memory": {
       "available": 9663676416,
       "total": 12884901888
     }
   },
   "name": "ddelnano-test",
   "objects": [
     "86a9757d-9c05-9fe0-e79a-8243cb1f37f3",
     "9b0fd2ac-c89a-93b4-5103-8506391395cd",
     "54dcb8d8-0af0-7739-61c5-144875958b1e"
   ],
   "subjects": [
     "60f3d51f-43f2-495d-80e1-8939633c468d"
   ]
   }
   ]

5. Try to create a vm with more CPUs than available

   $ xo-cli vm.create bootAfterCreate=true name_label="ddelnano testing resourceSets" template=9b0fd2ac-c89a-93b4-5103-8506391395cd coreOs=false CPUs=4 memoryMax=3221225472 VIFs=json:'[{"network": "54dcb8d8-0af0-7739-61c5-144875958b1e"}]' existingDisks=json:'{"0": {"$SR": "86a9757d-9c05-9fe0-e79a-8243cb1f37f3"}}' resourceSet=_g4tDg2z7dI
   ✖ not enough resources in resource set

[ddelnano]

@dgmcdona I am picturing this working by the following:

1. Create a xenorchestra_resource_set resource and data source. The resource will allow admins to create a resourceSet along with managing the amount of CPU, disk and memory, objects in the resourceSet and subjects (users). The data source will be used by whoever is launching resources associated with the resource set. Their account won't have permissions to create/update a resourceSet but we will want them to be able to look up the resourceSet via the friendly name (rather than hardcode an ID).
2. Update the xenorchestra_vm resource to allow users to provide a resource_set parameter. This will make the provider run the vm.create command I indicated above

Below you will see sample terraform code that would implement what I explained above:

# This file (admin/resource_set.tf) will exist in a separate
# directory from the terraform code students will write

data "xenorchestra_sr" "sr" {
  name_label = "Local storage"
}

data "xenorchestra_template" "template" {
  name = "vm-template"
}

data "xenorchestra_pif" "eth0" {
    device = "eth0"
}
resource "xenorchestra_resource_set" "students_resources" {
  name = "student-resource-set"
  objects = [
    "${data.xenorchestra_template.template.id}",
    "${data.xenorchestra_sr.sr.id}",
    "${data.xenorchestra_pif.eth0.network}",
    # Others could be provided but a minimum of a storage repository,
    # a network and a VM template
  ]
  memory = 12884901888
  disk = 107374182400
  cpus = 20
}

# Separate terraform directory that the students will use
# students/vm1.tf
data "xenorchestra_resource_set" "rs" {
  name = "student-resource-set"
}

data "xenorchestra_sr" "sr" {
  name_label = "Local storage"
}

data "xenorchestra_template" "template" {
  name = "vm-template"
}

data "xenorchestra_pif" "eth0" {
    device = "eth0"
}

resource "xenorchestra_cloud_config" "bar" {
  name = "cloud config name"
  template = <<EOF
#cloud-init

runcmd:
 - [ ls, -l, / ]
 - [ sh, -xc, "echo $(date) ': hello world!'" ]
 - [ sh, -c, echo "=========hello world'=========" ]
 - ls -l /root
EOF
}

resource "xenorchestra_vm" "bar" {
    memory_max = 1073733632
    cpus  = 1
    cloud_config = "${xenorchestra_cloud_config.bar.template}"
    name_label = "Name"
    name_description = "description"
    template = "${data.xenorchestra_template.template.id}"
    network {
    network_id = "${data.xenorchestra_pif.eth0.network}"
    }
   resource_set = "${data.xenorchestra_resource_set.rs.id}"

    disk {
      sr_id = "${data.xenorchestra_sr.sr.id}"
      name_label = "Root volume"
      size = 32212254720 
    }
}

@dgmcdona will what I've outlined above fit your use case?

[dgmcdona]

This looks great! The only thing that I would add, which I was thinking of today, would be the ability to create ACL entries as well. It might be the case that the instructor simply wants to apply the terraform configuration on their machine, and have each student automatically receive access to their own resources via ACL entries. Right now, I am accomplishing this via a python script that uses xo-cli vm.create, followed by xo-cli acl.add. It would be great to be able to deploy this classroom infrastructure entirely via terraform instead. I'm guessing that an ACL entry would need to be a resource of its own, as well.

Unfortunately, only XO admins would be able to use this ACL functionality, which under the current permissions structure cannot even be delegated to pool admins. I have seen some discussion in the XO forums about possible changes to the permission structure in the version 6 release, so we will see what happens there -- I would like to see some kind of 'resource set admin' feature in future releases.

[ddelnano]

I have the bulk of this done in #62. However, I'm waiting on a xenorchestra release (vatesfr/xen-orchestra#5265) since the resourceSet.removeSubject rpc call was broken.

[ddelnano]

I've created #72 to track the ACL support but the functionality I've outlined in my comment above now exists in v0.5.0. So this is complete!