vatesfr / terraform-provider-xenorchestra

Xen Orchestra provider for Terraform
MIT License
151 stars 32 forks source link

vms data soruce proposal #161

Closed gohumble closed 3 years ago

gohumble commented 3 years ago

Hello,

this PR is adding a vms data source. This data source is able to return a list of vms within a pool. Vms can be filtered by power_state and container.

I think there is still a bit more work to be done, like adding more variables/filters and tests.

Usage

When using this as is with #160 you can do this:

data "xenorchestra_pool" "pool" {
  name_label = "My-Pool"
}

data "xenorchestra_hosts" "hosts" {
  pool_id = data.xenorchestra_pool.pool.id
}

data "xenorchestra_vms" "vms" {
  pool_id = data.xenorchestra_pool.pool.id
  power_state = "Running"
  container = data.xenorchestra_pool.pool.master
}

output "vmMemoryUsagePerHost" {
  value = local.vmMemoryUsagePerHost
}

locals {
  vmMemoryUsagePerHost = tomap({
  for k, host in data.xenorchestra_hosts.hosts.hosts : host.id => {
    max_memory : host.memory,
    usedByVm : tomap({for k, vm in data.xenorchestra_vms.vms.vms : vm.id =>  vm.memory_max if vm.container == host.id}),
    used: sum(tolist([for k, vm in data.xenorchestra_vms.vms.vms : vm.memory_max if vm.container == host.id]))
  }
  })
}

OUTPUT

This will output a map where key is the host id and value contains memory information for every VM on this host.

(Only single host here ...)

vmMemoryUsagePerHost = tomap({
  "d43a3737-1e10-4b69-8db5-2d56f82a7ded" = {
    "max_memory" = 68487028736
    "used" = 34359758848
    "usedByVm" = tomap({
      "24abc494-47a6-3cc3-cae9-387712873710" = 4294967296
      "27c177d7-3834-7c36-0207-86ffa8907a1c" = 4294971392
      "36bc1483-3029-8f53-e7d5-b7595696866d" = 8589938688
      "60c59a40-86c0-2470-fc96-1c0cb7c98642" = 4294967296
      "7380f502-adfc-c283-99eb-ee3bb57f066f" = 4294971392
      "8f6c1f96-6f10-1645-11fe-eb4bf2051f64" = 4294971392
      "924640ba-f1c9-5d3c-7ea6-4a1db25c1fe8" = 2147487744
      "b00e6598-9b67-2538-15b8-d167df3978e2" = 2147483648
    })
  }
})

For me, the goal of this was to get some resource information about hosts and vms. This might come handy when trying to spread vms across hosts evenly.

I'm probably going to rethink and test this during the weekend. Appreciate any feedback.

Cheers

gohumble commented 3 years ago

This PR is getting confusing. The conflicts should be resolved by merging the latest upstream master to my forked branch. (or just move from internal.StringInSlice to validation.StringInSlice usage)

ddelnano commented 3 years ago

I rebased this successfully in rebased-gohumble-add-vms-data-source. Let me know if you'd like to continue to review here or if we should open up a new PR with the rebased branch. You can see the diff between this branch and the rebased one here.

$ git diff gohumble/add-vms-data-source origin/rebased-gohumble-add-vms-data-source

diff --git a/client/host.go b/client/host.go
index 307a203..3a2df6d 100644
--- a/client/host.go
+++ b/client/host.go
@@ -8,10 +8,17 @@ import (
 )

 type Host struct {
-       Id        string        `json:"id"`
-       NameLabel string        `json:"name_label"`
-       Tags      []interface{} `json:"tags,omitempty"`
-       Pool      string        `json:"$pool"`
+       Id        string           `json:"id"`
+       NameLabel string           `json:"name_label"`
+       Tags      []interface{}    `json:"tags,omitempty"`
+       Pool      string           `json:"$pool"`
+       Memory    HostMemoryObject `json:"memory"`
+       Cpus      CpuInfo          `json:"cpus"`
+}
+
+type HostMemoryObject struct {
+       Usage int `json:"usage"`
+       Size  int `json:"size"`
 }

 func (h Host) Compare(obj interface{}) bool {
diff --git a/client/storage_repository.go b/client/storage_repository.go
index 1c819c5..ce7efcb 100644
--- a/client/storage_repository.go
+++ b/client/storage_repository.go
@@ -7,12 +7,16 @@ import (
 )

 type StorageRepository struct {
-       Id        string   `json:"id"`
-       Uuid      string   `json:"uuid"`
-       NameLabel string   `json:"name_label"`
-       PoolId    string   `json:"$poolId"`
-       SRType    string   `json:"SR_type"`
-       Tags      []string `json:"tags,omitempty"`
+       Id            string   `json:"id"`
+       Uuid          string   `json:"uuid"`
+       NameLabel     string   `json:"name_label"`
+       PoolId        string   `json:"$poolId"`
+       SRType        string   `json:"SR_type"`
+       Container     string   `json:"$container"`
+       PhysicalUsage int      `json:"physical_usage"`
+       Size          int      `json:"size"`
+       Usage         int      `json:"usage"`
+       Tags          []string `json:"tags,omitempty"`
 }

 func (s StorageRepository) Compare(obj interface{}) bool {
diff --git a/docs/data-sources/host.md b/docs/data-sources/host.md
index 2ee1547..e591fa0 100644
--- a/docs/data-sources/host.md
+++ b/docs/data-sources/host.md
@@ -24,4 +24,12 @@ Terraform will fail. Ensure that your names are unique when
 using the data source.

 ## Attributes Reference
-* id - Id of the host.
+* id - The id of the host.
+* name_label - Name label of the host.
+* pool_id - Id of the pool that the host belongs to.
+* tags - The tags applied to the host.
+* memory - The memory size for the host.
+* memory_usage - The memory usage for the host.
+* cpus - Host cpu information.
+    * cores - The number of cores.
+    * sockets - The number of sockets.
diff --git a/docs/data-sources/hosts.md b/docs/data-sources/hosts.md
index f4bcfe9..854fd33 100644
--- a/docs/data-sources/hosts.md
+++ b/docs/data-sources/hosts.md
@@ -36,7 +36,12 @@ resource "xenorchestra_vm" "vm" {
 ## Attributes Reference
 * master - The primary host of the pool
 * hosts - List containing the matching hosts after applying the argument filtering.
-  * id - The id of the host
-  * name_label - Name label of the host
-  * pool_id - Id of the pool that the host belongs to
-  * tags - The tags applied to the host
+  * id - The id of the host.
+  * name_label - Name label of the host.
+  * pool_id - Id of the pool that the host belongs to.
+  * tags - The tags applied to the host.
+  * memory - The memory size for the host.
+  * memory_usage - The memory usage for the host.
+  * cpus - Host cpu information.
+    * cores - The number of cores.
+    * sockets - The number of sockets.
\ No newline at end of file
diff --git a/docs/data-sources/sr.md b/docs/data-sources/sr.md
index 8ab6190..1a798db 100644
--- a/docs/data-sources/sr.md
+++ b/docs/data-sources/sr.md
@@ -33,3 +33,7 @@ Ensure that your name_label, pool_id and tags identify a unique storage reposito
 * uuid - uuid of the storage repository. This is equivalent to the id.
 * pool_id - The Id of the pool the storage repository exists on.
 * sr_type - The type of storage repository (lvm, udev, iso, user, etc).
+* container - The storage container.
+* size - The storage size.
+* physical_size - The physical storage size.
+* usage - The current usage for this storage repository.
\ No newline at end of file
diff --git a/xoa/data_source_host.go b/xoa/data_source_host.go
index 43b5ad0..f360ce3 100644
--- a/xoa/data_source_host.go
+++ b/xoa/data_source_host.go
@@ -39,6 +39,19 @@ func resourceHostSchema() map[string]*schema.Schema {
                        Type:     schema.TypeString,
                        Computed: true,
                },
+               "cpus": &schema.Schema{
+                       Type:     schema.TypeMap,
+                       Computed: true,
+                       Elem:     &schema.Schema{Type: schema.TypeString},
+               },
+               "memory": &schema.Schema{
+                       Type:     schema.TypeInt,
+                       Computed: true,
+               },
+               "memory_usage": &schema.Schema{
+                       Type:     schema.TypeInt,
+                       Computed: true,
+               },
                "tags": resourceTags(),
        }
 }
@@ -57,6 +70,17 @@ func dataSourceHostRead(d *schema.ResourceData, m interface{}) error {
        }

        d.SetId(hosts[0].Id)
-
+       d.Set("pool_id", hosts[0].Pool)
+       d.Set("memory", hosts[0].Memory.Size)
+       d.Set("memory_usage", hosts[0].Memory.Usage)
+       d.Set("cpus", hostCpuInfoToMapList(hosts[0]))
+       d.Set("tags", hosts[0].Tags)
        return nil
 }
+
+func hostCpuInfoToMapList(host client.Host) map[string]string {
+       return map[string]string{
+               "sockets": fmt.Sprintf("%d", host.Cpus.Sockets),
+               "cores":   fmt.Sprintf("%d", host.Cpus.Cores),
+       }
+}
diff --git a/xoa/data_source_host_test.go b/xoa/data_source_host_test.go
index 1535ab4..cec63e2 100644
--- a/xoa/data_source_host_test.go
+++ b/xoa/data_source_host_test.go
@@ -21,6 +21,10 @@ func TestAccXenorchestraDataSource_host(t *testing.T) {
                                Check: resource.ComposeAggregateTestCheckFunc(
                                        testAccCheckXenorchestraDataSourceHost(resourceName),
                                        resource.TestCheckResourceAttrSet(resourceName, "id"),
+                                       resource.TestCheckResourceAttrSet(resourceName, "cpus.cores"),
+                                       resource.TestCheckResourceAttrSet(resourceName, "cpus.sockets"),
+                                       resource.TestCheckResourceAttrSet(resourceName, "memory"),
+                                       resource.TestCheckResourceAttrSet(resourceName, "memory_usage"),
                                        resource.TestCheckResourceAttr(resourceName, "name_label", accTestHost.NameLabel)),
                        },
                },
diff --git a/xoa/data_source_hosts.go b/xoa/data_source_hosts.go
index 06d458d..f228017 100644
--- a/xoa/data_source_hosts.go
+++ b/xoa/data_source_hosts.go
@@ -46,9 +46,12 @@ func dataSourceHostsRead(d *schema.ResourceData, m interface{}) error {
        if err != nil {
                return err
        }
-       hosts, err := c.GetSortedHosts(client.Host{Pool: pool[0].Id, Tags: tags}, d.Get("sort_by").(string), d.Get("sort_order").(string))
+       searchHost := client.Host{
+               Pool: pool[0].Id,
+               Tags: tags}
+       hosts, err := c.GetSortedHosts(searchHost, d.Get("sort_by").(string), d.Get("sort_order").(string))

-       log.Printf("[DEBUG] found the following hosts: %s", hosts)
+       log.Printf("[DEBUG] found the following hosts: %+v", hosts)
        if err != nil {
                return err
        }
@@ -68,10 +71,13 @@ func hostsToMapList(hosts []client.Host) []map[string]interface{} {
        result := make([]map[string]interface{}, 0, len(hosts))
        for _, host := range hosts {
                hostMap := map[string]interface{}{
-                       "id":         host.Id,
-                       "name_label": host.NameLabel,
-                       "pool_id":    host.Pool,
-                       "tags":       host.Tags,
+                       "id":           host.Id,
+                       "name_label":   host.NameLabel,
+                       "pool_id":      host.Pool,
+                       "tags":         host.Tags,
+                       "memory":       host.Memory.Size,
+                       "memory_usage": host.Memory.Usage,
+                       "cpus":         hostCpuInfoToMapList(host),
                }
                result = append(result, hostMap)
        }
diff --git a/xoa/data_source_hosts_test.go b/xoa/data_source_hosts_test.go
index a005250..244c32a 100644
--- a/xoa/data_source_hosts_test.go
+++ b/xoa/data_source_hosts_test.go
@@ -76,6 +76,10 @@ func getCompositeAggregateTestFunc(resourceName, sortBy, sortOrder string) resou
        return resource.ComposeAggregateTestCheckFunc(
                testAccCheckXenorchestraDataSourceHosts(resourceName),
                resource.TestCheckResourceAttrSet(resourceName, "id"),
+               resource.TestCheckResourceAttrSet(resourceName, "hosts.0.cpus.cores"),
+               resource.TestCheckResourceAttrSet(resourceName, "hosts.0.cpus.sockets"),
+               resource.TestCheckResourceAttrSet(resourceName, "hosts.0.memory"),
+               resource.TestCheckResourceAttrSet(resourceName, "hosts.0.memory_usage"),
                resource.TestCheckResourceAttr(resourceName, "pool_id", accTestPool.Id),
                // Verify that there are atleast 2 hosts returned
                // This is necessary to test the sorting logic
diff --git a/xoa/data_source_xenorchestra_sr.go b/xoa/data_source_xenorchestra_sr.go
index 3b7b544..4479c45 100644
--- a/xoa/data_source_xenorchestra_sr.go
+++ b/xoa/data_source_xenorchestra_sr.go
@@ -28,6 +28,22 @@ func dataSourceXoaStorageRepository() *schema.Resource {
                                Type:     schema.TypeString,
                                Computed: true,
                        },
+                       "container": &schema.Schema{
+                               Type:     schema.TypeString,
+                               Computed: true,
+                       },
+                       "size": &schema.Schema{
+                               Type:     schema.TypeInt,
+                               Computed: true,
+                       },
+                       "physical_usage": &schema.Schema{
+                               Type:     schema.TypeInt,
+                               Computed: true,
+                       },
+                       "usage": &schema.Schema{
+                               Type:     schema.TypeInt,
+                               Computed: true,
+                       },
                        "tags": resourceTags(),
                },
        }
@@ -63,6 +79,10 @@ func dataSourceStorageRepositoryRead(d *schema.ResourceData, m interface{}) erro
        d.Set("sr_type", sr.SRType)
        d.Set("uuid", sr.Uuid)
        d.Set("pool_id", sr.PoolId)
+       d.Set("size", sr.Size)
+       d.Set("physical_usage", sr.PhysicalUsage)
+       d.Set("usage", sr.Usage)
+       d.Set("container", sr.Container)
        return nil
 }

diff --git a/xoa/data_source_xenorchestra_sr_test.go b/xoa/data_source_xenorchestra_sr_test.go
index a63af9f..a2679c7 100644
--- a/xoa/data_source_xenorchestra_sr_test.go
+++ b/xoa/data_source_xenorchestra_sr_test.go
@@ -25,6 +25,10 @@ func TestAccXenorchestraDataSource_storageRepository(t *testing.T) {
                                        resource.TestCheckResourceAttrSet(resourceName, "sr_type"),
                                        resource.TestCheckResourceAttrSet(resourceName, "pool_id"),
                                        resource.TestCheckResourceAttrSet(resourceName, "uuid"),
+                                       resource.TestCheckResourceAttrSet(resourceName, "container"),
+                                       resource.TestCheckResourceAttrSet(resourceName, "size"),
+                                       resource.TestCheckResourceAttrSet(resourceName, "physical_usage"),
+                                       resource.TestCheckResourceAttrSet(resourceName, "usage"),
                                        resource.TestCheckResourceAttr(resourceName, "name_label", accDefaultSr.NameLabel)),
                        },
                },
diff --git a/xoa/internal/validation.go b/xoa/internal/validation.go
deleted file mode 100644
index 2d00e8e..0000000
--- a/xoa/internal/validation.go
+++ /dev/null
@@ -1,27 +0,0 @@
-package internal
-
-import (
-       "fmt"
-       "strings"
-
-       "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
-)
-
-func StringInSlice(valid []string, ignoreCase bool) schema.SchemaValidateFunc {
-       return func(i interface{}, k string) (s []string, es []error) {
-               v, ok := i.(string)
-               if !ok {
-                       es = append(es, fmt.Errorf("expected type of %s to be string", k))
-                       return
-               }
-
-               for _, str := range valid {
-                       if v == str || (ignoreCase && strings.ToLower(v) == strings.ToLower(str)) {
-                               return
-                       }
-               }
-
-               es = append(es, fmt.Errorf("expected %s to be one of %v, got %s", k, valid, v))
-               return
-       }
-}
diff --git a/xoa/resource_xenorchestra_acl.go b/xoa/resource_xenorchestra_acl.go
index 2e8d08e..606340a 100644
--- a/xoa/resource_xenorchestra_acl.go
+++ b/xoa/resource_xenorchestra_acl.go
@@ -2,8 +2,8 @@ package xoa

 import (
        "github.com/ddelnano/terraform-provider-xenorchestra/client"
-       "github.com/ddelnano/terraform-provider-xenorchestra/xoa/internal"
        "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
+       "github.com/hashicorp/terraform-plugin-sdk/v2/helper/validation"
 )

 var validActionOptions = []string{
@@ -36,7 +36,7 @@ func resourceAcl() *schema.Resource {
                                Type:         schema.TypeString,
                                Required:     true,
                                ForceNew:     true,
-                               ValidateFunc: internal.StringInSlice(validActionOptions, false),
+                               ValidateFunc: validation.StringInSlice(validActionOptions, false),
                        },
                },
        }
diff --git a/xoa/resource_xenorchestra_resource_set.go b/xoa/resource_xenorchestra_resource_set.go
index bac2edb..716ec97 100644
--- a/xoa/resource_xenorchestra_resource_set.go
+++ b/xoa/resource_xenorchestra_resource_set.go
@@ -4,8 +4,8 @@ import (
        "log"

        "github.com/ddelnano/terraform-provider-xenorchestra/client"
-       "github.com/ddelnano/terraform-provider-xenorchestra/xoa/internal"
        "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
+       "github.com/hashicorp/terraform-plugin-sdk/v2/helper/validation"
 )

 var validLimitType []string = []string{"cpus", "disk", "memory"}
@@ -46,7 +46,7 @@ func resourceResourceSet() *schema.Resource {
                                                "type": &schema.Schema{
                                                        Type:         schema.TypeString,
                                                        Required:     true,
-                                                       ValidateFunc: internal.StringInSlice(validLimitType, false),
+                                                       ValidateFunc: validation.StringInSlice(validLimitType, false),
                                                },
                                                "quantity": &schema.Schema{
                                                        Type:     schema.TypeInt,
diff --git a/xoa/resource_xenorchestra_vm.go b/xoa/resource_xenorchestra_vm.go
index 82eb4ef..abe416f 100644
--- a/xoa/resource_xenorchestra_vm.go
+++ b/xoa/resource_xenorchestra_vm.go
@@ -13,6 +13,7 @@ import (
        "github.com/ddelnano/terraform-provider-xenorchestra/client"
        "github.com/ddelnano/terraform-provider-xenorchestra/xoa/internal"
        "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
+       "github.com/hashicorp/terraform-plugin-sdk/v2/helper/validation"
 )

 var validHaOptions = []string{
@@ -33,10 +34,12 @@ func resourceVm() *schema.Resource {
                Type:     schema.TypeString,
                Required: true,
        }
+
        return &schema.Resource{
                Schema: vmSchema,
        }
 }
+
 func resourceVmSchema() map[string]*schema.Schema {
        return map[string]*schema.Schema{
                "affinity_host": &schema.Schema{
@@ -67,16 +70,14 @@ func resourceVmSchema() map[string]*schema.Schema {
                "installation_method": &schema.Schema{
                        Type:          schema.TypeString,
                        Optional:      true,
-                       ValidateFunc:  internal.StringInSlice(validInstallationMethods, false),
+                       ValidateFunc:  validation.StringInSlice(validInstallationMethods, false),
                        ConflictsWith: []string{"cdrom"},
                },
                "high_availability": &schema.Schema{
-                       Type:     schema.TypeString,
-                       Default:  "",
-                       Optional: true,
-                       // TODO: Replace with validation.StringInSlice when terraform
-                       // and the SDK are upgraded.
-                       ValidateFunc: internal.StringInSlice(validHaOptions, false),
+                       Type:         schema.TypeString,
+                       Default:      "",
+                       Optional:     true,
+                       ValidateFunc: validation.StringInSlice(validHaOptions, false),
                },
                "template": &schema.Schema{
                        Type:     schema.TypeString,
ddelnano commented 3 years ago

For some reason the comments are showing as "outdated" and I can't comment on them. But I believe I've responded to your new comments.

Screenshot_20210714_210102