search

ComputeCanada / magic_castle

Terraform modules to replicate the HPC user experience in the cloud
MIT License
124 stars 36 forks source link

Hieradata per instances group #291

Closed etiennedub closed 5 months ago

etiennedub commented 7 months ago

Close #287

This PR allows for setting custom hieradata per group of instances sharing the same prefix.

It works by setting a new prefix fact on each instance. Then, all prefix hieradata are gathered in one file (prefix.yaml) and copied to mgmt. The hieradata are mapped based on the prefix and parsed accordingly by Puppet.

This approach differs from the task definition in the issue. Initially, the idea was to copy one file per prefix, but Terraform provisioners don't support dynamic, and the number of files is only known at runtime.

See https://github.com/ComputeCanada/puppet-magic_castle/pull/324 for puppet logic

cmd-ntrf commented 7 months ago

Since the end result is a single YAML file instead of one per prefix, I am wondering if it would be better in terms of user experience to define the hieradata per prefix as a single data structure in the main.tf instead of one per instance type. i.e.:

hieradata_prefix = <<EOT
login:
  profile::base::packages: ['gcc', 'gfortran']
mgmt:
  profile::ceph::client::share_name: "your-project-shared-fs"
EOT
cmd-ntrf commented 7 months ago

Last bit of this comment: https://github.com/hashicorp/terraform/issues/21425#issuecomment-495680010 suggests uploading a directory content to handle multiple files.

cmd-ntrf commented 7 months ago

Another option would be to have a resource "terraform_data" per hieradata file.