Using the libvirt provider, not in the Terraform registry throws `Failed to query available provider packages`

[larssb]

Checklist

• [X] Upgrade Terraspace: Are you using the latest version of Terraspace? This allows Terraspace to fix issues fast.
• [X] Reproducibility: Are you reporting a bug others will be able to reproduce and not asking a question. If you're unsure or want to ask a question, do so on StackOverflow.
• [X] Code sample: Have you put together a code sample to reproduce the issue and make it available? Code samples help speed up fixes dramatically. If it's an easily reproducible issue, then code samples are not needed. If you're unsure, please include a code sample.

My Environment

Software	Version
Operating System	CentOS 8
Terraform	v0.13.5
Terraspace	v0.5.10
Ruby	ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux]

---

Expected Behaviour

I'm expecting/hoping to be able to use this libvirt provider with Terraspace. For now it can only be used as a local provider until the project have uploaded it to the HashiCorp Terraform registry.

Current Behavior

When executing terrspace up STACK_NAME Terraspace throws the following error:

Error: Failed to query available provider packages

Could not retrieve the list of available versions for provider
hashicorp/libvirt: provider registry.terraform.io/hashicorp/libvirt was not
found in any of the search locations

- /root/terraform-providers/

Re-producing

1. execute terraspace new project WHATEVER_NAME
2. execute terraspace new stack test-k3s-cluster
3. execute terraspace new module k3s-cluster

the files relative to the stack: test-k3s-cluster should contain

• Gist main.tf, find it here
• Gist variables.tf, find it here
• outputs.tf. that one is empty so no Gist for that

the files relative to the module: k3s-cluster should contain

• Gist main.tf, find it here
• variables.tf, contains the same as the one in the stack
• outputs.tf, that one is empty so no Gist for that

Terraspace/Terraform config files should contain:

• Gist backend.tf, find it here
• Gist provider.tf, find it here
• Gist terraform.tf, find it here

The project structure is:

.
├── app
│   ├── modules
│   │   └── k3s-cluster
│   │       ├── conf
│   │       │   ├── network-config.yaml
│   │       │   └── user-data.yaml
│   │       ├── main.tf
│   │       ├── outputs.tf
│   │       └── variables.tf
│   └── stacks
│       └── test-k3s-cluster
│           ├── main.tf
│           ├── outputs.tf
│           └── variables.tf
├── config
│   ├── app.rb
│   └── terraform
│       ├── backend.tf
│       ├── provider.tf
│       └── terraform.tf
├── Gemfile
├── Gemfile.lock
├── README.md
└── Terrafile

1. execute terraspace up test-k3s-cluster and the err should be seen after some time

Logs

The below is the Terraspace run log

Initializing modules...
- k3s-cluster in ../../modules/k3s-cluster

Initializing the backend...

Successfully configured the backend "remote"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding local/providers/libvirt versions matching "~> 0.6.3"...
- Finding latest version of hashicorp/libvirt...
- Installing local/providers/libvirt v0.6.3...
- Installed local/providers/libvirt v0.6.3 (unauthenticated)
Initializing modules...
- k3s-cluster in ../../modules/k3s-cluster

Initializing the backend...

Successfully configured the backend "remote"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding local/providers/libvirt versions matching "~> 0.6.3"...
- Finding latest version of hashicorp/libvirt...
- Using local/providers/libvirt v0.6.3 from the shared cache directory

Other comments

• I'm also befuddled on why the cache is created at this path: .terraspace-cache/us-east-1/dev/stacks/test-k3s-cluster/... as I have no region specified anywhere.

• Also its like Terraspace thinks that it should interact with AWS. As these following warnings are displayed.

Building .terraspace-cache/us-east-1/dev/stacks/test-k3s-cluster
INFO: You're missing AWS credentials. Only local services are currently available
INFO: You're missing AWS credentials. Only local services are currently available
Built in .terraspace-cache/us-east-1/dev/stacks/test-k3s-cluster
Syncing to Terraform Cloud: test-k3s-cluster => hospites-dev

** Also likely the culprit to the region mishap described above

• Its also strange that Terraspace believes I'm trying to use Terraform Cloud as the only feature of Terraform Cloud I'm using is the remote state feature.

---

Looking forward to all the help I can get. Any help is appreciated.

Thank you very much

[larssb]

And btw. The libvirt provider IS installed and works in a Terragrunt/Terraform setting.

[tongueroo]

Oh I see. Bummer. I was incorrect about terraspace working with only on-prem resources only. Just tested it. A few notes:

• When generating a new project, terraspace uses the aws plugin by default. You can see it in the Gemfile.
• There's currently no "onprem" plugin yet though. Would like to eventually see pure onprem working though also. Unsure when will dig into this. Will also consider PRs. 🧐

Note: Also tested with TFC and AWS also this morning and that works: https://gist.github.com/tongueroo/6f20cf014406021fc9ebf3f7f1bc139b

[larssb]

I'm no Ruby expert. But, I really like this framework and the way it tries to keep things DRY. However, maybe you could give me some pointers on where to start? It sounds like the tasks are:

1. Get rid of the default use of AWS
2. Get an on-prem plugin started

Does that sound about right?

Thank you

[tongueroo]

Sure. Thanks for the kind words. That sounds about right.

RE: plugin

Note, there's a plugin generator https://terraspace.cloud/reference/terraspace-new-plugin/ Created it after built 2 plugins. It's probably does not have the up-to-date interface anymore, but thinking it'll help. Lot of the terraspace plugin interfaces are based on dogfooding. The interfaces will probably change based on that dogfooding. Found that's usually the best way to build them. So there are some cavaets. It's also decent amount of effort. Of course, no sweat either way 👍