There are no functions in namespace "provider:assert::"

[b0bu]

Hey, getting a namespace error, really not sure where to go next with this one. It's like it can't see the provider. What's the recommended way to use this provider in tests, i.e. where should it be initialized from to make it available?

terraform version 1.9.2
assert provider 0.9.3

I'm gettting the following error when using http_success() as a condition

Call to unknown function

tests/integration_tests.tftest.hcl line 63, in run "test_created":
63     condition = provider::assert::http_success(data.http.get.status_code)

  data.http.get.status_code is 404

There are no functions in namespace "provider:assert::".

---

The code defiens an output called url, and the helper module defines http data

provider assert {}

run "test_create" {
  command = apply
}

run "test_created" {
  command = plan

  module {
   source = "./tests/final"
  }

  variables {
    url = run.test_create.url
  }

  assert {
    condition = provider::assert::http_success(data.http.get.url.status_code)
    error_message = "stuff"
  }
}

[bschaatsbergen]

Hey @b0bu,

Thanks for raising this issue. For provider-defined functions to be available you must declare the provider as a required provider somewhere in the directory, typically this is done in a providers.tf for instance.

Like this:

terraform {
  required_version = ">= 1.8.0"
  required_providers {
    assert = {
      source  = "bschaatsbergen/assert"
      version = "0.9.3"
    }
  }
}

Let me know if adding this to your Terraform configuration makes it work 😁

[b0bu]

@bschaatsbergen

I did have the required_providers in a providers.tf. In the root of my module and in tests/ while testing where the issue was coming from. It seems community providers don't initialize in either case I noticed after deleting my .terraform dir and re-initializing. The only way I could get it to work was to create a setup module, containing the exact code you have above. Creating a

run "setup" {

  module {
    source = "./tests/setup"
  }

}

made the provider available to the tests but I couldn't get it to work any other way. I maybe have jumped to conclusions way too early. I'll do some more testing.

[b0bu]

So I've tried required_providers at the root of the modules, in tests/ and the only way to download the providers is to add the setup code above, but I'm still getting the same error. After what looks like a successful init I've tried making both run blocks apply incase there was something more sinister going on but same error There are no functions in namespace "provider::assert::"

[b0bu]

After downloading the providers in the init (I hesitate to say successfully initing) the providers look like this: (as mentioned initing from root does not work. I tried to be explicit with the provider:

provider "assert" {}

run "" {

  providers = {
    assert = assert
  }

}

I got the error

This configureation requires provider registry.terraform.io/hashicorp/assert, but that provider isn't available. You may be able to install it automatically by running: terraform init

I also found this https://github.com/hashicorp/terraform/issues/35160 and wondered if they were related.

[b0bu]

From experiment this provider doesn't work in .tftest.hcl files... I’m not sure the extent of this limitation but I assume it's something to do with how the test framework is dealing with providers and I can only report what I’m seeing. So I’m left wondering in what scenario does this work in? There's obviously an example of using this is run blocks in the read me and I assume others are using it ok. I’m not doing anything weird or crazy in my setup and yet I can’t make this provider available regardless how it’s initialized.

You can’t reference this provider inside of a called module i.e. a module that should be independently tested, released and versioned - I’m having a hard time accepting that statement as a disclaimer. Some structural pseudo code to explain what I mean by this. Typical re-usable structure where providers are inherited or passed explicitly from the caller we’ll call it module1 as independently released and versioned module:

module1/
|_main.tf
|_providers.tf                    A

If it were tested using the terraform test framework it should have some internal dev requirements that exist as part of module1 ci.

When adding tests to this

module1/
|_main.tf
|_providers.tf                    A
|_tests/
  |_main.tftest.hcl
  |_providers.tf                  B

A and B are not possible because they’re ignored, not by terraform core workflow but by testing. You have to init from the root of module1 in order to test and not from module1/tests so structurally, B makes no sense anyway. However theoretically, if it did work, any requirements you place in B are requirements to testing module1 not to using module1. Adding requirements for testing to A also makes no sense as they’re dev dependencies and not dependencies to using the module.

Test provider blocks can be added to any .tftest.hcl as each file is completely namespaced so you can’t have a providers.tftest.hcl filled with providers that all.tftest.hcl files re-use, it can’t seem them. Point being, providers for tests work by defining provider which can’t pull the assert provider when initing, again from the root of module1. Again I’m not sure what other limitations exist here but I know this can pull hashicorp signed providers.

It seems the provider system for the test framework is missing some core concepts. You can force the provider to be pulled down in 2 scenarios the first one is where you define the provider as “required” in A (which makes no sense dependecy wise) and this does not make it available to tests only caches it locally and results in the error I’m getting above. The other is when you add a required_providers block as part of a test sub module and pull it is as setup as mentioned above, again this caches it locally and is also ignored by tests.

[bschaatsbergen]

Hey @b0bu, thank you for this deep dive you've done. I'm trying to get a better understanding of how you're exactly running into this, because I can't seem to reproduce it. I'm able to use the functions, using terraform test and the .tftest.hcl files.

With a folder structure like this:

├── main.tf
├── providers.tf
└── tests
    └── main.tftest.hcl

My main.tf looking like:

data "http" "terraform_io" {
  url = "https://www.terraform.io"
}

My providers.tf looking like this:

terraform {
  required_providers {
    assert = {
      source  = "bschaatsbergen/assert"
      version = "0.9.3"
    }
  }
}

And my main.tftest.hcl looking like this:

run "check_health" {

  command = plan

  assert {
    condition     = provider::assert::http_success(data.http.terraform_io.status_code)
    error_message = "Failed to fetch https://www.terraform.io"
  }
}

When running terraform test in the root directory (where my main.tf lives), I'm able to just use the provider-defined functions from this provider.

 $ terraform test
tests/main.tftest.hcl... in progress
  run "check_health"... pass
tests/main.tftest.hcl... tearing down
tests/main.tftest.hcl... pass

Success! 1 passed, 0 failed.

Changing the assertion to an incorrect assertion also seems to work:

 $ terraform test
tests/main.tftest.hcl... in progress
  run "check_health"... fail
╷
│ Error: Test assertion failed
│
│   on tests/main.tftest.hcl line 6, in run "check_health":
│    6:     condition     = provider::assert::http_client_error(data.http.terraform_io.status_code)
│     ├────────────────
│     │ data.http.terraform_io.status_code is 200
│
│ Failed to fetch https://www.terraform.io
╵
tests/main.tftest.hcl... tearing down
tests/main.tftest.hcl... fail

Failure! 0 passed, 1 failed.

I'm on Terraform version 1.9.2

[b0bu]

@bschaatsbergen Hey thanks for the response. I'll recreate your scenario and get back to you. Potentially one different is that I'm using the testing framework for a callable module and not a root caller module so ultimately in this scenario providers.tf wouldn't exist since they're passed in from the caller or inherited. I awkwardly named my file terraform.tf in the example above (so I'll rename it to providers for consistency) but I was putting the same provider block as you had in there with the same result.

├── main.tf
├── providers.tf
└── tests
    └── main.tftest.hcl

[b0bu]

@bschaatsbergen Ah so... My scenario is more closely mimicked by this structure.

.
├── main.tf
├── providers.tf
└── tests
    ├── final
    │   └── main.tf
    └── main.tftest.hcl

What you've done is added the data block to main.tf. In my more elaborate scenario, the data block is not part of my module code, my module code is using the integrations/github provider. The data http block for me is a dev dependency only required for the test. So I'm loading it via a called to module {} at source tests/final. I have no need for the data look up outside of this context.

my main.tf

// do github related things

my providers.tf

terraform {
  required_providers {
    assert = {
      source  = "bschaatsbergen/assert"
      version = "0.9.3"
    }
  }
}

my main.tftest.hcl

run "check_health" {

  command = plan

  module {
    source = "./tests/final"
  }

  assert {
    condition     = provider::assert::http_success(data.http.terraform_io.status_code)
    error_message = "Failed to fetch https://www.terraform.io"
  }
}

result is

terraform test
tests/main.tftest.hcl... in progress
  run "check_health"... fail
╷
│ Error: Call to unknown function
│
│   on tests/main.tftest.hcl line 10, in run "check_health":
│   10:     condition     = provider::assert::http_success(data.http.terraform_io.status_code)
│     ├────────────────
│     │ data.http.terraform_io.status_code is 200
│
│ There are no functions in namespace "provider::assert::".
╵
tests/main.tftest.hcl... tearing down
tests/main.tftest.hcl... fail

Failure! 0 passed, 1 failed.

switching my condition to this works

    condition     = data.http.terraform_io.status_code == 200

[bschaatsbergen]

Hmm @b0bu, I would love to work on this with you. Including a required_providers entry in the module that's being tested should make that provider's functions available to the test command as far as I know. Would you be open to hopping on a call to look at this together?

[b0bu]

@bschaatsbergen sure, happy to. What's your collaboration app of choice? Teams, slack other?

[liamcervante]

Hi @b0bu, thanks for the interest here!

I think you're missing a required_providers block within the submodule you are trying to test? Would you mind sharing the output of running the terraform providers command? If everything is set up correctly, I'd expect to see something similar to:

Providers required by configuration:
.
├── provider[registry.terraform.io/hashicorp/http]
├── provider[registry.terraform.io/bschaatsbergen/assert] 0.9.3
└── test.tests.main
    └── run.check_health
        ├── provider[registry.terraform.io/bschaatsbergen/assert] 0.9.3
        └── provider[registry.terraform.io/hashicorp/http]

You can see under run.check_health the assert provider is linked at the correct address. If that is missing then it means the provider isn't being loaded for that run block.

There is a known limitation within Terraform in general that it defaults to hashicorp providers by default, and since the assert provider isn't a hashicorp provider you need to specify the full qualified name in every module via the required_providers block.

[b0bu]

Hi, well as mentioned adding assert to the root of my module makes it a functional dependency. It's a non-functional requirement so I can't do that. Since my condition in main.tftest.hcl but data.http is being loaded via a helper module final

run "check_health" {

  command = plan

  module {
    source = "./tests/final"
  }

  assert {
    condition     = provider::assert::http_success(data.http.get.status_code)
    error_message = "Failed to fetch https://www.terraform.io"
  }
}

given this structure

.
├── main.tf
├── providers.tf
└── tests
    ├── final
    │   └── main.tf
    └── main.tftest.hcl

what actually needed to happen was that the required_providers block given above needed to be added to tests/final/providers.tf since that module is pulled in for 2 of my tests namely the _created tests placing the provider anywhere else doesn't work as explored above.

edit which is weird to me since it's not being used in that module but outside of that module in a test. But from an internal and providers perspective it puts it in the right namespace.

[bschaatsbergen]

Hey @b0bu,

I’m glad you figured out the issue. I’ll close this issue since it appears to be a limitation in Terraform core as you’ve described. Please feel free to raise a similar issue in the Terraform core repository: https://github.com/hashicorp/terraform/issues/new/choose.

Thank you for your detailed analysis and for using this provider. Your feedback is very valuable!

[bschaatsbergen]

Hey @b0bu 👋, just a heads up—we've moved this Terraform provider to the hashicorp namespace. If you’re using it, please consider migrating to hashicorp/assert. Thanks again for your valuable feedback earlier!

[b0bu]

Hey @b0bu 👋, just a heads up—we've moved this Terraform provider to the hashicorp namespace. If you’re using it, please consider migrating to hashicorp/assert. Thanks again for your valuable feedback earlier!

@bschaatsbergen will do thanks!