Panic + schemas not found.

[jlpedrosa]

tfplugingen-openapi CLI version

tfplugingen-openapi module: v0.3.0

OpenAPI Spec File

File is very large, copied here: https://gist.github.com/jlpedrosa/8c2718f71cd0c7da63d8ba76a2ce6c9b 

(TrueNas swagger)

Generator Config

provider:
  name: truenas
resources:
  api_key:
    create:
      path: /api_key
      method: POST
    read:
      path: /api_key/{id}
      method: GET
    update:
      path: /api_key
      method: PUT
    delete:
      path: /api_key/{id}
      method: DELETE

Expected Behavior

Not a crash :)

Actual Behavior

Panic + warning logs that seem incorrect.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7cb786]

goroutine 1 [running]:
github.com/pb33f/libopenapi/index.(*CircularReferenceResult).GenerateJourneyPath(0xc0039d6080?)
        /home/jose/go/pkg/mod/github.com/pb33f/libopenapi@v0.15.0/index/circular_reference_result.go:20 +0x26
github.com/hashicorp/terraform-plugin-codegen-openapi/internal/cmd.(*GenerateCommand).runInternal(0xc00014f580, 0xc00003fba0)
        /home/jose/go/pkg/mod/github.com/hashicorp/terraform-plugin-codegen-openapi@v0.3.0/internal/cmd/generate.go:139 +0x285
github.com/hashicorp/terraform-plugin-codegen-openapi/internal/cmd.(*GenerateCommand).Run(0xc00014f580, {0xc00012e020, 0x5, 0x5})
        /home/jose/go/pkg/mod/github.com/hashicorp/terraform-plugin-codegen-openapi@v0.3.0/internal/cmd/generate.go:100 +0x2c5
github.com/hashicorp/cli.(*CLI).Run(0xc00017fb80)
        /home/jose/go/pkg/mod/github.com/hashicorp/cli@v1.1.6/cli.go:265 +0x5b8
main.runCLI({0xc8706a, 0x13}, {0xc0001366f0, 0x22}, {0xc00012e010, 0x6, 0x6}, {0xd937e0, 0xc00011a040}, {0xd93800, ...}, ...)
        /home/jose/go/pkg/mod/github.com/hashicorp/terraform-plugin-codegen-openapi@v0.3.0/cmd/tfplugingen-openapi/main.go:67 +0x348
main.main()
        /home/jose/go/pkg/mod/github.com/hashicorp/terraform-plugin-codegen-openapi@v0.3.0/cmd/tfplugingen-openapi/main.go:23 +0x12b

Additional Information

Also logs shows many schemas not found, that seem correct via manual inspection: ie:

But I can find the relevant json:

  "return_schema_of_vmware_query": {
    "anyOf": [
     {
      "_name_": "query_result",
      "title": "query_result",
      "default": [],
      "_required_": false,
      "type": "array",
      "items": [
       {
        "type": "object",
        "properties": {
         "datastore": {
          "_name_": "datastore",
          "title": "datastore",
          "_required_": true,
          "type": "string"

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[bflad]

Hi @jlpedrosa 👋 Thank you for raising this and sorry you ran into trouble here.

I just tried the provided TrueNAS OpenAPI specification with the latest terraform-plugin-codegen-openapi on the main branch and it appears that the panics are now proper errors, e.g.

{"time":"2024-04-25T10:55:25.274613-04:00","level":"ERROR","msg":"error building path item: schema build failed: reference '#components/schemas/return_schema_of_core_debug_mode_enabled' cannot be found at line 71, col 18"}
{"time":"2024-04-25T10:55:25.275-04:00","level":"ERROR","msg":"error building path item: schema build failed: reference '#components/schemas/return_schema_of_core_get_jobs' cannot be found at line 138, col 18"}
{"time":"2024-04-25T10:55:25.276322-04:00","level":"ERROR","msg":"error building path item: schema build failed: reference '#components/schemas/return_schema_of_core_sessions' cannot be found at line 342, col 18"}
... etc ...

I'm guessing the panic versus error difference is some bug fixes in the underlying OpenAPI handling library that this tooling uses. I will ask the maintaining team if they believe we should cut a bug fix release of terraform-plugin-codegen-openapi to at least make these sorts of situations more clear for everyone. I would treat that sort of bug fix release as closing this issue from the standpoint that the tooling would no longer unexpectedly panic in this situation and instead would be raising actual errors for troubleshooting.

As for the actual errors themselves, I believe the OpenAPI specification parsing is complaining because some of those references are #components/schemas/... instead of #/components/schemas/... (note the lack of beginning /). You may want to double check if there is an updated OpenAPI specification file that resolves that sort of issue, or if that specification is seen as valid, we can raise a bug fix or feature request with the OpenAPI handling library.

[bflad]

Hi again 👋 After further testing, it appears replacing the #components/schemas/... references with #/components/schemas/... is what will prevent the panic from occurring in my environment. Having the OpenAPI specification be fully valid is a requirement of this tool, so I'm not sure if there is much else we can do in this tooling since we are quite dependent on the underlying OpenAPI parsing library behaviors when it encounters all the original errors but still tries to continue processing the specification to the point of it raising its own panic. It seems like the next best step might be raising an issue in https://github.com/pb33f/libopenapi/issues to potentially do something different to prevent the panic its raising.

[jlpedrosa]

Hi @bflad Thanks! I tracked down the issue with the provider with the specification and indeed they are working on a fix! thanks!

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.