search

databricks / terraform-provider-databricks

Databricks Terraform Provider
https://registry.terraform.io/providers/databricks/databricks/latest
Other
445 stars 384 forks source link

Nil Pointer reference on calculateLibraryChanges #55

Closed psg2 closed 4 years ago

psg2 commented 4 years ago

The method tries to check if the libraries have a non-empty string for its name but for the Pypi, Maven and Cran libraries are pointer to structs, so when it tries to check the len like the following, it can lead to an exception.

if len(library.Pypi.Package) > 0
sdebruyn commented 4 years ago

I got the same error with 0.2.0

module.azure-datalake.databricks_cluster.cluster: Modifying... [id=0610-175407-baton141]

Error: rpc error: code = Canceled desc = context canceled

panic: runtime error: invalid memory address or nil pointer dereference
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xd67367]
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: 
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: goroutine 42 [running]:
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: github.com/databrickslabs/databricks-terraform/databricks.calculateLibraryChanges(0xc00046b400, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x24, 0xc0003d6960, ...)
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0:   /home/runner/work/databricks-terraform/databricks-terraform/databricks/resource_databricks_cluster.go:895 +0x197
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: github.com/databrickslabs/databricks-terraform/databricks.resourceClusterUpdate(0xc0000ed5e0, 0x10388e0, 0xc00042c400, 0x24, 0x1a1f6c0)
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0:   /home/runner/work/databricks-terraform/databricks-terraform/databricks/resource_databricks_cluster.go:1057 +0x1df
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Resource).Apply(0xc00044c360, 0xc0000a5810, 0xc00000d180, 0x10388e0, 0xc00042c400, 0xc00045dc01, 0xc0003bc420, 0xc00045dd08)
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0:   /home/runner/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.13.0/helper/schema/resource.go:316 +0x263
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Provider).Apply(0xc000106a80, 0xc0001d19c8, 0xc0000a5810, 0xc00000d180, 0xc00014d048, 0xc00000eaa0, 0xef6320)
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0:   /home/runner/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.13.0/helper/schema/provider.go:294 +0x99
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ApplyResourceChange(0xc00000ebd8, 0x12c8f60, 0xc00017b7a0, 0xc0000ec070, 0xc00000ebd8, 0xc00017b7a0, 0xc0001c9b30)
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0:   /home/runner/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.13.0/internal/helper/plugin/grpc_provider.go:885 +0x8b4
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ApplyResourceChange_Handler(0x10307e0, 0xc00000ebd8, 0x12c8f60, 0xc00017b7a0, 0xc000078600, 0x0, 0x12c8f60, 0xc00017b7a0, 0xc00019f500, 0x91c)
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0:   /home/runner/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.13.0/internal/tfplugin5/tfplugin5.pb.go:3305 +0x217
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: google.golang.org/grpc.(*Server).processUnaryRPC(0xc000430780, 0x12d4b60, 0xc000431500, 0xc0000d8600, 0xc000084d20, 0x19f2c40, 0x0, 0x0, 0x0)
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0:   /home/runner/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:1024 +0x4f4
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: google.golang.org/grpc.(*Server).handleStream(0xc000430780, 0x12d4b60, 0xc000431500, 0xc0000d8600, 0x0)
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0:   /home/runner/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:1313 +0xd97
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc00003c420, 0xc000430780, 0x12d4b60, 0xc000431500, 0xc0000d8600)
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0:   /home/runner/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:722 +0xbb
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0: created by google.golang.org/grpc.(*Server).serveStreams.func1
2020-06-11T16:13:30.533Z [DEBUG] plugin.terraform-provider-databricks_v0.2.0:   /home/runner/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:720 +0xa1
2020-06-11T16:13:30.534Z [DEBUG] plugin: plugin process exited: path=/home/vsts/.terraform.d/plugins/terraform-provider-databricks_v0.2.0 pid=3638 error="exit status 2"
2020/06/11 16:13:30 [DEBUG] module.azure-datalake.databricks_cluster.cluster: apply errored, but we're indicating that via the Error pointer rather than returning it: rpc error: code = Canceled desc = context canceled
2020/06/11 16:13:30 [TRACE] module.azure-datalake: eval: *terraform.EvalMaybeTainted
2020/06/11 16:13:30 [TRACE] module.azure-datalake: eval: *terraform.EvalWriteState
2020/06/11 16:13:30 [TRACE] EvalWriteState: recording 4 dependencies for module.azure-datalake.databricks_cluster.cluster
2020/06/11 16:13:30 [TRACE] EvalWriteState: writing current state object for module.azure-datalake.databricks_cluster.cluster
2020/06/11 16:13:30 [TRACE] module.azure-datalake: eval: *terraform.EvalApplyProvisioners
2020/06/11 16:13:30 [TRACE] EvalApplyProvisioners: databricks_cluster.cluster is not freshly-created, so no provisioning is required
2020/06/11 16:13:30 [TRACE] module.azure-datalake: eval: *terraform.EvalMaybeTainted
2020/06/11 16:13:30 [TRACE] module.azure-datalake: eval: *terraform.EvalWriteState
2020/06/11 16:13:30 [TRACE] EvalWriteState: recording 4 dependencies for module.azure-datalake.databricks_cluster.cluster
2020/06/11 16:13:30 [TRACE] EvalWriteState: writing current state object for module.azure-datalake.databricks_cluster.cluster
2020/06/11 16:13:30 [TRACE] module.azure-datalake: eval: *terraform.EvalIf
2020/06/11 16:13:30 [TRACE] module.azure-datalake: eval: *terraform.EvalIf
2020/06/11 16:13:30 [TRACE] module.azure-datalake: eval: *terraform.EvalWriteDiff
2020/06/11 16:13:30 [TRACE] module.azure-datalake: eval: *terraform.EvalApplyPost
2020/06/11 16:13:30 [ERROR] module.azure-datalake: eval: *terraform.EvalApplyPost, err: rpc error: code = Canceled desc = context canceled
2020/06/11 16:13:30 [ERROR] module.azure-datalake: eval: *terraform.EvalSequence, err: rpc error: code = Canceled desc = context canceled
2020/06/11 16:13:30 [TRACE] [walkApply] Exiting eval tree: module.azure-datalake.databricks_cluster.cluster
2020/06/11 16:13:30 [TRACE] vertex "module.azure-datalake.databricks_cluster.cluster": visit complete
2020/06/11 16:13:30 [TRACE] dag/walk: upstream of "module.azure-datalake.provider.databricks (close)" errored, so skipping
2020/06/11 16:13:30 [TRACE] dag/walk: upstream of "module.azure-datalake.output.databricks_cluster_id" errored, so skipping
2020/06/11 16:13:30 [TRACE] dag/walk: upstream of "output.databricks_cluster_id" errored, so skipping
2020/06/11 16:13:30 [TRACE] dag/walk: upstream of "meta.count-boundary (EachMode fixup)" errored, so skipping
2020/06/11 16:13:30 [TRACE] dag/walk: upstream of "root" errored, so skipping
2020-06-11T16:13:30.792Z [DEBUG] plugin: plugin exited

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
A crash log has been placed at "crash.log" relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version. That
information is available on the first line of crash.log. You can also
get it by running 'terraform --version' on the command line.

SECURITY WARNING: the "crash.log" file that was created may contain 
sensitive information that must be redacted before it is safe to share 
on the issue tracker.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!
stikkireddy commented 4 years ago

@sdebruyn will pick this up with a acceptance test for clusters, been putting this off for a bit till I had some time as running acceptance tests for clusters is quite time consuming

stikkireddy commented 4 years ago

this should be captured in this https://github.com/databrickslabs/databricks-terraform/blob/fa5daa7721770a28b75e5cf530351b04f56a785f/databricks/resource_databricks_cluster.go for pr #102 but I will be adding a new acceptance test for clusters to verify libraries

stikkireddy commented 4 years ago

@sdebruyn can you verify that this is still an issue?

sdebruyn commented 4 years ago

I can confirm that this is fixed AFAIK :) I just removed some libraries from my cluster and it worked fine.

EliiseS commented 4 years ago

@stikkireddy This should be closed :)