Closed sk4zuzu closed 9 months ago
Additionally if you look at the Terraform configuration
section, the question would be if there is a better way to download a marketapp into OpenNebula, than downloading each image and template directly ? :thinking:
This issue is stale because it has been open for 30 days with no activity and it has not the 'status: confirmed' label or it is not in a milestone. Remove the 'status: stale' label or comment, or this will be closed in 5 days.
Closing this issue. After several tests, I've checked that this issue has been fixed by this PR.
Hello @vickmp,
I'm afraid we're not out of the woods yet.. :cry:
I repeated my tests from July and:
timeout block works just fine! :ok_hand:
when I stopped the apply operation with CRTL+C and tried the destroy operation, I got:
opennebula_service.oneke: Still creating... [30s elapsed]
opennebula_service.oneke: Still creating... [40s elapsed]
opennebula_service.oneke: Still creating... [50s elapsed]
opennebula_service.oneke: Still creating... [1m0s elapsed]
opennebula_service.oneke: Still creating... [1m10s elapsed]
opennebula_service.oneke: Still creating... [1m20s elapsed]
^C
Interrupt received.
Please wait for Terraform to exit or data loss may occur.
Gracefully shutting down...
Stopping operation...
╷
│ Error: execution halted
│
│
╵
╷
│ Error: execution halted
│
│
╵
╷
│ Error: Failed to wait service to be in RUNNING state
│
│ with opennebula_service.oneke,
│ on main.tf line 135, in resource "opennebula_service" "oneke":
│ 135: resource "opennebula_service" "oneke" {
│
│ service (ID: 3): context canceled
╵
~/_git/RESEARCH/tf+oneke$ tf destroy
data.http.appliances: Reading...
data.http.appliances: Read complete after 1s [id=https://marketplace.opennebula.io/appliance]
opennebula_image.oneke["4aa1f26fd6d835e78b827fc0961d2ccf"]: Refreshing state... [id=5]
opennebula_image.oneke["1d4589798b8a63a6afa7150492ca3193"]: Refreshing state... [id=3]
opennebula_image.oneke["d3044cba08d935b70d4f732c313362c9"]: Refreshing state... [id=4]
opennebula_template.oneke["master"]: Refreshing state... [id=7]
opennebula_template.oneke["vnf"]: Refreshing state... [id=5]
opennebula_template.oneke["worker"]: Refreshing state... [id=6]
opennebula_template.oneke["storage"]: Refreshing state... [id=4]
opennebula_service_template.oneke: Refreshing state... [id=2]
opennebula_service.oneke: Refreshing state... [id=3]
╷
│ Error: Plugin did not respond
│
│ with opennebula_service.oneke,
│ on main.tf line 135, in resource "opennebula_service" "oneke":
│ 135: resource "opennebula_service" "oneke" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
Stack trace from the terraform-provider-opennebula_v1.3.1 plugin:
panic: interface conversion: interface {} is string, not float64
goroutine 232 [running]:
github.com/OpenNebula/terraform-provider-opennebula/opennebula.resourceOpennebulaServiceRead({0xe86c00?, 0xc0000afe60?}, 0xc0000b2300, {0xc44800?, 0xc0007f32c0?})
github.com/OpenNebula/terraform-provider-opennebula/opennebula/resource_opennebula_service.go:317 +0xf25
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xc00042f340, {0xe86c00, 0xc0000afe60}, 0xd?, {0xc44800, 0xc0007f32c0})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:724 +0x12e
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc00042f340, {0xe86c00, 0xc0000afe60}, 0xc000392b60, {0xc44800, 0xc0007f32c0})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:1015 +0x585
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc0003ea420, {0xe86c00?, 0xc0000afaa0?}, 0xc0002553c0)
github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/grpc_provider.go:613 +0x497
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc000316aa0, {0xe86c00?, 0xc0000aef00?}, 0xc0004e37a0)
github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/tf5server/server.go:748 +0x49e
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xd08360?, 0xc000316aa0}, {0xe86c00, 0xc0000aef00}, 0xc00056bab0, 0x0)
github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:349 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0003141e0, {0xe89e00, 0xc0004fc000}, 0xc0003d0ea0, 0xc000423020, 0x1376690, 0x0)
google.golang.org/grpc@v1.50.1/server.go:1340 +0xd13
google.golang.org/grpc.(*Server).handleStream(0xc0003141e0, {0xe89e00, 0xc0004fc000}, 0xc0003d0ea0, 0x0)
google.golang.org/grpc@v1.50.1/server.go:1713 +0xa1b
google.golang.org/grpc.(*Server).serveStreams.func1.2()
google.golang.org/grpc@v1.50.1/server.go:965 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
google.golang.org/grpc@v1.50.1/server.go:963 +0x28a
Error: The terraform-provider-opennebula_v1.3.1 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.
opennebula_service.oneke: Still creating... [7m10s elapsed]
opennebula_service.oneke: Still creating... [7m20s elapsed]
opennebula_service.oneke: Still creating... [7m30s elapsed]
opennebula_service.oneke: Still creating... [7m40s elapsed]
opennebula_service.oneke: Still creating... [7m50s elapsed]
opennebula_service.oneke: Still creating... [8m0s elapsed]
╷
│ Error: Plugin did not respond
│
│ with opennebula_service.oneke,
│ on main.tf line 135, in resource "opennebula_service" "oneke":
│ 135: resource "opennebula_service" "oneke" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more
│ details.
╵
Stack trace from the terraform-provider-opennebula_v1.3.1 plugin:
panic: interface conversion: interface {} is string, not float64
goroutine 277 [running]:
github.com/OpenNebula/terraform-provider-opennebula/opennebula.resourceOpennebulaServiceRead({0xe86bc8?, 0xc00072b380?}, 0xc00036ef80, {0xc44800?, 0xc0002d7240?})
github.com/OpenNebula/terraform-provider-opennebula/opennebula/resource_opennebula_service.go:317 +0xf25
github.com/OpenNebula/terraform-provider-opennebula/opennebula.resourceOpennebulaServiceCreate({0xe86bc8, 0xc00072b380}, 0xc00036ef80, {0xc44800?, 0xc0002d7240})
github.com/OpenNebula/terraform-provider-opennebula/opennebula/resource_opennebula_service.go:258 +0x1126
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0xc000433340, {0xe86c00, 0xc00026acf0}, 0xd?, {0xc44800, 0xc0002d7240})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:707 +0x12e
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0xc000433340, {0xe86c00, 0xc00026acf0}, 0xc000455450, 0xc00036ed00, {0xc44800, 0xc0002d7240})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:837 +0xa7a
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0xc0003ea420, {0xe86c00?, 0xc00026abd0?}, 0xc000642140)
github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/grpc_provider.go:1021 +0xe3c
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0xc000316960, {0xe86c00?, 0xc00026a5a0?}, 0xc000222070)
github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/tf5server/server.go:818 +0x574
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0xd08360?, 0xc000316960}, {0xe86c00, 0xc00026a5a0}, 0xc000222000, 0x0)
github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:385 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0003141e0, {0xe89e00, 0xc000459380}, 0xc0003225a0, 0xc0004231d0, 0x13766c0, 0x0)
google.golang.org/grpc@v1.50.1/server.go:1340 +0xd13
google.golang.org/grpc.(*Server).handleStream(0xc0003141e0, {0xe89e00, 0xc000459380}, 0xc0003225a0, 0x0)
google.golang.org/grpc@v1.50.1/server.go:1713 +0xa1b
google.golang.org/grpc.(*Server).serveStreams.func1.2()
google.golang.org/grpc@v1.50.1/server.go:965 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
google.golang.org/grpc@v1.50.1/server.go:963 +0x28a
Error: The terraform-provider-opennebula_v1.3.1 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.
ID USER GROUP NAME STARTTIME STAT
7 oneadmin oneadmin Service OneKE 1.27 12/05 14:09:45 RUNNING
I used this updated tf code snippet:
terraform {
required_providers {
opennebula = {
source = "OpenNebula/opennebula"
version = "1.3.1"
}
}
}
variable "one" {
type = object({
endpoint = string
flow_endpoint = string
username = string
password = string
})
default = {
endpoint = "http://10.11.12.13:2633/RPC2"
flow_endpoint = "http://10.11.12.13:2474"
username = "oneadmin"
password = "asd123"
}
}
data "http" "appliances" {
url = "https://marketplace.opennebula.io/appliance"
request_headers = {
User-Agent = "OpenNebula 6.8.0"
Accept = "application/json"
}
}
locals {
appliances = {
for a in jsondecode(data.http.appliances.response_body).appliances : a.name => a
}
name = "Service OneKE 1.27"
service = local.appliances[local.name]
roles = {
for k, v in local.service.roles : k => merge(local.appliances[v], {
opennebula_template = jsondecode(local.appliances[v].opennebula_template)
})
}
md5_to_url = {
for f in distinct(flatten([
for r in values(local.roles) : concat(
try(r.files, []),
[for d in try(r.disks, []) : local.appliances[d].files]
)
])) : f.md5 => f.url
}
role_to_md5 = {
for k, v in local.roles : k => [
for f in flatten(concat(
try(v.files, []),
[for d in try(v.disks, []) : local.appliances[d].files]
)) : f.md5
]
}
}
provider "opennebula" {
endpoint = var.one.endpoint
flow_endpoint = var.one.flow_endpoint
username = var.one.username
password = var.one.password
}
resource "opennebula_image" "oneke" {
for_each = local.md5_to_url
name = "${local.name} ${each.key}"
datastore_id = 1
persistent = false
permissions = 642
dev_prefix = "vd"
driver = "qcow2"
path = each.value
}
resource "opennebula_template" "oneke" {
for_each = local.roles
name = "${local.name} ${each.key}"
cpu = try(each.value["opennebula_template"].CPU, null)
vcpu = try(each.value["opennebula_template"].VCPU, null)
memory = try(each.value["opennebula_template"].MEMORY, null)
context = try(each.value["opennebula_template"].CONTEXT, null)
dynamic "graphics" {
for_each = try([each.value["opennebula_template"].GRAPHICS], [])
content {
type = try(graphics.value.TYPE, null)
listen = try(graphics.value.LISTEN, null)
}
}
dynamic "os" {
for_each = try([each.value["opennebula_template"].OS], [])
content {
arch = try(os.value.ARCH, null)
boot = "disk0"
}
}
dynamic "disk" {
for_each = local.role_to_md5[each.key]
content {
image_id = opennebula_image.oneke[disk.value].id
}
}
}
resource "opennebula_service_template" "oneke" {
name = local.name
permissions = 642
uname = "oneadmin"
gname = "oneadmin"
template = jsonencode({ "TEMPLATE" = { "BODY" = merge(
jsondecode(local.service["opennebula_template"]),
{
"roles" : [
for r in jsondecode(local.service["opennebula_template"]).roles : merge(
r,
{ vm_template = tonumber(opennebula_template.oneke[r.name].id) }
)
]
}
) } })
}
resource "opennebula_service" "oneke" {
name = local.name
template_id = opennebula_service_template.oneke.id
extra_template = jsonencode({
networks_values = [
{ Public = { id = "0" } },
{ Private = { id = "1" } },
]
custom_attrs_values = {
ONEAPP_VROUTER_ETH0_VIP0 = "192.168.150.86"
ONEAPP_VROUTER_ETH1_VIP0 = "192.168.200.86"
ONEAPP_K8S_EXTRA_SANS = "localhost,127.0.0.1"
ONEAPP_K8S_MULTUS_ENABLED = "NO"
ONEAPP_K8S_MULTUS_CONFIG = ""
ONEAPP_K8S_CNI_PLUGIN = "cilium"
ONEAPP_K8S_CNI_CONFIG = ""
ONEAPP_K8S_CILIUM_RANGE = ""
ONEAPP_K8S_LONGHORN_ENABLED = "YES"
ONEAPP_STORAGE_DEVICE = "/dev/vdb"
ONEAPP_STORAGE_FILESYSTEM = "xfs"
ONEAPP_K8S_METALLB_ENABLED = "NO"
ONEAPP_K8S_METALLB_CONFIG = ""
ONEAPP_K8S_METALLB_RANGE = ""
ONEAPP_K8S_TRAEFIK_ENABLED = "YES"
ONEAPP_VNF_HAPROXY_INTERFACES = "eth0"
ONEAPP_VNF_HAPROXY_REFRESH_RATE = "30"
ONEAPP_VNF_HAPROXY_CONFIG = ""
ONEAPP_VNF_HAPROXY_LB2_PORT = "443"
ONEAPP_VNF_HAPROXY_LB3_PORT = "80"
ONEAPP_VNF_NAT4_ENABLED = "YES"
ONEAPP_VNF_NAT4_INTERFACES_OUT = "eth0"
ONEAPP_VNF_ROUTER4_ENABLED = "YES"
ONEAPP_VNF_ROUTER4_INTERFACES = "eth0,eth1"
ONEAPP_VNF_KEEPALIVED_VRID = "1"
}
})
timeouts {
create = "15m"
delete = "5m"
}
}
So the timeout itself seems to be a red herring after all, something more sinister is happening behind the scenes.. :scream:
@vickmp @sk4zuzu it's verbose but not necessarily a big deal, there's a small error in the code.
From the logs above:
panic: interface conversion: interface {} is string, not float64
There's also a stack trace to help you locate where it comes from.
@vickmp @treywelsh
I allowed myself to create a PR with fixes, as the issue is a blocker for me, sorry for that :). It fixes 2 bugs and makes all my problems above go away, I can deploy/destroy OneKE with that without any issues. Please feel free to improve on the PR if needed. :hugs:
Description
:warning: This one is related to https://github.com/OpenNebula/terraform-provider-opennebula/issues/467 and may be a direct consequence of it.. (not sure) :thinking:
Provider panics after
openenebula_service
times out whenterraform apply
is executed again + importing the service (RUNNING) causes similar error.Terraform and Provider version
Affected resources and data sources
opennebula_service
Terraform configuration
Expected behavior
The service can be imported and provider doesn't panic.
Actual behavior
Provider panics, the state file seems to be unusable. :thinking:
Steps to Reproduce
terraform apply
immediately for the second time -> provider panics.opennebula_service
from the state file manually and rerun apply there is no panic anymore.Stack trace from the terraform-provider-opennebula_v1.2.2 plugin:
panic: interface conversion: interface {} is string, not float64
goroutine 52 [running]: github.com/OpenNebula/terraform-provider-opennebula/opennebula.resourceOpennebulaServiceRead({0xe81f80?, 0xc00054e480?}, 0xc00020ba00, {0xc3f800?, 0xc00044d540?}) github.com/OpenNebula/terraform-provider-opennebula/opennebula/resource_opennebula_service.go:307 +0xf25 github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(Resource).read(0xc0003fe460, {0xe81f80, 0xc00054e480}, 0xd?, {0xc3f800, 0xc00044d540}) github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:724 +0x12e github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(Resource).RefreshWithoutUpgrade(0xc0003fe460, {0xe81f80, 0xc00054e480}, 0xc0004168f0, {0xc3f800, 0xc00044d540}) github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:1015 +0x585 github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(GRPCProviderServer).ReadResource(0xc000217770, {0xe81f80?, 0xc00054e360?}, 0xc00020f5c0) github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/grpc_provider.go:613 +0x497 github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(server).ReadResource(0xc0002dd220, {0xe81f80?, 0xc0001fd6b0?}, 0xc0001f4300) github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/tf5server/server.go:748 +0x49e github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xd03360?, 0xc0002dd220}, {0xe81f80, 0xc0001fd6b0}, 0xc00037e070, 0x0) github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:349 +0x170 google.golang.org/grpc.(Server).processUnaryRPC(0xc0002e41e0, {0xe85180, 0xc0001511e0}, 0xc000341560, 0xc00040e3c0, 0x136f630, 0x0) google.golang.org/grpc@v1.50.1/server.go:1340 +0xd13 google.golang.org/grpc.(Server).handleStream(0xc0002e41e0, {0xe85180, 0xc0001511e0}, 0xc000341560, 0x0) google.golang.org/grpc@v1.50.1/server.go:1713 +0xa1b google.golang.org/grpc.(Server).serveStreams.func1.2() google.golang.org/grpc@v1.50.1/server.go:965 +0x98 created by google.golang.org/grpc.(Server).serveStreams.func1 google.golang.org/grpc@v1.50.1/server.go:963 +0x28a
Error: The terraform-provider-opennebula_v1.2.2 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely helpful if you could report the crash with the plugin's maintainers so that it can be fixed. The output above should help diagnose the issue.
╷ │ Error: Request cancelled │ │ with opennebula_service.oneke, │ on main.tf line 135, in resource "opennebula_service" "oneke": │ 135: resource "opennebula_service" "oneke" { │ │ The plugin.(*GRPCProvider).ReadResource request was cancelled. ╵
Stack trace from the terraform-provider-opennebula_v1.2.2 plugin:
panic: interface conversion: interface {} is string, not float64
goroutine 219 [running]: github.com/OpenNebula/terraform-provider-opennebula/opennebula.resourceOpennebulaServiceRead({0xe81f80?, 0xc0007add70?}, 0xc0002ba880, {0xc3f800?, 0xc00053ae00?}) github.com/OpenNebula/terraform-provider-opennebula/opennebula/resource_opennebula_service.go:307 +0xf25 github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(Resource).read(0xc000452460, {0xe81f80, 0xc0007add70}, 0xd?, {0xc3f800, 0xc00053ae00}) github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:724 +0x12e github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(Resource).RefreshWithoutUpgrade(0xc000452460, {0xe81f80, 0xc0007add70}, 0xc000558c30, {0xc3f800, 0xc00053ae00}) github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:1015 +0x585 github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(GRPCProviderServer).ReadResource(0xc00000d788, {0xe81f80?, 0xc0007adb30?}, 0xc000398680) github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/grpc_provider.go:613 +0x497 github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(server).ReadResource(0xc00033d180, {0xe81f80?, 0xc0007ad560?}, 0xc00088e480) github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/tf5server/server.go:748 +0x49e github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xd03360?, 0xc00033d180}, {0xe81f80, 0xc0007ad560}, 0xc0003ea070, 0x0) github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:349 +0x170 google.golang.org/grpc.(Server).processUnaryRPC(0xc0000003c0, {0xe85180, 0xc00028e9c0}, 0xc00019ac60, 0xc000464390, 0x136f630, 0x0) google.golang.org/grpc@v1.50.1/server.go:1340 +0xd13 google.golang.org/grpc.(Server).handleStream(0xc0000003c0, {0xe85180, 0xc00028e9c0}, 0xc00019ac60, 0x0) google.golang.org/grpc@v1.50.1/server.go:1713 +0xa1b google.golang.org/grpc.(Server).serveStreams.func1.2() google.golang.org/grpc@v1.50.1/server.go:965 +0x98 created by google.golang.org/grpc.(Server).serveStreams.func1 google.golang.org/grpc@v1.50.1/server.go:963 +0x28a
Error: The terraform-provider-opennebula_v1.2.2 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely helpful if you could report the crash with the plugin's maintainers so that it can be fixed. The output above should help diagnose the issue.