search

outscale / terraform-provider-outscale

Mozilla Public License 2.0
28 stars 30 forks source link

Error on plan after migration v1.0.0 on link public IP #469

Closed sylvain987 closed 2 months ago

sylvain987 commented 2 months ago

Terraform Version

[svc_terraform@frparadm73 nginx]$ terraform version
Terraform v1.9.2
on linux_amd64
+ provider registry.terraform.io/hashicorp/local v2.4.0
+ provider registry.terraform.io/hashicorp/null v3.2.1
+ provider registry.terraform.io/hashicorp/outscale v1.0.0
+ provider registry.terraform.io/hashicorp/vault v3.22.0

Terraform Configuration Files

In attachment. I do not attach all files but the main. Main files of the module and main files of the configuration that create the VM.

Debug Output

You can check the whole output in debug mode here : https://gist.github.com/sylvain987/224c122b44cbe0cbcc69b9f7f1cad741

Crash Output

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform planned the following actions, but then encountered a problem:

  # module.VM["VM-PPROD-FRONT-PRE-AZa-NGINX-01"].outscale_nic.NIC-VM will be updated in-place
  ~ resource "outscale_nic" "NIC-VM" {
        id                     = "eni-dcc8df38"
      + link_public_ip         = (known after apply)
        # (14 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.VM["VM-PPROD-FRONT-PRE-AZb-NGINX-02"].outscale_nic.NIC-VM will be updated in-place
  ~ resource "outscale_nic" "NIC-VM" {
        id                     = "eni-9e3e7a9e"
      + link_public_ip         = (known after apply)
        # (14 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.
╷
│ Error: private_ips.0.link_public_ip: '': source data must be an array or slice, got map
│
│   with module.VM["VM-PPROD-FRONT-PRE-AZb-NGINX-04"].outscale_nic.NIC-VM,
│   on .terraform/modules/VM/main.tf line 6, in resource "outscale_nic" "NIC-VM":
│    6: resource "outscale_nic" "NIC-VM" {
│
╵
╷
│ Error: private_ips.0.link_public_ip: '': source data must be an array or slice, got map
│
│   with module.VM["VM-PPROD-FRONT-PRE-AZa-NGINX-03"].outscale_nic.NIC-VM,
│   on .terraform/modules/VM/main.tf line 6, in resource "outscale_nic" "NIC-VM":
│    6: resource "outscale_nic" "NIC-VM" {

Expected Behavior

After the migration and make a plan, no error.

Actual Behavior

After the migration and make a plan on a VM with public IP, error.

Steps to Reproduce

I make the migration procedure in "https://github.com/outscale/terraform-provider-outscale" on the chapter "Migration to v1"

Additional Context

Before to migrate on 1.0.0, if needed, we connected public IP through the tag osc.fcu.eip.auto-attach. All plans has even if no public IP. SInce the migration, if the VM has no public IP, no problem. But if the VM has a public IP connected, there is the error in the Crash Output.

References

conf-main.tf.txt conf-variables.tf.txt conf-vm.auto.tfvars.txt module-data.tf.txt module-locals.tf.txt module-main.tf.txt module-variables.tf.txt

outscale-toa commented 2 months ago

Hi @sylvain987, Thanks for reaching us, You need to make some sed in your terraform configuration/state. For your issue: 

sed -i '/"link_public_ip": {/, /},/d' terraform.tfstate
sed -i '/"link_nic": {/, /},/d' terraform.tfstate

Have a look on terraform-provider-outscale Readme for migration

sylvain987 commented 2 months ago

Hello,

This is the sed commands that I applied on all configurations and check on your migration guide : 

terraform fmt
sed -i '/"block_device_mappings_created": \[/, /\],/d' terraform.tfstate
sed -i 's/outscale_volumes_link/outscale_volume_link/g' terraform.tfstate
sed -i '/"link_public_ip": {/, /},/d' terraform.tfstate
sed -i '/"link_nic": {/, /},/d' terraform.tfstate
sed -i '/"flexible_gpu_id": "/, /",/d' terraform.tfstate
sed -i '/"accepter_net": {/, /},/d' terraform.tfstate
sed -i '/"source_net": {/, /},/d' terraform.tfstate
sed -i '/"state": {/, /},/d' terraform.tfstate 
sed -i 's/outscale_volumes_link/outscale_volume_link/g' *.tf
sed -i 's/flexible_gpu_id /flexible_gpu_ids /g' *.tf
sed -i '/flexible_gpu_ids /s/= /= \[/' *.tf
sed -i '/outscale_flexible_gpu\./s/$/ \]/' *.tf
terraform fmt

On all configurations, Terraform create a parameter call link_public_ip with a empty content on NIC resource. If I apply again the 2 sed commands and make a plan, I have the same result :

[svc_terraform@frparadm73 nginx]$ sed -i '/"link_public_ip": {/, /},/d' terraform.tfstate
[svc_terraform@frparadm73 nginx]$ sed -i '/"link_nic": {/, /},/d' terraform.tfstate
[svc_terraform@frparadm73 nginx]$ terraform plan -out=main.tfplan
data.vault_generic_secret.outscale_credentials: Reading...
module.VM["VM-PPROD-FRONT-PRE-AZb-NGINX-02"].data.vault_generic_secret.aap_credentials: Reading...
....
module.VM["VM-PPROD-FRONT-PRE-AZa-NGINX-03"].data.outscale_net.vpc: Read complete after 0s [id=vpc-8370cb49]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform planned the following actions, but then encountered a problem:

  # module.VM["VM-PPROD-FRONT-PRE-AZa-NGINX-01"].outscale_nic.NIC-VM will be updated in-place
  ~ resource "outscale_nic" "NIC-VM" {
        id                     = "eni-dcc8df38"
      + link_public_ip         = (known after apply)
        # (14 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.VM["VM-PPROD-FRONT-PRE-AZb-NGINX-02"].outscale_nic.NIC-VM will be updated in-place
  ~ resource "outscale_nic" "NIC-VM" {
        id                     = "eni-9e3e7a9e"
      + link_public_ip         = (known after apply)
        # (14 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.
╷
│ Error: private_ips.0.link_public_ip: '': source data must be an array or slice, got map
│
│   with module.VM["VM-PPROD-FRONT-PRE-AZa-NGINX-03"].outscale_nic.NIC-VM,
│   on .terraform/modules/VM/main.tf line 6, in resource "outscale_nic" "NIC-VM":
│    6: resource "outscale_nic" "NIC-VM" {
│
╵
╷
│ Error: private_ips.0.link_public_ip: '': source data must be an array or slice, got map
│
│   with module.VM["VM-PPROD-FRONT-PRE-AZb-NGINX-04"].outscale_nic.NIC-VM,
│   on .terraform/modules/VM/main.tf line 6, in resource "outscale_nic" "NIC-VM":
│    6: resource "outscale_nic" "NIC-VM" {
outscale-toa commented 2 months ago

Hi @sylvain987,

After sed commands can you please do terraform refresh before terraform plan -out=main.tfplan like : 

[svc_terraform@frparadm73 nginx]$ sed -i '/"link_public_ip": {/, /},/d' terraform.tfstate
[svc_terraform@frparadm73 nginx]$ sed -i '/"link_nic": {/, /},/d' terraform.tfstate
[svc_terraform@frparadm73 nginx]$ terraform refresh
[svc_terraform@frparadm73 nginx]$ terraform plan -out=main.tfplan

you will still get 

# module.VM["VM-PPROD-FRONT-PRE-AZa-NGINX-01"].outscale_nic.NIC-VM will be updated in-place
  ~ resource "outscale_nic" "NIC-VM" {
        id                     = "eni-dcc8df38"
      + link_public_ip         = (known after apply)
        # (14 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.VM["VM-PPROD-FRONT-PRE-AZb-NGINX-02"].outscale_nic.NIC-VM will be updated in-place
  ~ resource "outscale_nic" "NIC-VM" {
        id                     = "eni-9e3e7a9e"
      + link_public_ip         = (known after apply)
        # (14 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

But it will be OK after terraform apply without change your resources

In the state of terraform-plugin under v1.0.0-rc.1 we have 

....
  link_nic   = {                                                                                                                                                                                                                                                
        "delete_on_vm_deletion" = "false"                                                                                                                                                                                                                                     
        "device_number"         = "0"                                                                                                  
        "link_nic_id"           = null                                                                                                 
        "state"                 = null                            
        "vm_account_id"         = null                             
        "vm_id"                 = null                                                                                                                                                                                                                                        
    }                                         
    link_public_ip   = {
            "link_public_ip_id"    = null
            "public_dns_name"      = null
            "public_ip"            = null
            "public_ip_account_id" = null
            "public_ip_id"         = null
   }
.....

In v1.0.0-rc.1 We have

....
       link_nic          = []
       link_public_ip    = []
....

Best Regards,

sylvain987 commented 2 months ago

Hi,

I make a terraform refresh before terraform plan but I have the same error :

│ Error: private_ips.0.link_public_ip: '': source data must be an array or slice, got map
│
│   with module.VM["VM-PPROD-FRONT-PRE-AZa-NGINX-03"].outscale_nic.NIC-VM,
│   on .terraform/modules/VM/main.tf line 6, in resource "outscale_nic" "NIC-VM":
│    6: resource "outscale_nic" "NIC-VM" {
│
╵
╷
│ Error: private_ips.0.link_public_ip: '': source data must be an array or slice, got map
│
│   with module.VM["VM-PPROD-FRONT-PRE-AZb-NGINX-04"].outscale_nic.NIC-VM,
│   on .terraform/modules/VM/main.tf line 6, in resource "outscale_nic" "NIC-VM":
│    6: resource "outscale_nic" "NIC-VM" {
│

By default, the state is on Gitlab. So my migration procedure is a little bit different : 

terraform init -plugin-dir=/opt/terraform/plugins -upgrade
terraform state pull > terraform.tfstate
terraform fmt
sed -i '/"block_device_mappings_created": \[/, /\],/d' terraform.tfstate
sed -i 's/outscale_volumes_link/outscale_volume_link/g' terraform.tfstate
sed -i '/"link_public_ip": {/, /},/d' terraform.tfstate
sed -i '/"link_nic": {/, /},/d' terraform.tfstate
sed -i '/"flexible_gpu_id": "/, /",/d' terraform.tfstate
sed -i '/"accepter_net": {/, /},/d' terraform.tfstate
sed -i '/"source_net": {/, /},/d' terraform.tfstate
sed -i '/"state": {/, /},/d' terraform.tfstate 
sed -i 's/outscale_volumes_link/outscale_volume_link/g' *.tf
sed -i 's/flexible_gpu_id /flexible_gpu_ids /g' *.tf
sed -i '/flexible_gpu_ids /s/= /= \[/' *.tf
sed -i '/outscale_flexible_gpu\./s/$/ \]/' *.tf
terraform fmt
old_serial=$(grep serial terraform.tfstate | awk '{print $2}' | sed 's/.$//')
new_serial=$(echo $(($(grep serial terraform.tfstate | awk '{print $2}' | sed 's/.$//')+1)))
sed -i "s/  \"serial\": ${old_serial},/  \"serial\": ${new_serial},/g" terraform.tfstate
terraform state push terraform.tfstate
terraform refresh
terraform plan -out=main.tfplan

I can restore a old version of the state before the sed commands.

Regards.

outscale-rce commented 2 months ago

@sylvain987 With your migration procedure, is the refresh/plan working correctly now ?

sylvain987 commented 2 months ago

Hi,

No. I have the same error.

Regards.

outscale-rce commented 2 months ago

Ok, thank you. We are not able to reproduce the issue. Do you have the possibility to test your configuration without using GitLab, just to be sure it is not the cause?

sylvain987 commented 2 months ago

Hello,

I tested without Gitlab (on a local state file). I have the same error (to make the test without Gitlab, I create a POC VM) :

Planning failed. Terraform encountered an error while generating this plan.

╷ │ Error: private_ips.0.link_public_ip: '': source data must be an array or slice, got map │ │ with module.VM["VM-ADMIN-POC-AZa-DEMO-01"].outscale_nic.NIC-VM, │ on .terraform/modules/VM/main.tf line 6, in resource "outscale_nic" "NIC-VM": │ 6: resource "outscale_nic" "NIC-VM" { │

Do you want to check the state file without the secrets of course to continue the debug ?

Regards.

outscale-rce commented 2 months ago

Yes, state file will be useful. Thank you for your patience.

sylvain987 commented 2 months ago

Hello,

This is the state : terraform.tfstate.txt

I add again the error after the plan : 

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform planned the following actions, but then encountered a problem:

  # module.VM["VM-PPROD-FRONT-PRE-AZa-NGINX-01"].outscale_nic.NIC-VM will be updated in-place
  ~ resource "outscale_nic" "NIC-VM" {
        id                     = "eni-dcc8df38"
      + link_public_ip         = (known after apply)
        # (14 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.VM["VM-PPROD-FRONT-PRE-AZb-NGINX-02"].outscale_nic.NIC-VM will be updated in-place
  ~ resource "outscale_nic" "NIC-VM" {
        id                     = "eni-9e3e7a9e"
      + link_public_ip         = (known after apply)
        # (14 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.
╷
│ Error: private_ips.0.link_public_ip: '': source data must be an array or slice, got map
│
│   with module.VM["VM-PPROD-FRONT-PRE-AZa-NGINX-03"].outscale_nic.NIC-VM,
│   on .terraform/modules/VM/main.tf line 6, in resource "outscale_nic" "NIC-VM":
│    6: resource "outscale_nic" "NIC-VM" {
│
╵
╷
│ Error: private_ips.0.link_public_ip: '': source data must be an array or slice, got map
│
│   with module.VM["VM-PPROD-FRONT-PRE-AZb-NGINX-04"].outscale_nic.NIC-VM,
│   on .terraform/modules/VM/main.tf line 6, in resource "outscale_nic" "NIC-VM":
│    6: resource "outscale_nic" "NIC-VM" {

Regards.

outscale-toa commented 2 months ago

Hi @sylvain987,

We found the error: private_ips.0.link_public_ip : ' ' : source data must be an array or slice, got map And we made a PR to fix it.

Thanks a lot.

Best Regards,

sylvain987 commented 2 months ago

Hello, Perfect. Do you know when the new version will be release ? Regards.

outscale-rce commented 2 months ago

Hello @sylvain987 , A new version with some bug fixes should be available by the end of this week.

sylvain987 commented 2 months ago

Nice, thank you.