Closed yohanb closed 1 year ago
Hi @yohanb, Thank you for opening this!
Have you tried the is_backend_terraform_cloud = true
flag?
https://github.com/minamijoyo/tfmigrate/tree/v0.3.7#is_backend_terraform_cloud
Hi @minamijoyo! Yes I have and same result.
I’m aware that some folks use the tfmigrate with the Terraform Cloud. That is why the is_backend_terraform_cloud flag exists, which was added by the community. However, honestly, I don’t know how to integrate them because I’m not a TFC user :upside_down_face:
I suggest you provide more context on what you are trying and how to reproduce the error so that someone can identify the difference between you and others.
I'm also trying to use tool with TF cloud, but trying to run that locally.I guess that could work?. It passes successfully switch to local, plan phase and it fails at switching back to remote. I'm trying multistate migration, but I guess it is not matter here
2022/11/30 08:26:37 [DEBUG] [main] tfmigrate version: 0.3.8
2022/11/30 08:26:37 [DEBUG] [command] load configuration file: .tfmigrate.hcl
2022/11/30 08:26:37 [DEBUG] [command] config: &config.TfmigrateConfig{MigrationDir:"./tfmigrate", IsBackendTerraformCloud:true, History:(*history.Config)(nil)}
2022/11/30 08:26:37 [DEBUG] [command] option: &tfmigrate.MigratorOption{ExecPath:"", PlanOut:"", IsBackendTerraformCloud:false, BackendConfig:[]string(nil)}
2022/11/30 08:26:37 [INFO] [runner] load migration file: tfmigrate/migration.hcl
2022/11/30 08:26:37 [INFO] [migrator] start multi state migrator plan phase for apply
2022/11/30 08:26:37 [DEBUG] [executor@workspaceA]$ terraform version
2022/11/30 08:26:39 [INFO] [migrator@workspaceA] terraform version: 1.2.2
2022/11/30 08:26:39 [INFO] [migrator@workspaceA] initialize work dir
2022/11/30 08:26:39 [DEBUG] [executor@workspaceA]$ terraform init -input=false -no-color
2022/11/30 08:26:53 [DEBUG] [executor@workspaceA]$ terraform workspace show
2022/11/30 08:26:55 [DEBUG] [migrator@workspaceA] currentWorkspace = dev, workspace = dev
2022/11/30 08:26:55 [INFO] [migrator@workspaceA] get the current remote state
2022/11/30 08:26:55 [DEBUG] [executor@workspaceA]$ terraform state pull
2022/11/30 08:27:04 [INFO] [migrator@workspaceA] override backend to local
2022/11/30 08:27:04 [INFO] [executor@workspaceA] create an override file
2022/11/30 08:27:04 [INFO] [migrator@workspaceA] creating local workspace folder in: workspaceA/terraform.tfstate.d/dev
2022/11/30 08:27:04 [INFO] [executor@workspaceA] switch backend to local
2022/11/30 08:27:04 [DEBUG] [executor@workspaceA]$ terraform init -input=false -no-color -reconfigure
2022/11/30 08:27:12 [DEBUG] [executor@workspaceB]$ terraform version
2022/11/30 08:27:14 [INFO] [migrator@workspaceB] terraform version: 1.2.2
2022/11/30 08:27:14 [INFO] [migrator@workspaceB] initialize work dir
2022/11/30 08:27:14 [DEBUG] [executor@workspaceB]$ terraform init -input=false -no-color
2022/11/30 08:27:23 [DEBUG] [executor@workspaceB]$ terraform workspace show
2022/11/30 08:27:25 [DEBUG] [migrator@workspaceB] currentWorkspace = dev, workspace = dev
2022/11/30 08:27:25 [INFO] [migrator@workspaceB] get the current remote state
2022/11/30 08:27:25 [DEBUG] [executor@workspaceB]$ terraform state pull
2022/11/30 08:27:30 [INFO] [migrator@workspaceB] override backend to local
2022/11/30 08:27:30 [INFO] [executor@workspaceB] create an override file
2022/11/30 08:27:30 [INFO] [migrator@workspaceB] creating local workspace folder in: workspaceB/terraform.tfstate.d/dev
2022/11/30 08:27:30 [INFO] [executor@workspaceB] switch backend to local
2022/11/30 08:27:30 [DEBUG] [executor@workspaceB]$ terraform init -input=false -no-color -reconfigure
2022/11/30 08:27:36 [INFO] [migrator] compute new states (workspaceA => workspaceB)
2022/11/30 08:27:36 [DEBUG] [executor@workspaceA]$ terraform state mv -state=/var/folders/yh/7vgmx6qs6b33f6rs3xb0cy5c0000gn/T/tmp4017198946 -state-out=/var/folders/yh/7vgmx6qs6b33f6rs3xb0cy5c0000gn/T/tmp47193629 -backup=/dev/null module.audit_logs module.audit_logs
2022/11/30 08:27:41 [DEBUG] [executor@workspaceB]$ terraform state mv -state=/var/folders/yh/7vgmx6qs6b33f6rs3xb0cy5c0000gn/T/tmp4096028056 -state-out=/var/folders/yh/7vgmx6qs6b33f6rs3xb0cy5c0000gn/T/tmp2540095428 -backup=/dev/null module.audit_logs module.audit_logs
2022/11/30 08:27:43 [INFO] [migrator@workspaceA] check diffs
2022/11/30 08:27:43 [DEBUG] [executor@workspaceA]$ terraform plan -state=/var/folders/yh/7vgmx6qs6b33f6rs3xb0cy5c0000gn/T/tmp1484509652 -out=/var/folders/yh/7vgmx6qs6b33f6rs3xb0cy5c0000gn/T/tfplan3394747495 -input=false -no-color -detailed-exitcode
2022/11/30 08:30:08 [INFO] [migrator@workspaceB] check diffs
2022/11/30 08:30:08 [DEBUG] [executor@workspaceB]$ terraform plan -state=/var/folders/yh/7vgmx6qs6b33f6rs3xb0cy5c0000gn/T/tmp550612920 -out=/var/folders/yh/7vgmx6qs6b33f6rs3xb0cy5c0000gn/T/tfplan2407686944 -input=false -no-color -detailed-exitcode
2022/11/30 08:30:25 [INFO] [executor@workspaceB] remove the override file
2022/11/30 08:30:25 [INFO] [executor@workspaceB] remove the workspace state folder
2022/11/30 08:30:25 [INFO] [executor@workspaceB] switch back to remote
2022/11/30 08:30:25 [DEBUG] [executor@workspaceB]$ terraform init -input=false -no-color
2022/11/30 08:30:26 [DEBUG] [executor@workspaceB] failed to run command: (*exec.ExitError)(0xc000152040)(exit status 1)
2022/11/30 08:30:26 [ERROR] [executor@workspaceB] failed to switch back to remote: failed to run command (exited 1): terraform init -input=false -no-color
note missing --reconfigure
at line.
2022/11/30 08:30:25 [DEBUG] [executor@workspaceB]$ terraform init -input=false -no-color
at that point I need to run terraform init -reconfigure
manually to fix workspace.
So is it just a bug when switch local->remote is done?
Also, what could be helpful to save generated state somewhere (seems tmp files are removed after run). As workaround we can push such state manually, it will make migration much easier anyway than playing with import/rm command for each resource.
tfmigrate.hcl is simple as
tfmigrate {
migration_dir = "./tfmigrate"
is_backend_terraform_cloud = true
}
and migration.hcl
migration "multi_state" "mv_dir1_dir2" {
from_dir = "workspaceA"
from_workspace = "dev"
to_dir = "workspaceB"
to_workspace = "dev"
actions = [
"mv module.audit_logs module.audit_logs"
]
}
HI @yohanb, @mtamiola-priv,
Finally, I also had the opportunity to use Terraform Cloud and confirmed that tfmigrate works, at least for non-self-hosted Terraform Cloud. One thing to add other than is_backend_terraform_cloud = true is that I also needed to set the Terraform Cloud workspace name in a migration file.
I'm not sure exactly what environment the original problem occurred in, but I expect there is no additional configuration for self-hosted Terraform Cloud agents. This is because the terraform init and plan commands run in the same local environment as tfmigrate, not in the agent environment.
I will close this issue, but if you continue to be able to reproduce the problem, feel free to reopen it with a reproduction method so that we can investigate further. Thanks!
Hi! First off, thanks for creating this tool. I don't understand how Terraform can't have better support for state migration (
moved
blocks can only do so much). I have a use case where I would have liked to runtfmigrate
in the context of Terraform Cloud and it seems like the best way would be to add the functionality to a self-hosted Terraform Cloud Agent.Has anyone been able to use
tfmigrate
successfully in a tfc-agent? Getting stuck with this issue for the moment: