search

Azure / aztfexport

A tool to bring existing Azure resources under Terraform's management
https://azure.github.io/aztfexport/
Mozilla Public License 2.0
1.63k stars 190 forks source link

Error "the provider detected that no object exists with the given id" when attempting to import any existing resource #540

Closed manwithaplandy closed 4 months ago

manwithaplandy commented 4 months ago

I am attempting to use this tool to import existing resources into Terraform from an Azure environment, but no matter what I do I am getting the following error message:

While attempting to import an existing object to "\<resource>", the provider detected that no object exists with the given id. Only pre-existing objects can be imported; check that the id is correct and that it is associated with the provider's configured region or endpoint, or use "terraform apply" to create a new remote object for this resource.

I have tried the following command: aztfexport res --use-azure-cli-cred --log-level TRACE --log-path log.txt --env usgovernment -s xxxxxxxxx /subscriptions/xxxxxxxxxxx/resourceGroups/xxxxxxxxx/providers/Microsoft.MachineLearningServices/workspaces/xxxxxxx but it gives me the above error every time.

I have also tried using the rg import command: aztfexport rg --log-level TRACE --log-path log.txt --env usgovernment xxxxxxxxxxxxx and it gives me the above error for every single resource in the resource group, including the resource group itself.

I have used the az cli to list the resources successfully.

All of these resources have public access disabled and are accessible via private endpoint on a vnet. My machine is on a network that can reach resources in the vnet via privatelink.

Below is the (sanitized) log from using the first command:

2024-07-16T16:00:57.628-0700 [INFO] aztfexport: aztfexport starts with config: config.Config{CommonConfig:config.CommonConfig{SubscriptionId:"xxxxxxxxxx", AzureSDKCredential:(azidentity.AzureCLICredential)(0xc006048080), AzureSDKClientOption:policy.ClientOptions{ClientOptions:policy.ClientOptions{APIVersion:"", Cloud:cloud.Configuration{ActiveDirectoryAuthorityHost:"https://login.microsoftonline.us/", Services:map[cloud.ServiceName]cloud.ServiceConfiguration{"resourceManager":cloud.ServiceConfiguration{Audience:"https://management.core.usgovcloudapi.net", Endpoint:"https://management.usgovcloudapi.net"}}}, Logging:policy.LogOptions{IncludeBody:true, AllowedHeaders:[]string(nil), AllowedQueryParams:[]string(nil)}, Retry:policy.RetryOptions{MaxRetries:0, TryTimeout:0, RetryDelay:0, MaxRetryDelay:0, StatusCodes:[]int(nil), ShouldRetry:(func(http.Response, error) bool)(nil)}, Telemetry:policy.TelemetryOptions{ApplicationID:"aztfexport(azurerm)", Disabled:false}, TracingProvider:tracing.Provider{newTracerFn:(func(string, string) tracing.Tracer)(nil)}, Transport:exported.Transporter(nil), PerCallPolicies:[]exported.Policy(nil), PerRetryPolicies:[]exported.Policy(nil)}, AuxiliaryTenants:[]string(nil), DisableRPRegistration:false}, OutputDir:"/home/xxxxxxxxxx/Scripts/Synapse_tf_export", OutputFileNames:config.OutputFileNames{TerraformFileName:"", ProviderFileName:"", MainFileName:"", ImportBlockFileName:""}, ProviderVersion:"", DevProvider:false, ProviderName:"azurerm", ContinueOnError:false, BackendType:"local", BackendConfig:[]string(nil), ProviderConfig:map[string]cty.Value(nil), FullConfig:false, Parallelism:10, ModulePath:"", HCLOnly:false, TFClient:tfclient.Client(nil), TelemetryClient:telemetry.AppInsightClient{TelemetryClient:(*appinsights.telemetryClient)(0xc006043a40), subscriptionId:"xxxxxxxxxx", installId:"xxxxxxxxxx", sessionId:"xxxxxxxxxx"}, GenerateImportBlock:false}, ResourceId:"/subscriptions/xxxxxxxxxx/resourceGroups/xxxxxxxxxx/providers/Microsoft.MachineLearningServices/workspaces/xxxxxxxxxx", ResourceGroupName:"", ARGPredicate:"", MappingFile:"", ResourceNamePattern:"", RecursiveQuery:false, TFResourceName:"res-0", TFResourceType:"", IncludeRoleAssignment:false, IncludeResourceGroup:false}

2024-07-16T16:00:57.629-0700 [INFO] aztfexport: New resource meta 2024-07-16T16:00:57.631-0700 [DEBUG] aztfexport: UI update. STATUS: initializing | MSG: tea.WindowSizeMsg{Width:207, Height:39} 2024-07-16T16:00:57.631-0700 [DEBUG] aztfexport: UI update. STATUS: initializing | MSG: &meta.MetaResource{baseMeta:meta.baseMeta{subscriptionId:"xxxxxxxxxx", azureSDKCred:(azidentity.AzureCLICredential)(0xc006048080), azureSDKClientOpt:policy.ClientOptions{ClientOptions:policy.ClientOptions{APIVersion:"", Cloud:cloud.Configuration{ActiveDirectoryAuthorityHost:"https://login.microsoftonline.us/", Services:map[cloud.ServiceName]cloud.ServiceConfiguration{"resourceManager":cloud.ServiceConfiguration{Audience:"https://management.core.usgovcloudapi.net", Endpoint:"https://management.usgovcloudapi.net"}}}, Logging:policy.LogOptions{IncludeBody:true, AllowedHeaders:[]string(nil), AllowedQueryParams:[]string(nil)}, Retry:policy.RetryOptions{MaxRetries:0, TryTimeout:0, RetryDelay:0, MaxRetryDelay:0, StatusCodes:[]int(nil), ShouldRetry:(func(http.Response, error) bool)(nil)}, Telemetry:policy.TelemetryOptions{ApplicationID:"aztfexport(azurerm)", Disabled:false}, TracingProvider:tracing.Provider{newTracerFn:(func(string, string) tracing.Tracer)(nil)}, Transport:exported.Transporter(nil), PerCallPolicies:[]exported.Policy(nil), PerRetryPolicies:[]exported.Policy(nil)}, AuxiliaryTenants:[]string(nil), DisableRPRegistration:false}, outdir:"/home/xxxxxxxxxx/Scripts/Synapse_tf_export", outputFileNames:config.OutputFileNames{TerraformFileName:"terraform.tf", ProviderFileName:"provider.tf", MainFileName:"main.tf", ImportBlockFileName:"import.tf"}, tf:(tfexec.Terraform)(nil), resourceClient:(armresources.Client)(0xc00047e078), providerVersion:"3.99.0", devProvider:false, providerName:"azurerm", backendType:"local", backendConfig:[]string(nil), providerConfig:map[string]cty.Value(nil), fullConfig:false, parallelism:10, generateImportFile:false, hclOnly:false, tfclient:tfclient.Client(nil), moduleAddr:"", moduleDir:"/home/xxxxxxxxxx/Scripts/Synapse_tf_export", importBaseDirs:[]string(nil), importModuleDirs:[]string(nil), importTFs:[]tfexec.Terraform(nil), originBaseState:[]uint8(nil), baseState:[]uint8(nil), tc:telemetry.AppInsightClient{TelemetryClient:(appinsights.telemetryClient)(0xc006043a40), subscriptionId:"xxxxxxxxxx", installId:"xxxxxxxxxx", sessionId:"xxxxxxxxxx"}}, AzureId:(armid.ScopedResourceId)(0xc001d00190), ResourceName:"res-0", ResourceType:""} 2024-07-16T16:00:57.635-0700 [INFO] aztfexport: Init Terraform 2024-07-16T16:00:57.812-0700 [INFO] aztfexport: Find terraform binary at /usr/bin/terraform 2024-07-16T16:00:57.812-0700 [INFO] aztfexport: Init provider 2024-07-16T16:00:57.812-0700 [INFO] aztfexport: Output directory doesn't contain provider setting, create one then 2024-07-16T16:00:57.813-0700 [INFO] aztfexport: Output directory doesn't contain terraform block, create one then 2024-07-16T16:00:57.813-0700 [DEBUG] aztfexport: Run "terraform init" for the output directory /home/xxxxxxxxxx/Scripts/Synapse_tf_export 2024-07-16T16:01:02.269-0700 [DEBUG] aztfexport: Run "terraform init" for the import directory /tmp/aztfexport-1144619604 2024-07-16T16:01:02.269-0700 [DEBUG] aztfexport: Run "terraform init" for the import directory /tmp/aztfexport-1811785705 2024-07-16T16:01:02.269-0700 [DEBUG] aztfexport: Run "terraform init" for the import directory /tmp/aztfexport-1404755313 2024-07-16T16:01:02.269-0700 [DEBUG] aztfexport: Run "terraform init" for the import directory /tmp/aztfexport-1906308737 2024-07-16T16:01:02.269-0700 [DEBUG] aztfexport: Run "terraform init" for the import directory /tmp/aztfexport-350614429 2024-07-16T16:01:02.269-0700 [DEBUG] aztfexport: Run "terraform init" for the import directory /tmp/aztfexport-1944491435 2024-07-16T16:01:02.269-0700 [DEBUG] aztfexport: Run "terraform init" for the import directory /tmp/aztfexport-3955052224 2024-07-16T16:01:02.270-0700 [DEBUG] aztfexport: Run "terraform init" for the import directory /tmp/aztfexport-9591396 2024-07-16T16:01:02.270-0700 [DEBUG] aztfexport: Run "terraform init" for the import directory /tmp/aztfexport-1121354430 2024-07-16T16:01:02.270-0700 [DEBUG] aztfexport: Run "terraform init" for the import directory /tmp/aztfexport-2830121908 2024-07-16T16:01:19.136-0700 [DEBUG] aztfexport: UI update. STATUS: initializing | MSG: aztfexportclient.InitProviderDoneMsg{} 2024-07-16T16:01:19.136-0700 [DEBUG] aztfexport: Azure Resource set map to TF resource set 2024-07-16T16:01:19.153-0700 [DEBUG] aztfexport: UI update. STATUS: listing Azure resources | MSG: aztfexportclient.ListResourceDoneMsg{List:meta.ImportList{meta.ImportItem{AzureResourceID:(armid.ScopedResourceId)(0xc001d83db0), TFResourceId:"/subscriptions/xxxxxxxxxx/resourceGroups/xxxxxxxxxx/providers/Microsoft.MachineLearningServices/workspaces/xxxxxxxxxx", ImportError:error(nil), Imported:false, ValidateError:error(nil), TFAddr:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, TFAddrCache:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, IsRecommended:false, Recommendations:[]string(nil), State:cty.NilVal}}} 2024-07-16T16:01:19.155-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: tea.WindowSizeMsg{Width:207, Height:39} 2024-07-16T16:01:57.491-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: tea.KeyMsg{Type:-1, Runes:[]int32{119}, Alt:false} 2024-07-16T16:01:57.493-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: aztfexportclient.StartImportMsg{List:meta.ImportList{meta.ImportItem{AzureResourceID:(*armid.ScopedResourceId)(0xc001d83db0), TFResourceId:"/subscriptions/xxxxxxxxxx/resourceGroups/xxxxxxxxxx/providers/Microsoft.MachineLearningServices/workspaces/xxxxxxxxxx", ImportError:error(nil), Imported:false, ValidateError:error(nil), TFAddr:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, TFAddrCache:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, IsRecommended:false, Recommendations:[]string(nil), State:cty.NilVal}}} 2024-07-16T16:01:57.493-0700 [DEBUG] aztfexport: UI update. STATUS: importing | MSG: tea.WindowSizeMsg{Width:207, Height:39} 2024-07-16T16:01:57.493-0700 [INFO] aztfexport: Importing /subscriptions/xxxxxxxxxx/resourceGroups/xxxxxxxxxx/providers/Microsoft.MachineLearningServices/workspaces/xxxxxxxxxx as azurerm_machine_learning_workspace.res-0 2024-07-16T16:02:01.566-0700 [ERROR] aztfexport: Importing azurerm_machine_learning_workspace.res-0: exit status 1

Error: Cannot import non-existent remote object

While attempting to import an existing object to "azurerm_machine_learning_workspace.res-0", the provider detected that no object exists with the given id. Only pre-existing objects can be imported; check that the id is correct and that it is associated with the provider's configured region or endpoint, or use "terraform apply" to create a new remote object for this resource. 2024-07-16T16:02:01.567-0700 [DEBUG] aztfexport: UI update. STATUS: importing | MSG: aztfexportclient.ImportItemsDoneMsg{Items:[]meta.ImportItem{meta.ImportItem{AzureResourceID:(armid.ScopedResourceId)(0xc001d83db0), TFResourceId:"/subscriptions/xxxxxxxxxx/resourceGroups/xxxxxxxxxx/providers/Microsoft.MachineLearningServices/workspaces/xxxxxxxxxx", ImportError:(fmt.wrapError)(0xc001d04400), Imported:false, ValidateError:error(nil), TFAddr:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, TFAddrCache:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, IsRecommended:false, Recommendations:[]string(nil), State:cty.NilVal}}} 2024-07-16T16:02:01.567-0700 [DEBUG] aztfexport: UI update. STATUS: importing | MSG: aztfexportclient.ImportDoneMsg{List:meta.ImportList{meta.ImportItem{AzureResourceID:(armid.ScopedResourceId)(0xc001d83db0), TFResourceId:"/subscriptions/xxxxxxxxxx/resourceGroups/xxxxxxxxxx/providers/Microsoft.MachineLearningServices/workspaces/xxxxxxxxxx", ImportError:(fmt.wrapError)(0xc001d04400), Imported:false, ValidateError:error(nil), TFAddr:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, TFAddrCache:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, IsRecommended:false, Recommendations:[]string(nil), State:cty.NilVal}}} 2024-07-16T16:02:01.568-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: tea.WindowSizeMsg{Width:207, Height:39} 2024-07-16T16:02:01.584-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: progress.FrameMsg{id:1, tag:1} 2024-07-16T16:02:03.731-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: tea.KeyMsg{Type:-1, Runes:[]int32{101}, Alt:false} 2024-07-16T16:02:03.732-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: aztfexportclient.ShowImportErrorMsg{Item:meta.ImportItem{AzureResourceID:(armid.ScopedResourceId)(0xc001d83db0), TFResourceId:"/subscriptions/xxxxxxxxxx/resourceGroups/xxxxxxxxxx/providers/Microsoft.MachineLearningServices/workspaces/xxxxxxxxxx", ImportError:(fmt.wrapError)(0xc001d04400), Imported:false, ValidateError:error(nil), TFAddr:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, TFAddrCache:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, IsRecommended:false, Recommendations:[]string(nil), State:cty.NilVal}, Index:0, List:meta.ImportList{meta.ImportItem{AzureResourceID:(armid.ScopedResourceId)(0xc001d83db0), TFResourceId:"/subscriptions/xxxxxxxxxx/resourceGroups/xxxxxxxxxx/providers/Microsoft.MachineLearningServices/workspaces/xxxxxxxxxx", ImportError:(fmt.wrapError)(0xc001d04400), Imported:false, ValidateError:error(nil), TFAddr:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, TFAddrCache:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, IsRecommended:false, Recommendations:[]string(nil), State:cty.NilVal}}} 2024-07-16T16:02:04.419-0700 [DEBUG] aztfexport: UI update. STATUS: import error message | MSG: tea.KeyMsg{Type:-1, Runes:[]int32{113}, Alt:false} 2024-07-16T16:02:04.420-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: tea.WindowSizeMsg{Width:207, Height:39} 2024-07-16T16:02:06.661-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: tea.KeyMsg{Type:-1, Runes:[]int32{101}, Alt:false} 2024-07-16T16:02:06.662-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: aztfexportclient.ShowImportErrorMsg{Item:meta.ImportItem{AzureResourceID:(armid.ScopedResourceId)(0xc001d83db0), TFResourceId:"/subscriptions/xxxxxxxxxx/resourceGroups/xxxxxxxxxx/providers/Microsoft.MachineLearningServices/workspaces/xxxxxxxxxx", ImportError:(fmt.wrapError)(0xc001d04400), Imported:false, ValidateError:error(nil), TFAddr:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, TFAddrCache:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, IsRecommended:false, Recommendations:[]string(nil), State:cty.NilVal}, Index:0, List:meta.ImportList{meta.ImportItem{AzureResourceID:(armid.ScopedResourceId)(0xc001d83db0), TFResourceId:"/subscriptions/xxxxxxxxxx/resourceGroups/xxxxxxxxxx/providers/Microsoft.MachineLearningServices/workspaces/xxxxxxxxxx", ImportError:(fmt.wrapError)(0xc001d04400), Imported:false, ValidateError:error(nil), TFAddr:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, TFAddrCache:tfaddr.TFAddr{Type:"azurerm_machine_learning_workspace", Name:"res-0"}, IsRecommended:false, Recommendations:[]string(nil), State:cty.NilVal}}} 2024-07-16T16:02:06.883-0700 [DEBUG] aztfexport: UI update. STATUS: import error message | MSG: tea.KeyMsg{Type:-1, Runes:[]int32{113}, Alt:false} 2024-07-16T16:02:06.884-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: tea.WindowSizeMsg{Width:207, Height:39} 2024-07-16T16:02:08.310-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: tea.KeyMsg{Type:-1, Runes:[]int32{113}, Alt:false} 2024-07-16T16:02:08.333-0700 [DEBUG] aztfexport: UI update. STATUS: building import list | MSG: aztfexportclient.QuitMsg{} 2024-07-16T16:02:08.334-0700 [INFO] aztfexport: aztfexport ends

magodo commented 4 months ago

@manwithaplandy Sorry to let you run into this issue. From the log, it seems the auth works fine for the aztfexport side, but something goes wrong at the provider side. Did you have environment variable exported (e.g. ARM_XXX) that is conflicts with your Azure CLI setting?

To better understand what's happening in the provider side, could you re-run your command, with the following env vars exported:

And share the log here.

manwithaplandy commented 4 months ago

I realize I forgot to include it in the original issue, but the environment is WSL Ubuntu 20.04 running on Windows 11.

To answer your questions, no, I did not have any environment variables exported that conflict with the Azure CLI setting. Azure CLI is configured and authenticated correctly, but I tried with and without the --use-azure-cli-cred flag to make sure that was use. The Azure CLI part seems to be working just fine - it is getting all of the resource information. It's the actual import part that is failing.

I agree it's on the terraform side, but I don't think it's the provider itself. When the .tf file is generated, the cloud environment isn't set, so terraform is still making API calls to the commercial endpoints and not the usgovernment endpoints.

The line environment = "usgovernment" would need to be added to the provider.tf to set it up to use the correct endpoints. I tried adding that line manually and setting aztfexport to just append the state file instead of replacing the files, but that didn't seem to resolve the issue - it still makes API calls to the commercial endpoints.

TF Log relevant section

magodo commented 4 months ago

@manwithaplandy Currently, these auth related settings only affect the aztfexport part, but not the provider. The provider is expected to be configured via its environment variables. So in your case, you shall set the env via ARM_ENVIRONMENT.

The reason we didn't pass through the option for aztfexport to the provider setting is that currently there are two providers supported: azapi and azurerm. Both settings are similar but have slightly nuances. Meanwhile, these settings are evolving in the long run. To ensure users have the most flexibility about the provider config, we suggest to use the env vars defined by each provider.

manwithaplandy commented 4 months ago

That worked! Thank you!

Maybe it's worth adding that to the README because I didn't see that documented anywhere. It's confusing that there is a flag to set your cloud environment as part of the aztfexport tool, but it doesn't actually work. Also, both the azapi and azurerm providers handle setting the cloud environment in the exact same way, either by setting the ARM_ENVIRONMENT variable or adding an environment block to the provider config, so I would think you could add the environment block to the provider config if the env flag is passed in.

Either way, I was able to use the tool successfully so I will mark this issue as closed.

stemaMSFT commented 4 months ago

@magodo I agree with @manwithaplandy here that we should update our docs/clarify if ARM_ENVIRONMENT can be used to change the environment as expected. Just a good issue to think about from a usability perspective.

manwithaplandy commented 4 months ago

@stemaMSFT to clarify, not that it CAN be used, but that it MUST be used for the tool to work properly in non-commercial Azure environments.

magodo commented 4 months ago

@manwithaplandy Can you please try the main branch and confirm it solved your issue? Then I'll make a new release for this.