Failed to import Project via resource option "import_"

[andy-nix]

What happened?

I'm trying to do import MongoAtlas project. For this I have transformation that add import_ and ignore_changes on all fileds for resource

return pulumi.ResourceTransformationResult(
            props=args.props,
            opts=pulumi.ResourceOptions.merge(args.opts, pulumi.ResourceOptions(
                import_=resource_id,
                # We ignore absolutelly all inputs differences during update, to make it finished successfully
                # But once you comment out `imports` block in config - you will face drift
                ignore_changes = [prop for prop in args.props if prop not in []]
            )))

But when I run pulumi pre I receive error:

 error: mongodbatlas:index/project:Project resource 'mystack' has a problem: Missing required property 'orgId'
 error: Preview failed: one or more inputs failed to validate

And when I don't add 'org_id' to ingore_changes like defined below:

return pulumi.ResourceTransformationResult(
            props=args.props,
            opts=pulumi.ResourceOptions.merge(args.opts, pulumi.ResourceOptions(
                import_=resource_id,
                # We ignore absolutelly all inputs differences during update, to make it finished successfully
                # But once you comment out `imports` block in config - you will face drift
                ignore_changes = [prop for prop in args.props if prop not in ['org_id']]
            )))

then I receive drift and message that resource will fail to import:

            warning: inputs to import do not match the existing resource; importing this resource will fail
             = mongodbatlas:index/project:Project: (import)
                 [id=XXXXXXXXXXXXXXXX]
                 [urn=urn:pulumi:mystack::idp-core::hide:platform:mystack$hide:idp-module:MongoGlobal$mongodbatlas:index/project:Project::mystack]
                 [provider=urn:pulumi:mystack::idp-core::pulumi:providers:mongodbatlas::default_3_11_0::04da6b54-80e4-46f7-96ec-b56ff0331ba9]
               + orgId: "XXXXXXXXXXXXXXXX"

Note that I able to import other mongoatlas resources in such way, problem happen only with Project

Example

I have complex codebase, but in general to reproduce the error you need only following snippets:

class MongoGlobalResource(pulumi.ComponentResource):

def __init__(self, name: str, env: str, opts: pulumi.ResourceOptions = None):
        # Register our component resource with specific type and name. See https://www.pulumi.com/docs/concepts/resources/components/ for more details
        super().__init__("hide:idp-module:MongoGlobal", f"{name}-{env}", None, opts)
        base_name =  f"{name}-{env}"
        org_id = mongodbatlas.get_roles_org_id().org_id

        project = mongodbatlas.Project(base_name,
            name = base_name,
            org_id = org_id,
            is_collect_database_specifics_statistics_enabled = True,
            is_data_explorer_enabled = True,
            is_performance_advisor_enabled = True,
            is_realtime_performance_panel_enabled = True,
            is_schema_advisor_enabled = True,
            opts = pulumi.ResourceOptions(delete_before_replace=True)
        )

transformation that inject import_ option:

def import_transformation(args):
    # don't do transformation if imports block not specified in config
    config = pulumi.Config()
    imports = config.get_object("imports", {})
    if not imports:
      return pulumi.ResourceTransformationResult(props=args.props, opts=args.opts)

    # Get resources identificators required for import
    parent_name = args.opts.parent.__class__.__name__ if args.opts.parent else None
    parent_msg = f"in {args.opts.parent.__class__.__name__} Component resource" if args.opts.parent else "in the Stack root"
    resource_import_id = f"{parent_name}:{args.type_}:{args.name}" if parent_name else f'{args.type_}:{args.name}'

    aws_config = pulumi.Config("aws")
    env = aws_config.get_object("defaultTags", {}).get("tags", {}).get("environment")
    env = env if env else os.getenv('TARGET_ENV')
    if not env:
      pulumi.error(f"Can't import resources since can't determine current environment nor from `aws:defaultTags.tags.environment` Stack config option neither from `TARGET_ENV` environment variable")
      return pulumi.ResourceTransformationResult(props=args.props, opts=args.opts)

    if imports.get("_print_resources_id", False) and not isinstance(args.resource, pulumi.ComponentResource):
      # if imports._print_resources_id is set to True - Pulumi will print all IDs of resources registered in current stack, so they could be used
      # for specify import configuration. There is no way to determine is resource was already provisioned or not, so we print IDs for all resources
      pulumi.info(f"* Resource to import: {resource_import_id}")

    # if <resource_import_id> specified in `imports.<env>` we add to resource options to import cloud resource with id `imports.<env>.<resource_import_id>`
    # during `pulumi up` command
    env_imports = imports.get(env, {})
    resource_id = env_imports.get(resource_import_id, None)
    if not resource_id:
      return pulumi.ResourceTransformationResult(props=args.props, opts=args.opts)
    else:
      pulumi.warn(f"Importing {resource_id} as resource of type {args.type_} with logical name '{args.name}' {parent_msg}")
      return pulumi.ResourceTransformationResult(
            props=args.props,
            opts=pulumi.ResourceOptions.merge(args.opts, pulumi.ResourceOptions(
                import_=resource_id,
                # We ignore absolutelly all inputs differences during update, to make it finished successfully
                # But once you comment out `imports` block in config - you will face drift
                ignore_changes = [prop for prop in args.props if prop not in []] # or ["org_id"]]
            )))

pulumi.runtime.register_stack_transformation(import_transformation)

tansformation require following configuration in Stack config:

config:
  idp-core:imports:
    _print_resources_id: False
    sandbox:
      MongoGlobalResource:mongodbatlas:index/project:Project:mystack: XXXXXXXXXXXXXXX

Output of pulumi about

 CLI          
 Version      3.89.0
 Go Version   go1.21.1
 Go Compiler  gc

 Plugins
 NAME           VERSION
 aws            5.42.0
 mongodbatlas   3.11.0
 pulumiservice  0.10.1
 python         unknown
 random         4.14.0

 Host     
 OS       debian
 Version  12.2
 Arch     x86_64

 This project is written in python: executable='/platform/.venv/bin/python3' version='3.9.18'

 Backend        
 Name           cccf25686f4f
 URL            s3://hide?region=eu-central-1&awssdk=v2&profile=platform-core
 User           root
 Organizations  
 Token type     personal

 Dependencies:
 NAME                      VERSION
 idp-module-mongodb-atlas  1.0.0
 jsonpath-ng               1.6.0
 pip                       23.3.1
 pulumi-pulumiservice      0.10.1
 setuptools                68.2.2
 wheel                     0.41.2

 Pulumi locates its logs in /tmp by default
 warning: Failed to get information about the current stack: No current stack

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[andy-nix]

Code to reproduce could be simplified to:

import pulumi_mongodbatlas as mongodbatlas
org_id = mongodbatlas.get_roles_org_id().org_id

project = mongodbatlas.Project("myproject",
            name = "myproject",
            org_id = org_id,
            is_collect_database_specifics_statistics_enabled = True,
            is_data_explorer_enabled = True,
            is_performance_advisor_enabled = True,
            is_realtime_performance_panel_enabled = True,
            is_schema_advisor_enabled = True,
            opts = pulumi.ResourceOptions(
                delete_before_replace=True
                import_= "XXXXXXXXXXXXXX",
                ignore_changes = ['is_collect_database_specifics_statistics_enabled', 'is_data_explorer_enabled', 'is_extended_storage_sizes_enabled', 'is_performance_advisor_enabled', 'is_realtime_performance_panel_enabled', 'is_schema_advisor_enabled', 'limits', 'name', 'org_id', 'project_owner_id', 'region_usage_restrictions', 'teams', 'with_default_alerts_settings', 'cluster_count, 'created']
           )
        )

[mikhailshilkov]

Have you tried importing resources in a pulumi import command? Generally, it's less strict and would import the resource state even if the generated code is not perfect. You can then adjust the code and continue managing resources without the import option.

[andy-nix]

Not yet, in scope of developing internal developing platform I need ability to do import via code, so I was concentrated on this approach. But I believe I need to check import via CLI at least to compare behavior. I'll let you know a bit later results

[andy-nix]

I was able to import project via Pulumi CLI but got following warnings during this:

Diagnostics:
  mongodbatlas:index:Project (xxx-sandbox):
    warning: One or more imported inputs failed to validate. This is almost certainly a bug in the `mongodbatlas` provider. The import will still proceed, but you will need to edit the generated code after copying it into your program.
    warning: mongodbatlas:index/project:Project resource 'xxx-sandbox' has a problem: Missing required property 'orgId'

And in the end import command finished with error: error: anonymous.pp:3,10-16: undefined variable parent;

But seems it was able to successfully import resource in state since I can see Project resource in output of pulumi stack and have followin in state:

{
                "urn": "urn:pulumi:xxx-sandbox::idp-core::yyy:platform:xxx-sandbox$yyy:idp-module:MongoGlobal$mongodbatlas:index/project:Project::xxx-sandbox",
                "custom": true,
                "id": "62ccxxxxxxxxxxxxxxx43ef",
                "type": "mongodbatlas:index/project:Project",
                "outputs": {
                    "id": "62ccxxxxxxxxxxxxxxx43ef",
                    "limits": [],
                    "teams": []
                },
                "parent": "urn:pulumi:xxx-sandbox::idp-core::yyy:platform:xxx-sandbox$yyy:idp-module:MongoGlobal::xxx-sandbox",
                "protect": true,
                "provider": "urn:pulumi:xxx-sandbox::idp-core::pulumi:providers:mongodbatlas::default_3_11_1::3bb7ca96-fba1-44ad-a75e-b3234b376563",
                "created": "2023-11-01T16:31:53.754265335Z",
                "modified": "2023-11-01T16:31:53.754265335Z"
            }

So looks like there really some bug in Project resource implementation

[mjeffryes]

Unfortunately, it looks like this issue hasn't seen any updates in a while. If you're still encountering this problem, could you leave a quick comment to let us know so we can prioritize it? (Commenting will bump it back into our triage queue.)