ZoneSettingsOverride state prevents upgrade to 4.0

gleb-jiji commented 2 years ago

Hello!

Vote on this issue by adding a 👍 reaction
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already)

Issue details

Upgrading from 3.5 to 4.0 with an existing ZoneSettingsOverride resource breaks with an Unexpected type error.

Steps to reproduce

Have a resource, e.g.

ZoneSettingsOverride(
    some_name,
    zone_id=zone.id,
    settings=ZoneSettingsOverrideSettingsArgs(
        always_use_https="on",
        email_obfuscation="off",
        server_side_exclude="off",
    ),
)

Upgrade pulumi-cloudflare from 3.5 to 4.0 and run pulumi up.

Pulumi and package versions

``` $ pulumi version v3.18.1 $ pip list Package Version ------------------ --------- Arpeggio 1.10.2 attrs 21.2.0 awscli 1.22.11 beautifulsoup4 4.10.0 botocore 1.23.11 certifi 2021.10.8 charset-normalizer 2.0.7 cloudflare 2.8.15 colorama 0.4.3 dill 0.3.4 docutils 0.15.2 grpcio 1.42.0 idna 3.3 jmespath 0.10.0 jsonlines 2.0.0 parver 0.3.1 pip 21.0.1 protobuf 3.19.1 pulumi 3.18.1 pulumi-aws 4.29.0 pulumi-cloudflare 4.0.0 pulumi-hcloud 1.7.0 pyasn1 0.4.8 python-dateutil 2.8.2 PyYAML 5.4.1 requests 2.26.0 rsa 4.7.2 s3transfer 0.5.0 semver 2.13.0 setuptools 54.1.2 six 1.16.0 soupsieve 2.3.1 urllib3 1.26.7 wheel 0.36.2 ```

Expected: Seamless provider version upgrade. Actual:

    error: Program failed with an unhandled exception:
    error: Traceback (most recent call last):
      File "/home/user/opt/pulumi/pulumi-language-python-exec", line 107, in <module>
        loop.run_until_complete(coro)
      File "/usr/lib64/python3.9/asyncio/base_events.py", line 642, in run_until_complete
        return future.result()
      File "/home/user/.cache/pypoetry/virtualenvs/infra-PmXxtYjb-py3.9/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 110, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File "/home/user/.cache/pypoetry/virtualenvs/infra-PmXxtYjb-py3.9/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 45, in run_pulumi_func
        await wait_for_rpcs()
      File "/home/user/.cache/pypoetry/virtualenvs/infra-PmXxtYjb-py3.9/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 65, in wait_for_rpcs
        await RPC_MANAGER.rpcs.pop()
      File "/home/user/.cache/pypoetry/virtualenvs/infra-PmXxtYjb-py3.9/lib/python3.9/site-packages/pulumi/runtime/rpc_manager.py", line 65, in rpc_wrapper
        result = await rpc
      File "/home/user/.cache/pypoetry/virtualenvs/infra-PmXxtYjb-py3.9/lib/python3.9/site-packages/pulumi/output.py", line 94, in is_value_known
        return await is_known and not contains_unknowns(await future)
      File "/home/user/.cache/pypoetry/virtualenvs/infra-PmXxtYjb-py3.9/lib/python3.9/site-packages/pulumi/output.py", line 94, in is_value_known
        return await is_known and not contains_unknowns(await future)
      File "/home/user/.cache/pypoetry/virtualenvs/infra-PmXxtYjb-py3.9/lib/python3.9/site-packages/pulumi/output.py", line 94, in is_value_known
        return await is_known and not contains_unknowns(await future)
      [Previous line repeated 4 more times]
      File "/home/user/.cache/pypoetry/virtualenvs/infra-PmXxtYjb-py3.9/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 568, in do_register
        rpc.resolve_outputs(res, resolver.serialized_props, resp.object, deps, resolvers, transform_using_type_metadata)
      File "/home/user/.cache/pypoetry/virtualenvs/infra-PmXxtYjb-py3.9/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 850, in resolve_outputs
        translated_value = translate_output_properties(value, translate_to_pass, types.get(key),
      File "/home/user/.cache/pypoetry/virtualenvs/infra-PmXxtYjb-py3.9/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 733, in translate_output_properties
        raise AssertionError((f"Unexpected type; expected a value of type `{typ}`"
    AssertionError: Unexpected type; expected a value of type `typing.Sequence[pulumi_cloudflare.outputs.ZoneSettingsOverrideInitialSetting]` but got a value of type `<class 'dict'>` at resource `xx.yy.zz`, property `initial_settings`: {'tls12Only': 'off', 'opportunisticOnion': 'on', 'minTlsVersion': '1.0', 'tls13': 'on', 'websockets': 'on', 'hotlinkProtection': 'off', 'browserCheck': 'on', 'pseudoIpv4': 'off', 'serverSideExclude': 'off', 'ssl': 'strict', 'minify': {'js': 'off', 'html': 'off', 'css': 'off'}, 'mirage': 'off', 'alwaysUseHttps': 'on', 'browserCacheTtl': 14400.0, 'zeroRtt': 'off', 'universalSsl': '', 'securityLevel': 'medium', 'privacyPass': 'on', 'responseBuffering': 'off', 'securityHeader': {'enabled': True, 'includeSubdomains': True, 'preload': True, 'nosniff': False, 'maxAge': 31536000.0}, 'challengeTtl': 1800.0, 'prefetchPreload': 'off', 'automaticHttpsRewrites': 'off', 'cacheLevel': 'aggressive', 'trueClientIpHeader': 'off', 'ipv6': 'on', 'h2Prioritization': 'on', 'polish': 'off', 'http3': 'off', 'originErrorPagePassThru': 'off', 'developmentMode': 'off', 'http2': 'on', 'cnameFlattening': 'flatten_at_root', 'emailObfuscation': 'off', 'imageResizing': 'off', 'brotli': 'on', 'waf': 'off', 'webp': 'off', 'maxUpload': 100.0, 'opportunisticEncryption': 'on', 'mobileRedirect': {'stripUri': False, 'mobileSubdomain': '', 'status': 'off'}, 'sortQueryStringForCache': 'off', 'alwaysOnline': 'on', 'rocketLoader': 'off', 'ipGeolocation': 'on', 'tlsClientAuth': 'off'}
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

leezen commented 2 years ago

Are you able to work around this by performing a PULUMI_ENABLE_LEGACY_PLUGIN_SEARCH pulumi refresh? Depending on the underlying state issue, this should cause a read of the resource along with attempting to write it to the state using the newer version of the plugin (assuming you have the 4.0 resource plugin installed).

gleb-jiji commented 2 years ago

With version 4.0 of the python library, PULUMI_ENABLE_LEGACY_PLUGIN_SEARCH=true pulumi refresh produces a bunch of

* HTTP status 400: Could not route to /zones/dns_records/88f61edb99a9dc44932285f3cddbcdb3, perhaps your object identifier is invalid? (7003), No route for that URI (7000)

one for each Record I have. I have both 3.5 and 4.0 resource plugins in ~/.pulumi/plugins

Just a pulumi refresh succeeds but doesn't solve the issue.

leezen commented 2 years ago

Thanks for the update @gleb-jiji -- a plain refresh succeeding makes sense (since it'll use the 3.5 version of the plugin) whereas with that env var set, pulumi will attempt to use the latest version of the plugin. It looks like the identifiers may have even changed between versions somehow? Else, it's very odd that it cannot perform a read against those records. One other idea for how to move forward here would be to manually fix up the state via pulumi stack export and editing the resulting JSON to have initialSettings be an array and then importing the state via pulumi stack import

gleb-jiji commented 2 years ago

Since this is a virtual resource I might as well just delete it from the state. Should I create a more complete reproducible example?

leezen commented 2 years ago

Thanks for offering, but I don't think it's necessary. From https://github.com/cloudflare/terraform-provider-cloudflare/commit/49ba3a43633adbbc185f5f3881ecaa3c1e9b8fc2#diff-c4993bf7897ba11d3fd5d356a5c80d40d54ef3650adfe90a1180ae8e047f1858 we can see the change in schema to go from MaxItems: 1 to allowing the array, which will cause this type of breakage as there doesn't seem to be any corresponding migration for this.

mikhailshilkov commented 11 months ago

With the Cloudflare provider being in 5.x version now, I don't think we'll do anything about 3.x to 4.x migration anymore

pulumi / pulumi-cloudflare