rquitales
commented
1 year ago @cristiciortea Apologies for this issue that you're running into. It appears that you may have gotten closer with a likely solution in the additional context that you have provided.
When importing a GCP storage bucket, and having it managed by Pulumi, you will need to just use pulumi_gcp.storage.Bucket, whereas the pulumi_gcp.storage.Bucket.get method is simply just a GET API request that does not import information into state.
Would you be able to provide the code you're using when you just did:
pulumi_gcp.storage.Bucket(..., opts=pulumi.ResourceOptions(..., import_=bucket_name), )In particular, it'd be helpful for me to see how you're importing your resources and attempting to update it using this approach.
I've attempted to reproduce what you're trying to attempt, and was able to successfully import and create the bucket without issues. Here is a minimal code for reference:
from pulumi import automation as auto
import pulumi_gcp
import os
import pulumi
def program_get() -> pulumi_gcp.storage.Bucket:
bucket_name = 'my-bucket' # this already exists in google cloud platform
buckets_location = 'us-central1'
# Import the existent bucket.
pulumi_gcp.storage.Bucket(
resource_name=bucket_name,
location=buckets_location,
name=bucket_name,
project=os.environ["GOOGLE_PROJECT"],
opts=pulumi.ResourceOptions(import_=bucket_name), # Note the import_ option here.
)
def program_create():
bucket_name = 'my-bucket' # this already exists in google cloud platform
buckets_location = 'us-central1'
pulumi_gcp.storage.Bucket(
resource_name=bucket_name,
location=buckets_location,
name=bucket_name,
force_destroy=True,
project=os.environ["GOOGLE_PROJECT"],
opts=pulumi.ResourceOptions(delete_before_replace=True),
)
auto.create_or_select_stack("test_stack", program=program_get, project_name="my-proj").up(on_output=print, diff=False)
auto.create_or_select_stack("test_stack", program=program_create, project_name="my-proj").up(on_output=print, diff=False)The error that is returned by the Google API in your example: Your previous request to create the named bucket succeeded and you already own it., conflict indicates that the import you're trying to do was not done correctly, and its state is not stored for Pulumi to use. As such, when you attempt to update the bucket, Pulumi will attempt to create the bucket from scratch, rather than attempting to update the existing pre-created bucket, hence resulting in the conflict error.
cristiciortea
mjeffryes
What happened?
I encountered an issue while trying to manage Google Cloud Storage buckets using Pulumi with the pulumi-gcp provider. My goal is to import an existing bucket, and then create a bucket with the same parameters (in my usecase, this happens when I import the bucket and then when I update my infrastructure). However, I'm facing two main problems:
After importing the existing bucket and attempting to create a new one with identical parameters, Pulumi still wants to replace the existing bucket. This suggests that there is a non-deterministic behavior in the way buckets are created and imported.
Furthermore the delete_before_replace=True option does not seem to function as expected. Pulumi attempts to replace the existing bucket but fails to delete it first, resulting in an error:
Error creating bucket nuc-test-bucket-0001: googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflictStack trace
Expected Behavior
The expected behavior for the given scenario should have been as follows:
Upon importing the existing bucket with the same parameters, Pulumi should recognize that the new bucket has the same configuration as the existing one and avoid any replacement. Essentially, it should detect no differences between the two configurations, treating them as identical.
If there was any difference between the configurations and Pulumi decided to replace the existing bucket, the delete_before_replace=True option should have instructed Pulumi to first delete the existing bucket and then create a new one with the updated configuration. This would have avoided the conflict error (Error 409) caused by attempting to create a bucket with the same name as an existing one.
Steps to reproduce
In linux:
mkdir -p /pulumi-work-dir-testing/workspace_testcd /pulumi-work-dir-testing/workspace_testecho "description: Test infrastracture\n name: test-project-bucket\n runtime: python" >> Pulumi.yamlpulumi login file:///pulumi-work-dir-testing/workspace_testpulumi stack init test_stackIn ipython interractive shell:
Output of
pulumi aboutIn [4]: !pulumi about CLI
Version 3.30.0 Go Version go1.17.9 Go Compiler gc
Plugins NAME VERSION gcp 6.53.0 kubernetes 3.24.2 python unknown
Host
OS debian Version 11.1 Arch x86_64
This project is written in python (/usr/local/bin/python3 v3.9.7)
Current Stack: test_stack
TYPE URN pulumi:pulumi:Stack urn:pulumi:test_stack::test-project-bucket::pulumi:pulumi:Stack::test-project-bucket-test_stack pulumi:providers:gcp urn:pulumi:test_stack::test-project-bucket::pulumi:providers:gcp::default_6_53_0 gcp:storage/bucket:Bucket urn:pulumi:test_stack::test-project-bucket::gcp:storage/bucket:Bucket::nuc-test-bucket-0001
Found no pending operations associated with test_stack
Backend
Name backend-54777f96d-z69pz URL file:///pulumi-work-dir-testing/workspace_test User root Organizations
NAME VERSION central-backend 0.1.0 ipython 8.12.0 pip 21.2.4 pre-commit 2.21.0 pydevd-pycharm 222.4554.11 pytest-aioresponses 0.2.0 pytest-dotenv 0.5.2 ruff 0.0.254 types-toml 0.10.8.6 wheel 0.37.0
Pulumi locates its logs in /tmp by default warning: A new version of Pulumi is available. To upgrade from version '3.30.0' to '3.64.0', visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.
Additional context
I found a workaround, if import the bucket with pulumi import instead of pulumi_gcp.storage.Bucket.get, this bug/s do not reproduce.
Additionally, if I attempt to import the bucket using the following approach, the issue still persists:
pulumi_gcp.storage.Bucket( resource_name=bucket_name, cors=[pulumi_gcp.storage.BucketCorArgs( max_age_seconds=3600, methods=[ "GET", "HEAD", "PUT", "POST", "DELETE", ], )], location=buckets_location, name=bucket_name, project=os.environ["GOOGLE_PROJECT"], storage_class=storage_class, uniform_bucket_level_access=True, opts=pulumi.ResourceOptions(delete_before_replace=True, id=bucket_name, import_=bucket_name), )Contributing
Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).