pulumi / pulumi

Pulumi - Infrastructure as Code in any programming language 🚀
https://www.pulumi.com
Apache License 2.0
21.68k stars 1.11k forks source link

Create a new command that is optimized for continuously reading infra from a cloud account into pulumi state #12587

Open EvanBoyle opened 1 year ago

EvanBoyle commented 1 year ago

Pulumi import is not idempotent, and is designed to be run a single time, generate code, and then have that resource managed by a pulumi program.

Increasingly, we have use cases where it would be useful to just read cloud resources into a state file without having a pulumi program that modifies them.

It would be really helpful to create a new command similar to import that does has the following behavior:

Frassle commented 1 year ago

I think we could do this as a small modification to "import", its just that we want to mark the imported resources as "external" not managed and skip code generation. Both of those could be done separately, e.g. "--external --skip-code-generation".

Resources marked external wouldn't error about being re-imported over (it would just treat it as a refresh).

lukehoban commented 1 year ago

Can be run multiple times on the same resource without failing (idempotent)

I was actually wrong about this - it appears that import of the same ID multiple times does work - so for at least the common case this is idempotent. I'm not precisely sure what if anything would lead to a diff/replace/failure on re-import. But at least superficially I don't think there is a problem here.

we want to mark the imported resources as "external"

Yes! I think this is exactly what we need, both to ensure these resources don't get deleted, but also to ensure they can be differentiated from Pulumi-managed resources in the state file, Service and Insights features. A stack that contains only "external=true" resources could be interpreted as a "virtual stack" by the Service.

it would just treat it as a refresh)

I wasn't quite sure on this - do "external=true" resources get refreshsed today? And in particular, if they are removed from the cloud provider, do they get removed from the state by a refresh? ( I think for this scenario we need the answer to be yes).

--skip-code-generation

We support this one today with --generate-code=false.