cloudposse / atmos

👽 Terraform Orchestration Tool for DevOps. Keep environment configuration DRY with hierarchical imports of configurations, inheritance, and WAY more. Native support for Terraform and Helmfile.
https://atmos.tools
Apache License 2.0
751 stars 94 forks source link

Feature request: `atmos vendor clean` #701

Open mss opened 3 weeks ago

mss commented 3 weeks ago

Describe the Feature

We are using the vendor feature a lot and especially while developing it is sometimes necessary to purge a vendored directories.

In our case it boils down to a rm -rf components/terraform and we could wrap it in a custom command. But it is a dangerous command to script due to the required -f flag (required for the .git directory within .terraform/modules ) and thus it would be nice if there was a "safer" command to give to our novice colleagues.

It would also be useful if flags like --component and the like were supported as well.

Expected Behavior

An official, safe way to remove vendored stuff exists.

Use Case

A safe way to remove vendored directories for novice users.

Describe Ideal Solution

Calling atmos vendor clean will remove everything added by atmos vendor pull. If directories were vendored, they should be deleted recursively with everything included, including non-vendored files and directories.

Alternatives Considered

No response

Additional Context

No response

mss commented 3 weeks ago

Potential problem to consider: What about mixins which are vendored (as directories, not single files) into non-vendored directories? Maybe #488 would be required to make that safe.

For example: I have something like this vendor configuration and v2.0 and v2.1 are actual vendored directories whereas develop is a symlink to the code I am working on. Ie. I'd expect v2.0 and v2.1 to be deleted due to their other vendor config but only the context.tf (which comes from the mixin) be removed from develop.

    - component: develop-context-mixin-v1.0
      source: components/terraform/mixins/context/v1.0
      targets:
        - components/terraform/foo/develop
        - components/terraform/foo/v2.0
        - components/terraform/foo/v2.1
GabisCampana commented 3 weeks ago

@aknysh

osterman commented 3 weeks ago

Makes sense as a feature. We will be working more on vendoring to improve it in the coming months. Will take this under consideration.

osterman commented 1 week ago

Potential problem to consider: What about mixins which are vendored (as directories, not single files) into non-vendored directories? Maybe #488 would be required to make that safe.

That's really tricky, without a state file (e.g. like a vendor.yaml.lock). I think we can handle the simple case, but supporting the lock file would be a larger scope.

mss commented 1 week ago

Potential problem to consider: What about mixins which are vendored (as directories, not single files) into non-vendored directories? Maybe #488 would be required to make that safe.

That's really tricky, without a state file (e.g. like a vendor.yaml.lock). I think we can handle the simple case, but supporting the lock file would be a larger scope.

Yes, I also came to that conclusion: Without a lock file my expectations couldn't really be met. Feel free to close this feature request if out of scope.

A lock file would have other advantages though like potentially speeding up vendoring by not re-downloading stuff

osterman commented 1 week ago

One other idea is to ignore any file that is in git. So it will delete any file not committed in the destination folder. Would that work?

mss commented 1 week ago

Hmmm... not sure. Maybe for others, probably not in my use case since we have components/terraform in .gitignore so the vendored directories are technically not in git but for develoment I have symlinked another git repo into that directory structure which is in git but another git. I have the feeling that there are too many edge cases which might purge data unexpectedly and make people unhappy.

osterman commented 1 week ago

I'll create another task internally, to introduce a vendor.yaml.lock file