bakdata / kpops

Deploy Kafka pipelines to Kubernetes
https://bakdata.github.io/kpops
MIT License
12 stars 1 forks source link

Enable transparent OS environment lookups from internal environment #368

Closed disrupted closed 11 months ago

disrupted commented 11 months ago

Fixes #317

mkcode92 commented 11 months ago

maybe a stupid question, but why is this the Environment class still necessary at all? It seems to be an enriched os.environ, but this enrichment is never used. So it should behave like using os.environ everywhere you now use ENV. I think originally it was used to handle differently cased variables but this mechanism is now seemingly obsolete...

disrupted commented 11 months ago

maybe a stupid question, but why is this the Environment class still necessary at all? It seems to be an enriched os.environ, but this enrichment is never used. So it should behave like using os.environ everywhere you now use ENV. I think originally it was used to handle differently cased variables but this mechanism is now seemingly obsolete...

we have a set of internal KPOps variables, such as the ${pipeline_name} variables, for substitution. We want to store them in a local environment without polluting the OS environment. Likewise, we want environment variables to also work without replicating them to our internal environment – as we did previously. The Environment class is a thin abstraction combining both environments for convenience. It wraps the OS environment in a read-only way. Internal (lowercase) variables are only stored in memory without exporting. We are no longer transforming the casing of variables, so the OS-specific workaround isn't necessary anymore.