sblair-metrostar
commented
1 year ago Thanks for starting this discussion, @iameskild. Really excited to get this extensions rewrite out the door. Ken and I will be spending some time this week testing all this in preparation for release, at least on the AWS side. Aside from any issues that come out of that, here are some of my thoughts from having worked with the plugins so far.
Make users explicitly determine the stage plugins they want to use instead of registering all plugins that are pip installed
Completely agree with this point. Auto-installing plugins based on what's found in the environment has revealed several logical problems already.
- Managing multiple environments with different plugin configurations gets messy and is really easy to screw up. Several times I've accidentally installed a plugin in an environment I didn't mean to.
- Possibly a subcommand to handle installing and uninstalling plugins. Can update the nebari-config.yaml, handle cleanup and uninstallation, etc...
Create Keycloak (auth) provider object (to create new Keycloak clients, roles, etc)
While I agree a reusable component that would serve to configure Keycloak clients and whatnot for downstream plugins would be valuable, my concern with creating a provider (presumably without Terraform) would just lead to reinventing the Terraform provider. Would like to maybe discuss what the failings of the Terraform approach are and brainstorm solutions to those before reinventing any wheels.
- Difficult to
nebari destroyKeycloak stages if the cluster is unavailable or the Keycloak API is inaccessible. Should just be able to skip stages like this anyway since there's no need to deconfigure Keycloak if I'm just destroying the environment anyway. - Maybe have
nebari rendercapable of generating shared Terraform modules that a Nebari Keycloak class can reference and be reused in plugin stages.
"Composition over Inheritance"
I think this one is going to be critical for breaking out of the Terraform mold, as the current inheritance based approach coupled with the limitations in managing subcomponents is naturally leading us to a single stage + Terraform lowest common denominator pattern. Being able to more easily split plugins into subcomponents would improve testability and maintainability, as well as expanding options for non-Terraform solutions where appropriate.
- Base stage class that defines properties/methods for composition of plugins.
- Add subcomponents to a list or something that the base class is responsible for orchestrating the render/deploy/destroy behaviors, shared configuration, output composition, whatever.
Confirm that the DNS auto-provision is working as expected
Suggest maybe moving toward something like External DNS as a more versatile solution. The Cloudflare auto-provision is helpful for my specific case but is overly specific to Cloudflare and can't realistically be expanded to cover other DNS providers without a ton of work.
Other issues/items I'd add to the conversation:
- [ ] Can't install a plugin more than once since 1) the package is detected in the environment and can only be
pip installonce, and 2) the name is hard coded in the plugin so multiple instances would conflict anyway. As an example, I'd like to be able to create a generic Helm chart plugin to replace the old helm extensions but lets me fix some of its limitations (namespacing, private repos, secrets, whatever). - [ ] A stage plugin package should be treated as a single component by Nebari. Currently, if my plugin exports multiple subcomponents (AWS Terraform stage, Keycloak, etc...) they are treated as separate stages by Nebari and have no inherent connection to each other besides the packaging. This relates to the composition/inheritance issue, so moving away from an inheritance model would be a big step.
- [ ] No clear way to uninstall a plugin aside from destroying the whole environment. This ties into the point about pivoting away from automagically registering plugins that are present in the environment toward a more deterministic configuration based solution. Will need a way to have the code for a plugin present in the system but not assumed to want it deployed, also probably a subcommand to manage uninstalling it. Should have something cleaner though so it can be managed with CI/CD pipelines.
I'll probably have more to add to this, just wanted to get some initial thoughts down.
dharhas
pavithraes
iameskild
"Composition over Inheritance"
NebariStage-->NebariTerraformStage-->KubernetesInfrastructureStage). This means there is a higher likelihood of changes that are made to base classes causing breaking changes for their inherited class.renderdeploydestroy