Shared configurations between services

[ChadJessup]

What should we add or change to make your life better?

One of the biggest problems I have with microservices and aspnet core is updating configurations across a dozen+ services, often the exact same change needs to happen as well... I absolutely love the Microsoft.Extensions.Configuration way of doing things, and that has made me a 'configuration-first' dev, yay. But, now I have a ton of appsettings.*.json files...boo.

Common things I've needed to change for various reasons or set up the exact same way for multiple projects:

1. Create the same four or five appsettings.*.json files (appsettings.json, appsettings.{Environment}, appsettings.{USER}.json, appsettings.{machine}.json).
2. Setup the exact same configuration providers and order in each of these services' setup code, and somehow don't break auto-reload...
3. Logger changes
4. Health/Readiness/Metrics endpoints
5. Toggling FeatureFlags
6. Other service urls:ports
7. Downstream Service details (Kafka brokers, ES clusters, redis, etc - all different depending on local/dev/prod)

It'd be nice if tye helped out with this. I'd be more than willing to add a tye configuration provider to these services if I could consolidate most of these settings, setup, dedupe the files, and be able to push live configuration changes to all services at once.

Some features I think this could enable:

1. tye deploy could create a configmap in k8s and/or inject settings files into containers, or even a config volume.
2. Dashboard could show and modify configurations in running apps 'for free' to the services (something like https://github.com/NFig)
3. Extension method on IConfiguration could help address service discovery (like the useful .GetConnectionString() - #73 - e.g., .GetService("frontend") could return a configured HttpClient - (maybe chaos-monkey behavior as well... 🤔?)
4. Use same mechanisms with layered json files- appsettings.Development, appsettings.Production - to point services to remote/local services - #46
5. I believe it'd also touch/help with #19 (HealthChecks), #35 (Config injection), and #44 (Cluster Development).
6. Individual services could still have their own settings/overrides:

config.AddTyeConfiguration(optional: true, reloadOnChange: true)
.AddJsonFile($"appsettings.{env.EnvironmentName}.json",  optional: true, reloadOnChange: true)
.AddJsonFile("appsettings.{Service}.json", optional: true, reloadOnChange: true)
.AddEnvironmentVariables();

Why is this important to you?

1. I believe this is a common problem, especially in a corporate environment where external teams often own infrastructure or platform and requirements can change over time from external sources.
2. Microservices have an overhead to them as engineers often have to touch multiple projects in regards to configuration changes, with the overhead of code reviews, new builds.
3. Single place to configure a bunch of services, weeee. Fewer bugs or misconfigurations.
4. It'd be a great killer feature to tye (for me, at least)!

[rynowak]

Thanks for posting, this is really great feedback and will take a little time for us to digest.

We definitely plan to add libraries for the service discovery conventions, and to make the kind of thing done here trivial.

The log-levels thing is something we've thought about doing as well, it has a few moving parts in development and production. There's a 30% done version of it for k8s here - but I imagined this would work similar to how tye deals with connection strings - a file on disk backed by a configmap.

[ChadJessup]

@rynowak - I have a POC that is a TyeConfigurationProvider, with the tye CLI sending to services any config changes via signalr.

Is this something that I should put up for PR? Even if you all go a different direction it might be a good scaffold, or dev tool to help out while the other bits come in. I think it'd help with several open issues with a little bit more work...

To round it out, I was going to next add a configuration tab to the dashboard getting live updates, and create a sample where one can point multiple services to dev/prod Kafka brokers to show how it'd work (and to explore how tye would work with services that don't do http...)

[rynowak]

Is this something that I should put up for PR?

For reasons that I can't really express - I'm just a little bit unsure about jumping right to a config server. This becomes yet another thing to administer in production - or do you think the SignalR configuration aspect is only for dev?

Another path would be to do the same thing via files, which is likely what we'd want to do for k8s.

[ChadJessup]

@rynowak - Sorry, I could have been more clear. I think the live reloading of shared configs would be a dev/local thing. During Deploy/Publish, the configurations would automatically be handled as expected (configmaps/volumes, etc). So, files while in kubernetes production as you're expecting. The shared configs would be automatically placed with the services.

If Tye cli is running, the configuration could be shared and auto-reloaded to all services, does nothing if Tye isn't running (prod).

Did I explain that a bit better? Thanks