search

Particular / ServicePulse

Production monitoring for distributed systems.
https://docs.particular.net/servicepulse/
Other
33 stars 27 forks source link

Enable cloud configuration as a 1st class option for containers #1724

Closed dbelcham closed 2 days ago

dbelcham commented 7 months ago

Describe the feature.

This feature request is a copy of an feature request in the Service Control repository as it is the exact same functionality being requested for Service Pulse docker containers. The implementation might differ slightly as the configuration needs of both systems are different.

Is your feature related to a problem? Please describe.

Service Pulse has a docker container, but there is no information about how you go about configuring that instance in a way that is sustainable and persistent.

In a cloud first world, configuration of SP needs to allow for the use of cloud native configuration tooling such as Azure App Configuration. As security conscious cloud users, we need to be able to leverage things like Azure Key Vault as a storage location for sensitive information, like connection strings to our transports.

There are solutions to this in cloud native environments and we should be able to use them to manage all of the configuration of any container.

Describe the requested feature

  1. Have Service Pulse load configuration from a default appsettings.json file that is bundled in the container
  2. Subsequently, SP should load configuration values from other locations based on values passed into the container as environment variables when it is started. For example -e ENABLE_AZURE_APPCONFIG='true' -e AZURE_APPCONFIG_CONNECTIONSTRING='......'. This could be done for any number of different providers as long as the appropriate extensions for loading configurations are available inside the SP container.
  3. Any configuration that is available in the embedded appsettings.json file will be overridable if the same entry exists in the configuration provider passed in via the environment variables.
  4. We should be able to configure the Service Pulse instance to point at any instance of Service Control using this technique.
  5. We should be able to configure Heartbeats using this technique.
  6. In the Azure world, the 1st class choice would be Azure App Configuration. There are extensions to the .NET Configuration ecosystem to seamlessly integrate this capability during .NET application startup. Azure App Configuration already fronts Azure Key Vault (the 1st class choice for secret storage in Azure) so no direct connection to Key Vault would be needed.
  7. A second choice for Azure, and possibly other environments, would be Consul.

Describe alternatives you've considered

It's unclear from the current Service Pulse docker documentation what configuration options are available. I presume one or both of the follow can be made to work, but neither is a good option for the reasons listed.

Mounting volumes and modifying a configuration file in that volume.

I have used this technique on containers in the past and it adds a lot of overhead, and a rather large failure point, to the operations side of the container management story. This also creates a situation where we have to manage our configurations in both Azure App Configuration/Key Vault (for our endpoint containers) as well as config files that are pushed to containers. This creates a lot of build pipeline overhead, the possibility for diverging values, and a challenge to maintaining a configuration-as-code policy.

Modifying a configuration file and rebuilding the SC container to include the new file with the new settings.

This pushes container build and management onto the companies that are using SC which will require an intimate knowledge of how to properly do these tasks. This solution will also create a potential barrier for those companies to run on the latest versions of the codebase/tooling as they might not update their container pipelines as quickly as Particular does. As with the previous alternative, there are also concerns with this solution surrounding pipeline overhead, diverging values, and configuration-as-code.

Additional Context

No response

danielmarbach commented 7 months ago

Thanks @dbelcham for the great suggestions. I have added it to the backlog.

DavidBoike commented 2 days ago