Lift NatsOptions handling into discovery.EngineConfig 🍫 [minting]

[DavidS-ovm]

With MANAGED and LOCAL sources' configs drifting apart further, handling of ApiUrl, NATS_SERVERS, and other options needs to be fully centralised here, to be able to make a single decision and keep MANAGED considerations out of custom sources.

Take the entirety of https://github.com/overmindtech/stdlib-source/blob/a54fd03513e797a031324415a794a60c150817f1/cmd/root.go#L50-L126 and lift it into discovery.EngineConfigFromViper. This includes the following distinct bits:

• load apiUrl from OvermindInstance, if and only if this is a LOCAL source
• NEW: load apiUrl from net.JoinHostPort(os.Getenv("API_SERVER_SERVICE_HOST"), os.Getenv("API_SERVER_SERVICE_PORT")) IFF MANAGED
• create clients through CreateClients using above information
• lift existing --nats-jwt, --nats-nkey-seed, and --nats-connection-name flag handling into discovery
• select apiKey, jwtToken or natsKeys based on MANAGED status and above information for tokenClient, heartbeatOptions
• evaluate ALLOW_UNAUTHENTICATED for testing
• load nats_servers from OvermindInstance, IFF this is a LOCAL source
• NEW: load nats_servers from os.Getenv("NATS_SERVERS") IFF MANAGED (see https://github.com/overmindtech/srcman/blob/feb5020b160bb6bdbc31b9ae4b05e9759fabb9a6/controllers/source_resources.go#L381-L382)
• create natsOptions based on above information
• do the Got Config logging for all discovery-based stuff; leave sources to do their own logging for local config
• NEW: properly redact all JWT tokens and other secrets
• NEW: refactor discovery.NewEngine(ec) into ec.NewEngine()

Then all sources need to be upgraded to excise the existing code and replace it with a double whammy of discovery.EngineConfigFromViper(...).NewEngine(....).

• [ ] aws-source
• [ ] k8s-source
• [ ] stdlib-source
• [ ] Ensure that the READMEs on all sources are updated to reflect the new functionality. Maybe provide a common text fragment in discovery?

[tphoney]

from comparing the sources https://github.com/overmindtech/aws-source/blob/main/cmd/root.go https://github.com/overmindtech/harness-source/blob/main/cmd/root.go // copy of aws-source https://github.com/overmindtech/stdlib-source/blob/main/cmd/root.go https://github.com/overmindtech/k8s-source/blob/main/cmd/root.go is the outlier, doing everything manually

using https://github.com/overmindtech/discovery/blob/main/cmd.go https://github.com/overmindtech/discovery/blob/main/engine.go

first step ✅

we can pull the following cli options/env vars centrally into cmd.go in AddEngineFlags "nats-servers": natsServers, "nats-jwt": natsJWT, "nats-nkey-seed": natsNKeySeedLog, "nats-connection-name": natsConnectionName, and healthCheckPort := viper.GetInt("health-check-port")

second step

centralise var tokenClient auth.TokenClient

• creation of tokenclient for auth
• creation of heartbeatoptions
• setting of natsoptions

into discovery supplimenting EngineConfigFromViper

third step ✅

modify each of the sources, if we overide the sane defaults. eg k8s vs aws: nats ReconnectWait

fourth step ✅

update NewEngine in discovery to print out the variables used