Open dnephin opened 3 years ago
I like this idea and have added some feedback in this comment.
It would be nice if the work on each of the cloud providers done in the go-discover package could be retained by releasing each of them as individual binaries that would be loaded via the mechanism proposed in this issue.
That's an excellent point. I believe go-discover
already provides a binary, so as long as this mechanism allows us to add arguments it should be easy to keep using go-discover
as a binary instead of a library. Currently the binary uses spaces to separate addresses in the output. Either we'll need to change that to newlines, or this new feature should accept any whitespace separated addresses.
What about a simple HTTP request which needs to return eg. a json array with IP-addresses to join? Everyone can implement a simple webserver which serves a static config, forwards the request to Proxmox, vSphere or whatever.
Feature Description
Currently the cloud auto-join feature allows a user to query a cloud provider API using go-discover to discover nodes that are part of a Consul cluster.
This feature works well for some use cases, but has a couple limitations:
go-discovery
. Supporting additional providers requires thatgo-discovery
be updated.go-discovery
are limited to a single tag, there's no way to do a more sophisticated query with multiple tags or any other properties that may be unique to a particular environment.In addition to the
go-discovery
provider, we could allow the discovery of addresses by any external command. Using a value ofexec=/usr/local/bin/query-ec2
would run the script at that path. The script would exit zero and return one server per line on success, or exit non-zero and print an error message of up to 1024 bytes to stderr on error.The script could use the cloud provider CLI, or any other command, to perform the query. Since these operations are already performing the API requests, the overhead of starting a new process shouldn't be too significant.
Use Case(s)
This approach could potentially solve a number of use cases:
go-discover
.Background
go-discover
also accounts for about 30 of the 107Mb used by the Consul binary. It also the library that brings in the most number of dependencies, which can make upgrading to new major versions of shared dependencies challenging, and also causes a lot of false positives on security scans. None of these issues by themselves are all that significant, but together they do add up. An approach that removes the need to include every provider in the binary would help mitigate these problems.Other related issues #9754