search

jrasell / sherpa

Sherpa is a highly available, fast, and flexible horizontal job scaling for HashiCorp Nomad. It is capable of running in a number of different modes to suit different requirements, and can scale based on Nomad resource metrics or external sources.
Mozilla Public License 2.0
163 stars 8 forks source link

Influxdb metrics provider #135

Open mylesw42 opened 4 years ago

mylesw42 commented 4 years ago

What type of PR is this? Add support for more external metrics providers

What this PR does / why we need it: This adds support for InfluxDB as a metric provider. This would be super helpful in adding Nomad autoscale support for the InfluxDB community.

Which issue(s) this PR fixes: Fixes #134

Special notes for your reviewer:

4 additional sherpa server flags have been added:

      --metric-provider-influxdb-addr string       The address of the InfluxDB server in the form <protocol>://<addr>:<port>
      --metric-provider-influxdb-insecure          Skip TLS validation of InfluxDB server certificate
      --metric-provider-influxdb-password string   InfluxDB password
      --metric-provider-influxdb-username string   InfluxDB username

I had some issues with go modules while attempting to use the packaged ./vendor files.

build github.com/jrasell/sherpa/cmd: cannot load github.com/hashicorp/consul/api: ambiguous import: found github.com/hashicorp/consul/api in multiple modules:
    github.com/hashicorp/consul v1.4.0 (/Users/myles.williams/go/pkg/mod/github.com/hashicorp/consul@v1.4.0/api)
    github.com/hashicorp/consul/api v1.3.0 (/Users/myles.williams/go/pkg/mod/github.com/hashicorp/consul/api@v1.3.0)

The only way for me to resolve it was to add the following line to go.mod:

replace github.com/hashicorp/consul v1.4.0 => github.com/hashicorp/consul v1.7.0

Once that was in place, I also updated the associated vendor packages and the build started working.

I have tested with the influxdb example policy in the guide against a non-prod environment, and watched the job scale up and down! Thanks for considering this pull request :)

Does this PR introduce a user-facing change?: NONE

Add InfluxDB external metrics provider
chuckyz commented 4 years ago

@mylesw42 @jrasell commenting on this as it bumps consul's local version, but should this project utilize github.com/hashicorp/consul/api instead of github.com/hashicorp/consul ?

mylesw42 commented 4 years ago

@mylesw42 @jrasell commenting on this as it bumps consul's local version, but should this project utilize github.com/hashicorp/consul/api instead of github.com/hashicorp/consul ?

Yeah, this one confuses me. I think I actually tracked it down. It appears the influxdata client library is still pulling in a reference to an archived influxdata/platform go mod that requires github.com/hashicorp/consul@v1.4.0. This release is prior to go modules being used by consul, which is why I kept seeing the ambiguous import issue. Using the replace line to pin it to v1.7.0 is kind of a workaround, even though it's not used.

Maybe instead of using the client library, I could update it to use the HTTP API, similar to how the Prometheus provider is already implemented.

mylesw42 commented 4 years ago

Well, I found an issue on my part with using the wrong import for the influxdb client! Anyway, this resolved the other import issue, and allowed me to cleanup the original PR. Vendored files are back to the previous release, with the influxdb client being the only addition. The tests and build passed on my machine prior to the push, so I'm not sure why TravisCI failed...