prometheus / compliance

A set of tests to check compliance with various Prometheus interfaces
Apache License 2.0
127 stars 44 forks source link

Allow running tests against specific opentelemetry collector binary #97

Open rapphil opened 1 year ago

rapphil commented 1 year ago

According to the documentation, if we want to test a development version of the target of the remote_write_sender tests, we have to copy the binary in the bin directory.

Right now most of the targets of tests are hard coded to a specific version and the binary that will be used in the tests will use the name of the artifact published and copy the content in the bin directory. However this is not stable as the filename will usually contain the version e.g.: otelcol_0.42.0_linux_amd64

I'm proposing that we use the filename in archive instead of deriving the filename from the published artifact, since this will be more stable.

Alternative approach:

Allow to use different binaries for tests based on environment variables. We could stablish a convention for environment variables that could be used by all targets.