Closed randytpierce closed 7 months ago
A couple questions:
Yes, I believed I had done that on the last PR but if I missed it we definitely should. randy
On Wed, Jan 24, 2024 at 10:03 AM Ian McGinnis @.***> wrote:
A couple questions:
- Should we go ahead and remove the promql binary from the repo? It's architecture specific so will break on Apple Silicon/ARM dev laptops. It may be enough to test for a promql binary and exit with a warning to go install it if it's not found.
- I don't think the container has promql installed on PATH so this will break the container. We aren't currently using the container as far as I know but it'd be nice to switch to.
— Reply to this email directly, view it on GitHub https://github.com/NOAA-GSL/VxIngest/pull/301#issuecomment-1908557080, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGDVQPRYDITY4GXHZBIUSVTYQE5FPAVCNFSM6AAAAABCFY6ORWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBYGU2TOMBYGA . You are receiving this because you were assigned.Message ID: @.***>
-- Randy Pierce
Gotcha. It looks like promql was removed earlier. Sorry I missed that.
Otherwise, the import container image doesn't have the promql
cli tool installed so we should update the docker/import/Dockerfile
to make sure it's installed there. It looks like we should be able to get prebuilt binaries from this project: https://github.com/nalbury/promql-cli/releases/tag/v0.3.0.
I could see installing it the same way we do the Couchbase tooling: https://github.com/NOAA-GSL/VxIngest/blob/38e7bb58bea4fca7a3f9213260c558eddf600896/docker/import/Dockerfile#L34-L36
Oh wait, I had included the linux x86 64 image in the repo because that is the container architecture. Shucks! It is just a binary, so how do we handle the install to the container? randy
On Wed, Jan 24, 2024 at 10:25 AM Ian McGinnis @.***> wrote:
Gotcha. It looks like promql was removed earlier. Sorry I missed that.
Otherwise, the import container image doesn't have the promql cli tool installed so we should update the docker/import/Dockerfile to make sure it's installed there. It looks like we should be able to get prebuilt binaries from this project: https://github.com/nalbury/promql-cli/releases/tag/v0.3.0.
I could see installing it the same way we do the Couchbase tooling: https://github.com/NOAA-GSL/VxIngest/blob/38e7bb58bea4fca7a3f9213260c558eddf600896/docker/import/Dockerfile#L34-L36
— Reply to this email directly, view it on GitHub https://github.com/NOAA-GSL/VxIngest/pull/301#issuecomment-1908595787, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGDVQPX26LRBPQRE6XYHZM3YQE7ZLAVCNFSM6AAAAABCFY6ORWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBYGU4TKNZYG4 . You are receiving this because you were assigned.Message ID: @.***>
-- Randy Pierce
I'd handle it like in the example I linked for Couchbase. It uses uname - m
to get the architecture - though I'd need to verify it gets amd64
and arm64
as needed. For promql, the URLs are like: https://github.com/nalbury/promql-cli/releases/download/v0.3.0/promql-v0.3.0-linux-amd64.tar.gz
. So we'd do:
ENV PROMQL_VERSION=v0.3.0
RUN curl -L https://github.com/nalbury/promql-cli/releases/download/${PROMQL_VERSION}/promql-${PROMQL_VERSION}-linux-$(uname -m).tar.gz | tar xz --directory /usr/local
I added the promql install to the Dockerfile, Good catch!
Package | Line Rate | Branch Rate | Health |
---|---|---|---|
vxingest | 33% | 42% | ❌ |
vxingest.builder_common | 29% | 17% | ❌ |
vxingest.ctc_to_cb | 12% | 1% | ❌ |
vxingest.grib2_to_cb | 13% | 1% | ❌ |
vxingest.netcdf_to_cb | 13% | 1% | ❌ |
vxingest.partial_sums_to_cb | 12% | 1% | ❌ |
vxingest.utilities | 30% | 32% | ❌ |
Summary | 16% (488 / 2989) | 6% (40 / 684) | ❌ |
This makes the scraper work with the new log file format. It also accounts for promql being an architecture specific binary that must be installed on the import platofrm (it is installed in the container image but not necessarily when running the import as a script). This exercise reinforces the need to migrate the metrics into the code somehow, although that won't be a trivial exercise.