yangyang919
commented
2 years ago Update:
By reading source codes, we made it work finally. First port-forward Thanos StoreAPI, then run command:
go run ./cmd/obslytics export --input-config='{"type":"STOREAPI","endpoint":"127.0.0.1:10901","tls_config":{"insecure_skip_verify":true}}' --output-config='{"type":"parquet", "path":"metrics/up", "storage":{"type":"s3", "config": {"bucket": "xxx", "endpoint": "s3.us-east-1.amazonaws.com", "insecure": true, "access_key": "key", "secret_key": "secret"}}}' --match="up" --resolution=2h --min-time="2021-08-24T23:59:59Z" --max-time="2021-08-26T00:59:59Z" --debug
bwplotka
Hi, we are a bunch of data scientists. We are trying to leverage prometheus metrics data to do AIOPS. Metrics data are already stored in AWS S3 via Thanos sidecar approach. To make data consumable by Spark, need a tool to convert data. Looks like this tool can be helpful, but it's really hard to understand how to use this tool given neither of us have Go programming experiences...
For instance, we are testing below commands:
go run ./cmd/obslytics export --input-config='{"endpoint":"127.0.0.1:10901","tls_config":{"insecure_skip_verify":true}}' --match="container_cpu_load_average_10s" --resolution=2h --output-config-file=container_cpu_load_average_10s.parquet --debug --min-time="2021-08-24T20:00:00+00:00" --max-time="2021-08-25T20:00:00+00:00"The console gets error codes:
level=error ts=2021-08-25T09:32:26.912354Z caller=main.go:104 msg="running command failed" err="unsupported Reader type " exit status 1No idea why, something related to StoreAPI. We deployed Thanos sidecar and Querier, and then kubectl port-forward querier pod like this: kubectl port-forward thanos-addons-query-7fc47d8cff-tf4rg 10901:10902