search

thanos-community / obslytics

Tools and Services allowing seamless usage of Observability data from Prometheus, Thanos, Cortex, M3DB, Loki and more!
Apache License 2.0
30 stars 8 forks source link

Refactored output and input into more concrete types.; Added S3 writer. #15

Closed bwplotka closed 3 years ago

bwplotka commented 3 years ago

cc @4n4nd

4n4nd commented 3 years ago

I was able to test using the remoteread api with the following command:

 go run ./cmd/obslytics export --input-config='{"type":"remoteread","endpoint":"http://demo.robustperception.io:9090/api/v1/read","tls_config":{"insecure_skip_verify":true}}'\
      --output-config='{"type":"parquet","path": "output/up.parquet","storage": {"type":"s3", "config": {"bucket": "DH-DEV-DATA-SCIENCE-TUTORIAL", "endpoint":  "s3.xxxxx.com", }}}' --match="up"\
      --resolution=1h --min-time="2020-09-19T23:59:59Z" --max-time="2020-09-20T00:59:59Z"\
      --debug

It works :+1: But I was not able to test the Thanos StoreAPI method, because I don't have access to a thanos instance.

bwplotka commented 3 years ago

Thanks @4n4nd for quick review :heart:

Hope @iNecas you don't mind if we merge so we can move forward.

iNecas commented 3 years ago

I sure don't mind: whatever it takes to move this project forward! :+1: