Create a CLI for uploading and downloading blocks

kushalShukla-web commented 1 month ago

This PR introduces a powerful CLI tool for seamless block uploading and downloading. The initial implementation leverages an object storage as the storage backend, with the potential for other storage backends in the future.

Added a dedicated upload function for uploading data to the object store, with error handling and status logs using log/slog.
A separate download function was created to fetch data from the object store. Structured logging is a key feature of this PR, made possible by the utilization of the log/slog package. This enables easier traceability and debugging during storage operations.
Insured clear separation of concerns by modularizing upload and download logic into distinct functions.
Integrated the Thanos object storage package to facilitate efficient data storage and retrieval handling.

Test plan

[ ] Manual testing locally against a Minio service.

kushalShukla-web commented 1 month ago

I see 'upload'; will you be downloading data at some point?

yes @bboreham I have to implement both upload and download functionality.

kakkoyun commented 1 month ago

@kushalShukla-web Also, we should work on the PR title and the description.

Please also sign your commits https://github.com/prometheus/test-infra/pull/760/checks?check_run_id=30633470860

bwplotka commented 2 weeks ago

Is this something that would be useful in promtool? 🤔 https://github.com/prometheus/prometheus/tree/main/cmd/promtool

bwplotka commented 2 weeks ago

Ah this is for old data in prombench, cool.

prometheus / test-infra

Create a CLI for uploading and downloading blocks #760

Test plan