search

stellar / go

Stellar's public monorepo of go code
https://stellar.org/developers
Apache License 2.0
1.3k stars 499 forks source link

Feature Request: stellar-archivist Generic Backend #1956

Open brahman81 opened 4 years ago

brahman81 commented 4 years ago

What problem does your feature solve?

Currently it is not easy to interact with history archives hosted on Azure, GCP blob stores. Most users choose to sync from an existing history archive to their local file system and then use Cloud specific tools to push to blob stores but even this does not allow for stellar-archivist scan or stellar-archivist status.

What would you like to see?

Ideally stellar-archivist is able to read a put and get command from a configuration file which would enable users to configure stellar-archivist read/write using external tools or shell scripts.

This would be inline with how stellar-core manages history.

What alternatives are there?

Currently there are 2 alternatives, the first is to sync stellar-core from the genesis ledger on a full validator (>15 days) and the second is to use stellar-archivist to sync from http://history.stellar.org/ to a local file system and then push from local file system to cloud based blob stores manually using provider specific tooling.

Both of these solutions are cumbersome and neither allow monitoring of history archives via the status and scan sub commands.

graydon commented 4 years ago

Oh, yeah, this is a good idea! I don't seem to have the requisite admin rights on this repo but .. could someone assign this to me?