tricorder-observability / Starship

Starship: next-generation Observability platform built with eBPF+WASM
GNU Affero General Public License v3.0
164 stars 24 forks source link

Consider multi-node Postgres #30

Open nascentcore-eng opened 1 year ago

nascentcore-eng commented 1 year ago

Right now there is a single instance of DB node. There is an issue of this single instance being unreliable, therefore losing data.

  1. One question is how long we continue to use single instance PromScale/TimescaleDB. 0.1 version will be using this Then prefer PG instance only server as a cluster-local buffering storage, i.e., the data is supposed to be fetch or pushed to cloud storage of SaaS, or user's own cloud storage backend.

  2. Will we move to multi-node PG for reliability Might be, but it could be solved by real-time streaming by API server from Postgres storage to Cloud storage.