search

yearn / yearn-exporter

Realtime and historical Yearn metrics
https://yearn.vision
MIT License
95 stars 146 forks source link

Consider replacing victoria with timescaledb #208

Closed banteg closed 1 year ago

banteg commented 2 years ago

as per @mariuspod idea, we could avoid duplicating the data and store everything in postgres, getting the best of both worlds

we could even keep the grafana dashboard code intact by using promscale, which would translate the promql queries

0xkofee commented 2 years ago

I'm working on making something work end to end for this as a POC https://github.com/yearn/yearn-exporter/pull/337

0xkofee commented 2 years ago

Summary

To get setup, we need timescale running with the promscale extension installed. The image I selected already has that. We also need promscale itself which is just a layer above timescale that provides the prometheus read/write apis. Think of promscale as a microservice that sits on top of timescaledb.

Other than that, the way we format the json to be POSTed to the /write apis was all I needed to change. Since promscale supports prometheus apis, grafana had no issues querying the data.

PR

https://github.com/yearn/yearn-exporter/pull/337

DarkGhost7 commented 1 year ago

Closing for now as the historical rebuild showed multiple issues with the promscale <-> timescale integration which deleted datapoints after insertion under certain conditions:

ERROR: duplicate key value violates unique constraint \"_hyper_627_195351_chunk_data_series_id_time_308

Also promscale has been officially discontinued: https://docs.timescale.com/promscale/latest/