orchestracities / ngsi-timeseries-api

QuantumLeap: a FIWARE Generic Enabler to support the usage of NGSIv2 (and NGSI-LD experimentally) data in time-series databases
https://quantumleap.rtfd.io/
MIT License
37 stars 49 forks source link

Ask about Databases #733

Closed tzzed closed 11 months ago

tzzed commented 11 months ago

I would like to know why do you provide 2 db as backend CrateDB and Timescale? As Timescale and CrateDB are supported as a backend db, do you have any benchmark crateDB vs Timescale ?

c0c0n3 commented 11 months ago

Hello @tzzed :-)

why do you provide 2 db as backend CrateDB and Timescale?

Why not? :-) Supporting multiple backends was a design goal since QL's inception. The rationale behind the decision to support multiple time series DBs was (I think, I wasn't working on QL at that time) that different DBs usually come with different implementation trade-offs which make each more suitable than the others for specific circumstances. For example, we've found that in one of our prod deployments where we have resource constrained machines, Timescale performed better than Crate. On the other hand, we've found that Crate is much easier to scale, well provided you've got beefy hardware.

do you have any benchmark crateDB vs Timescale ?

We benchmarked both DBs in the past, but we never published any comparison since we believe numbers taken out of context can be misleading. There are benchmark results out there (ask Google) but I've found that at the end of the day there's no substitute for measuring performance yourself for the specific deployment scenario you're interested in. We made it relatively easy to do that by including a telemetry package in Quantum Leap and examples of analysing the collected metrics with Pandas