HTTP API /api/query support cache

OpenTSDB / opentsdb

A scalable, distributed Time Series Database.

http://opentsdb.net

GNU Lesser General Public License v2.1

5k stars 1.25k forks source link

HTTP API /api/query support cache #801

Open taurenyu opened 8 years ago

taurenyu commented 8 years ago

We use grafana+tsdb as monitor system, but I find that HTTP API /api/query not creating cache files. The throughput of tsdbs is 80wqps, the sendbytes of hbase is over 20G+bytes sometimes. 413c7ad8-0e9e-4a22-ba9e-73e18c6d9fd7

johann8384 commented 8 years ago

The API does not cache the data. There is a 3rd party tool which will accept queries from OpenTSDB and do two things with them. 1) Cache the results in redis 2) shard the query into 1 hour blocks and perform the queries for each 1h block in parallel, so if you ask for 8h worth of data it will execute 8 1h queries against OpenTSDB and cache 8 1h blocks in Redis.

johann8384 commented 8 years ago

Also, Splicer sends the queries to TSD instances on the same regiinservers holding the region which cuts down the network transfer. This functionality from Splicer could be implemented as an HTTP RPC plugin and could use the service discovery plugins framework added in 2.3.0 to detect TSD instances available where the regions are.