criteo / biggraphite

Simple Scalable Time Series Database
Apache License 2.0
129 stars 36 forks source link

WIP Implement a new way to do disk cache #470

Closed geobeau closed 5 years ago

geobeau commented 5 years ago

Attempt at improving disk cache and simplify the implementation.

coveralls commented 5 years ago

Coverage Status

Coverage remained the same at ?% when pulling 3ff1b7a1ed1b5ea4add777d308c66d47bb87ae2b on geobeau:disk_cache_new into d93acfe3548689987a3ae363254885678c1becd1 on criteo:master.

iksaif commented 5 years ago

Note: if we finish re-writing the way metrics are globbed, the disk cache can go away because graphite-web will not need it anymore :)

geobeau commented 5 years ago

It's used by carbon. But we are investigating if we need something like this or not

iksaif commented 5 years ago

Carbon is not supposed to use the disk cache

Le mer. 17 oct. 2018 à 10:04, Geoffrey Beausire notifications@github.com a écrit :

It's used by carbon. But we are investigating if we need something like this or not

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/criteo/biggraphite/pull/470#issuecomment-430530360, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_DA5mdhbVEe8gRFc-GzdqI9EYUc1J5ks5uluR2gaJpZM4XMxS9 .

geobeau commented 5 years ago

Because it's too slow? Goal would be to have a cache that survives the restart of a process

iksaif commented 5 years ago

Because it was too problematic (because of locking) and too slow (including the has function). The workaround was to make startup work without the cache and slowly create the metrics if necessary after startup (the rate is not configurable but could be, it also probably lacks back-pressure).

geobeau commented 5 years ago

Ok, thanks for the informations!