ytyou / ticktock

TickTockDB is an OpenTSDB-like time series database, with much better performance.
GNU General Public License v3.0
76 stars 9 forks source link

Is it possible to delete data points? #57

Closed vilisseranen closed 1 year ago

vilisseranen commented 1 year ago

I don't seem to be able to delete data points. OpenTSDB doc says tsd.http.query.allow_delete needs to be set to true but I don't see such option in the ticktock config. Any suggestion?

ylin30 commented 1 year ago

@vilisseranen Unfortunately TT doesn't support deletion yet. We don't have a plan to support it since we don't see very compelling uses cases for deletion, at lease for the time being.

According to our understanding, deletion may be used to

  1. save disk spaces
  2. correct a data point due to human mistake

To solve 1, TT saves data in many folders, one per TSDB ordered by time range (default 1 tsdb for 1 day). So you can just delete outdated TSDB folders or move them to other disks for backup, to save disk spaces.

To solve 2, you can reinsert a data point (with same time series and same timestamp) with a different value. TT will override the old data point with the new one, instead of adding a new data point at the same timestamp.

Please let us know if the two cases fit your cases, or you have a good reason to support explicit deletion.

vilisseranen commented 1 year ago

Thank you for your answer. I think I can work with solution 2.

Just for context, I am using ticktock on an orangepi zero2 and I use it to store the temperature for different sensors I have in the house. Sometimes a sensor will send some bad data. I guess I could also filter the data before storing them in ticktock.

ylin30 commented 1 year ago

Just for context, I am using ticktock on an orangepi zero2 and I use it to store the temperature for different sensors I have in the house. Sometimes a sensor will send some bad data. I guess I could also filter the data before storing them in ticktock.

By 'bad', do u mean bad format or bad value? If the former, just send them to TT and TT should be able to reject them.

vilisseranen commented 1 year ago

I meant bad value. For example it's 20degres in the room and the sensor for some reason sends -5degres.