man-group / arctic

High performance datastore for time series and tick data

https://arctic.readthedocs.io/en/latest/

GNU Lesser General Public License v2.1

3.06k stars 583 forks source link

Retention Policy in Tickstore #793

Closed PalakMunet closed 5 years ago

PalakMunet commented 5 years ago

Arctic Version

# 1.79.2

Arctic Store

#  TickStore

Platform and version

Red Hat Enterprise Linux Server release 7.4 (Maipo), 64-bit

Description of problem and/or code sample that reproduces the issue

I want to put retention policy on my data on tickstore so that data automatically gets deleted 5 days from upload. I am not able to figure out how to do that.

shashank88 commented 5 years ago

This is not something that Tickstore supports at the moment. You can use VersionStore and with that, you specify the keep_min time period when you write/append stuff.

You can probably create a cron job to do this if you want to use tickstore

PalakMunet commented 5 years ago

And how about using chunk store?Does it have support for retention policies?

PalakMunet commented 5 years ago

Also even after reading many issues by people I am unable to gain clarity on which store should I use. I have historical tick data in GBs and i want to store and use it further to perform analytics. And I want to reduce latency in retrieving data along with making it less memory intensive and to have better management and control over my data.

bmoscon commented 5 years ago

version store is for versioned objects, appends generate a new version. chunk store is for storing data in specific time windowed chunks (no versioning), but you can edit data already written. tickstore is the most basic of all, you store data in specific time windows.

PalakMunet commented 5 years ago

Ok, I am going with chunk store for my use case. Thanks for quick response!