search

agoragames / kairos

Python module for time series data in Redis and Mongo
BSD 3-Clause "New" or "Revised" License
207 stars 38 forks source link

Default value for 'key_prefix' would simplify mass key deletion #9

Closed inactivist closed 12 years ago

inactivist commented 12 years ago

In keeping with your Dragons! warning (680842014454c618ddc6d84b81663b2d98d42f43) and issue #8, I'd like to be able to delete all keys used by the TimeSeries class given a set of parameters provided to the TimeSeries constructor. I see that I can do this easily if I supply a key_prefix to the constructor, but I'm out of luck otherwise.

I'd like to implement a cleanup method in Kairos, but given the current key structure when no key_prefix is given, I don't see a way to do it without knowing the name parameters used in the various method calls.

Perhaps it would simplify matters if there was a sensible, non-empty default value for key_prefix in the constructor.

In any case, I'm looking for suggestions on how to do this cleanly.

awestendorf commented 12 years ago

The beta refactor adds support but I haven't had time to merge this into master.

https://github.com/agoragames/kairos/blob/beta-refactor/kairos/timeseries.py#L162

Like RRD, the Timeseries class is used to define the structure of the series, and then one inserts into that for a given name. For example, Timeseries "daily average", to which you insert data for 'requests', 'users', 'bytes', etc.

Does the new API address your concerns? All the methods for interacting with a Timeseries have a name parameter and I'd like to get more feedback before changing the Timeseries API in such a fundamental way.

inactivist commented 12 years ago

Thanks for calling out your refactor branch. The delete method is a welcome addition, and is what I was looking for.

By way of example, say I'm counting references to site visitor referrer domain names or URLs, with a hash of the domain name or URL, but I'm not using a key_prefix (granted, given the current key structure, I should be, but I'm not.) In this example I cannot mass-delete all keys tracked by the Kairos Timeseries instance without resetting the Redis DB, unless I track all the unique domain names or URLs ('names') separately.

Since there's a reasonable workaround (the developer can provide a key_prefix value) this is an acceptable trade-off. The developer should be aware of this issue and plan accordingly.

And, upon reflection, I'm not sure what a sensible default key_prefix would look like or is even possible.

This is probably better handled via a README update; I'll write up a snippet and submit a pull request if you're interested.

inactivist commented 12 years ago

Another way of dealing with this is to specify a Redis DB index for use with the Kairos Timeseries instance; that way you can just reset the specific database. This may not work for everyone as there are a limited number of databases available.

(Mentioning it here for future reference...)