search

TritonDataCenter / dragnet

event stream analysis
MIT License
11 stars 4 forks source link

indexes could have implicit time field #13

Closed davepacheco closed 9 years ago

davepacheco commented 9 years ago

A user reports this odd behavior:

$ dn query-mjob --before=2014-08-01T00:00:00Z --after=2014-07-01T00:00:00Z -b 'storage[aggr=quantize]' ...
dn: must specify a "date" field to use "before" and "after" constraints

The problem is that the date string is applied both by pruning inputs and as a filter on the inputs it finds (in order to deal with files whose contents include points both before and after one of the endpoints). In order to do that, dragnet has to know which field represents the timestamp. But I think it should already have a field for that, which it used to chunk the index files into hours or days. I think this is stored in the index. It might not have sufficient resolution, though.

davepacheco commented 9 years ago

This is obviated by #22, which moves the time field to the datasource definition. You need it there in order to build the index. Once you've built the index, it's not used as part of querying it.