search

graphite-project / ceres

Distributable time-series database (not actively maintained)
Apache License 2.0
355 stars 77 forks source link

Data from several slices should be merged, not appended. #21

Open Civil opened 9 years ago

Civil commented 9 years ago

If for some reasons you have situation when you have overlapping slices, reading data from the interval that matched one of them, will result in more points, then it should be there.

For example - you have some metric that due to broken merge/rollup got into this shape for the interval 1404000000 - 1412640000:

1404000000@86400.slice with 100 points in it (up to 1412640000)

1412294400@21600.slice with 100 points in it (up to 1414454400)

If you'll try to read the data from 1404000000 to 1412640000, you'll get in result 104 points (100 from 1404000000@86400.slice and 4 approximated points from 1412294400@21600.slice appended to the end).

I understand that it shouldn't happen, but right now there is no way to fix this situation (except to remove wrong data by hands/script), but I think that it should be handled by ceres.

Civil commented 9 years ago

Should be mostly solved by this commit (against dkulikovsky's fork): https://github.com/vladimir-smirnov-sociomantic/ceres/commit/5d1b7424b69dd591a4ba12690d5ea24939b21289

Civil commented 9 years ago

(Sorry no tests for that yet, currently I don't have time to create proper test for that, but at least steps to reproduce the issue are in this ticket)