Feature: TLOG cursor

[amclain]

I have been using a TLOG to store time-series data for a small project to evaluate some technology. I am retrieving the data from Jylis and displaying it on a graph:

What I've noticed is that the query time is roughly 100ms per 10,000 records. Considering I now have close to 90,000 data points in the TLOG, the response time is about 900ms. This is causing a noticeable delay when loading the graph. If I only return the first 100 records, the response time is about 10ms.

Based on this I'm wondering if implementing a cursor in Jylis for the TLOG data type makes sense. That way I could fetch the first 10k data points at a reasonable response time. Then if the user decides to scroll the graph I can fetch the next 10k, and so on.

An alternative I've thought about is sharding the TLOG on the application side if this isn't an appropriate feature for Jylis. For example, I could create a new TLOG for each device for each day, and then put logic in my application to fetch from the appropriate TLOG based on the section of data the user is retrieving.

Thoughts?

[jemc]

It probably does make sense to provide some kind of mechanism to iterate/paginate the TLOG results, but for your particular case, it seems like it might be more appropriate to provide a way to filter by time range.

That is, if your graph is showing a certain time range, then you could use a TLOG command that returns only entries between those two timestamps.

[amclain]

you could use a TLOG command that returns only entries between those two timestamps.

Yeah, that works too.