micro-analytics / micro-analytics-cli

Public analytics as a Node.js microservice. No sysadmin experience required! šŸ“ˆ
MIT License
734 stars 39 forks source link

Time based segmenting #3

Closed sean-roberts closed 7 years ago

sean-roberts commented 7 years ago

As it is right now, the service returns two types of data, aggregated counts and ?all to get the full records back.

The problem is rooted with the aggregated counts. They, as of today, are the sum total since the beginning of time. Which, becomes less useful as time goes on. Because in 3 months from now if I have 13,000 views on a page, this doesn't help me know how my app trends (getting better or worse view counts). This is where the usefulness of ?all comes in. With that param, you can get all of the time data about your metrics. However, this is also since the beginning of time. So, in your aggregation of your trends over time, you will need to fetch all records in order to calculate the time segments of the last day, weeks, months, etc.

So, the proposal is really for both endpoint styles, count and ?all. You could have them support a ?since={time} and a ?before={time} to do really basic time range filtering.

The important benefit here is that it is done on the server side query so it doesn't affect the performance of the endpoint over time.

mxstbr commented 7 years ago

What should {time} be structured like in the param? Is there some time module in Node that'd allow us to transform 31.11.2016, 11/31/2016 to UTC?

At that point the filtering would be super easy, just a .filter(view => view.time < toUTC(query.before)/ .filter(view => view.time > toUTC(query.since)

sean-roberts commented 7 years ago

One thought would be that you could simply have the end user submit the UTC millies and they can use whatever mechanism they like.

And layer a friendlier impl if the value is not an int. I know moment can handle these transforms easily.

I have always loved the simplicity of ?since=2m1w4d3h3mi which is also pretty simple to have moment support as well

mxstbr commented 7 years ago

I'm hesitant to ship a module that big just for that simple feature, I'd rather people send us UTC directly and take care of that?

sean-roberts commented 7 years ago

Yeah, that's totally viable V1 of this. Good (shipped) > perfect

sean-roberts commented 7 years ago

This is fixed šŸ‘