Data Caching?

[bensheldon]

It seems like every front page load requires an API fetch. That seems less than ideal. Could I suggest tossing results into mencached with an time-based expiration of whatever granularity the uptime monitor has?

I suggest memcached because most libs have breaker behavior which makes it pretty resilient and doesn't require setting up a db for local development.

[lippytak]

Let me try to speak for @Mr0grog when I say 'PRs welcome!'

On a more serious note :+1: for the absolutely-correct advice - we're just still in the very experimental phase and haven't decided what visualizations to include so no need to optimize anything quite yet.

[Mr0grog]

Hahaha, I was just about to comment here :)

So. We should do caching, but not now. What’s here at the moment is just experimental and I doubt it will live for long. At the moment, any work is pretty much purely exploratory (besides drafting a narrative, see #6, #13). But for sure: before we make public noise about this site being here, there should be caching. As well as plenty of other things that should be robustified (see also #18).

[pingometer]

@Mr0grog On a related topic, in v1.1 of our API, we're including an average for each report section.

Example:

https://gist.github.com/pingometer/eeff26361df682432645

It's built from all collected data (historical) and self-updating - not sure if this is useful for you.

[Mr0grog]

@pingometer Cool! What’s the timeframe that’s calculated over, though? All time? Not sure that would be super useful for us:

• Some of the monitors weren’t set up perfectly at the start (see, for example, all the issues in this repo about fixing keywords and such on various ones), so including data from all time is probably inaccurate.
• If things improve, we’d probably like that to be visible. The giant mass of past poor performance will hold the numbers down (or inflate them if things are getting worse) for a long time.

More useful might be per week or month statistics.

[pingometer]

@Mr0grog Good point!

It makes sense to include the averages for all-time, the past month, and the past week.

Is there anything else that might be handy for v1.1?

[Mr0grog]

Naively, I’d guess other basic statistics (min, max, stdev) might be useful to some people, but I don’t know that we really have any plans to make use of those. Average over different timeframes is probably the biggest thing.

[Mr0grog]

Welp, I went ahead and did most of this over the last couple evenings when I tried to show screenshots and the page loads finally just got too slow for me. Last bit left is to store all the metadata about each monitor (we now have events, snapshots, and synthesized “incidents” in the local database).

Need to also build a way to dump this data into a local DB for dev updates or into JSON/CSV files for public consumption (#15).

[Mr0grog]

@bensheldon If you want to do the modeling and loading for monitors, have at it.

[alanjosephwilliams]

@Mr0grog seems like most of this has been addressed, and will be addressed by @bensheldon in the rails port. I'll leave open for now until you confirm this issue can be closed.

[Mr0grog]

Yeah, I guess we may-as-well call this done. We should also clean up the way monitors themselves are handled (as opposed to events and incidents), but @bensheldon was already doing that on his rails branch.