Display sensors with stale data

[abesto]

Fixes #86

• The naive implementation leads to an extremely slow InfluxDB query (consistent timeouts at ~5 days back). So I went and learned a bit about optimizing InfluxDB queries, and you can see the result. We can now query for sensors 30 days into the past no problem.
• While working on this, I noticed the dashboard is currently very spammy with DB requests, so I added on server-side caching with a 1-minute TTL.

Sensors whose latest measurement is more than 2 hours in the past are shown with lower opacity:

Hovering a sensor shows the timestamp of the latest measurement:

Clicking a sensor with no data in the queried range reports this sad state of affairs (instead of, say, crashing the page):

[keenanjohnson]

This looks really good! However, when I run this branch locally, I don't see the opaque "stale" sensors. Can you confirm if this is a my machine problem or if that exists when you run it as well?

@abesto

[keenanjohnson]

Nevermind, it was a problem on my end. Looks great!

[keenanjohnson]

I did receive this error after running though:

[abesto]

Hm, I thought I'd fixed that. Lemme see...

[abesto]

Hooo boy. OK so. This symptom comes up when we try to get Pandas to tell us / to operate on the Time field, which doesn't exist in one particular condition: when there is no data from the sensor at all in the queried period. This is what initially prompted me to add that message. In this case, what seems to be happening is:

• You click a sensor with no data
• You reload the page - the selected sensor is saved in the server, which is a big huge red flag in itself. Every user will load the page with the sensor selected that the last user who happened to be around clicked last.
• The next time you load the page, we try to show data about the last selected sensor, without hitting the if sensor_data.empty check. Because sensor_data is a global. Which it has NO business being.

Let me try to untangle this.

[abesto]

... further complicating matters, update_graphs is triggered even when clicking a cluster of sensors, at which point we don't have the information "which sensor was previously displayed". So either I need a way to not trigger update_graphs on cluster clicks, or else store the last displayed sensor in the browser (as opposed to: on the server, as currently)

[keenanjohnson]

Thanks for digging into this! I'm taking a look now

[keenanjohnson]

This looks generally good @abesto , but I still don't see the stale sensors on my local copy when I run it?

Can you confirm it works locally for you?

[keenanjohnson]

Ah I see the changes locally now. Something must have been funny with my browser cache.

[keenanjohnson]

Perfect!

On Mon, Jul 25, 2022 at 12:34 PM Zoltán Nagy @.***> wrote:

@.**** commented on this pull request.

In app.py https://github.com/Ribbit-Network/ribbit-network-dashboard/pull/87#discussion_r929229907 :

@@ -112,6 +112,12 @@ def serve_layout(): const {min, max, colorscale, circleOptions, colorProp} = context.props.hideout; const csc = chroma.scale(colorscale).domain([min, max]); circleOptions.fillColor = csc(feature.properties[colorProp]);

• // Lower opacity for sensors with stale data
• if (new Date() - new Date(feature.properties._time) > 1000 60 60 * 2) {

Yeah that's a good idea. #90 https://github.com/Ribbit-Network/ribbit-network-dashboard/pull/90 moves JS code out into .js files, so let's do this on top of that.

— Reply to this email directly, view it on GitHub https://github.com/Ribbit-Network/ribbit-network-dashboard/pull/87#discussion_r929229907, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATQ3FQB546YRBX643C6GPDVV3T2ZANCNFSM54M2447A . You are receiving this because you modified the open/close state.Message ID: <Ribbit-Network/ribbit-network-dashboard/pull/87/review/1049963656@ github.com>