search

ConnectedHumber / Air-Quality-Web

The web interface and JSON api for the ConnectedHumber Air Quality Monitoring Project.
https://sensors.connectedhumber.org/
Mozilla Public License 2.0
9 stars 4 forks source link

HeatMap blob combining #30

Closed bsimmo closed 5 years ago

bsimmo commented 5 years ago

as noticed and talk about on Slack, the count is increasing on merge, it should be an average (and whatever drop off) at it's simplest

bsimmo commented 5 years ago

starting here (think this link might work) https://connectedhumber.slack.com/archives/CGAEG1G5T/p1556111115009800

sbrl commented 5 years ago

Ahhh, I see what you mean - though it took me several re-reads. That is a very awkward bug to fix - I'll have to think about it.

For reference, the bug in question is that if multiple sensors have the same location and report the same reading, the heatmap adds the readings together, instead of averaging them.

The solution here is not exactly straight-forward. We can comb through the settings that Heatmap.js provides, but it's probably that there won't be anything that does the trick. To this end, we have a few options:

  1. Open an issue against Heatmap.js to ask for an option
  2. We may end up having to completely re-implement this ourselves, which will require some serious effort. If this is the case, I'll probably derive the solution from my Masters project, which I'll be doing soon. The timescale for a fix here would be in ~September time.

Either way, I'd like to ponder over this for a while.

Also, just a note: When reporting issues, please report them on here first instead of mentioning them in the Slack chat. This serves as a point of reference where it won't get lost and I can easily refer back to it.

bsimmo commented 5 years ago

Sorry for not copying it out and rewriting it here (I would have eventually). Just have to juggle my time to do it :-)

It not just the same spot though, when two heatmaps combine, at the overlaps it adds them together though. Great for a 'heatmap' but not for our project. I scanned through the links last night, that you gave and couldn't see anything to select 'average' or similar. Though that's also me reading yet another new language for me.

But thought I would point it out as it is unfortunately wrong for the display of our data.

Hard to see at the moment as all sensors are reading low.

For anyone following If you scroll back in time to 2019-04-05T19:00:00 (nice addition by the way, and as I turns out, useful :-)) Then have a play with blob radius, you can see how it goes wrong with the Cottingham area sensors. Also look at Temperature when you are there, you'll see blue turns to yellow/orange.

Ben

On Thu, 25 Apr 2019, 4:41 pm Starbeamrainbowlabs, notifications@github.com wrote:

Ahhh, I see what you mean - though it took me several re-reads. That is a very awkward bug to fix - I'll have to think about it.

For reference, the bug in question is that if multiple sensors have the same location and report the same reading, the heatmap adds the readings together, instead of averaging them.

The solution here is not exactly straight-forward. We can comb through the settings that Heatmap.js provides, but it's probably that there won't be anything that does the trick. To this end, we have a few options:

  1. Open an issue against Heatmap.js to ask for an option
  2. We may end up having to completely re-implement this ourselves, which will require some serious effort. If this is the case, I'll probably derive the solution from my Masters project, which I'll be doing soon. The timescale for a fix here would be in ~September time.

Either way, I'd like to ponder over this for a while.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ConnectedHumber/Air-Quality-Web/issues/30#issuecomment-486727145, or mute the thread https://github.com/notifications/unsubscribe-auth/ACYAXN4VHND5YFLBEK47KPTPSHGMJANCNFSM4HIDXWWA .

sbrl commented 5 years ago

I've been having a little think about this. I think the solution is going to be very awkward and messy. One solution I'm toying with at the moment is to pre-render the map on the server-side. I'll (probably) be doing this for my summer project, so I can port the code for that once I'm done. Another would be to write a custom overlay, but that sounds like a lot of work, and I don't really know how to start that. I'd have to research it some more. Either way, I think my summer project will shed some light on this, so I'll keep this project in mind as I work on that.

sbrl commented 5 years ago

Suddenly, an idea! We could use a voronoi diagram + https://dynmeth.github.io/RaphaelLayer/

robinharris commented 5 years ago

I need help to understand enough to comment....maybe you can explain at the meeting this evening?

sbrl commented 5 years ago

Hey, @robinharris! Sorry I didn't see this before the meeting.

Basically, with the current heatmap, when more than 1 device is located in the same place on the map, the heatmap adds the 2 values together to get the final value that's used to render the heatmap on your screen.

What we actually want is an average, as more than 1 device in the same location should serve to improve the accuracy of the existing reading, instead of add to existing readings.

Under the current system, if you've got enough devices in the same place, PM2.5 will always show as purple - even if each device individually is reporting a low value.

In addition, the heatmap tails off at the edges as it gets further and further away from the device marker itself. This is also incorrect, as if we don't have a sensor somewhere it doesn't mean that there isn't any pollution there - it simply means we have no data for that location. In reality, we want it to interpolate between 2 device markers to determine the correct colour to show on the heatmap.

Currently, I'm considering using a library called Raphael.js to render arbitrary shapes onto an overlay on to of the map. Ive got multiple ways I could use this to generate the kind of map overlay we're after.

  1. We could generate a voronoi diagram as I've talked about in #31.
  2. We could render squares and interpolate the value based on the value of the closest 3 devices.

We'll have to measure the performance of these approaches carefully to avoid issues.

For either approach, a quadtree would probably go a long way to improving lookup times - and many high-performance implementations exist already that we can take advantage of.

For approach 1, we'll need a relatively optimised implementation of fortune's algorithm.

Currently, I'm undecided which approach to take. Thoughts?