search

openaq / openaq-fetch

A tool to collect data for OpenAQ platform.
MIT License
83 stars 39 forks source link

coordinates PA SASKATCHEWAN CA AirNow #169

Closed maelle closed 7 years ago

maelle commented 8 years ago

They are 0,0 currently.

jflasher commented 8 years ago

Yeah, we've got confirmation from AirNow operators that when they don't have data, they pass 0,0. We can either:

  1. Keep saving the 0, 0 since it's the data coming from the source
  2. Detect that it's 0, 0 and don't save any coordinates

I'm generally a bit hesitant to make too many changes to source data, but it seems like in this case, doing #2 should be alright?

@dolugen any thoughts?

maelle commented 8 years ago

I've noticed this when plotting all sources. I'd vote for saying the coordinates are not available when they are 0,0 ;-)

dolugen commented 8 years ago

I agree with @jflasher, I think it's safe to treat 0,0 as none coordinates.

Revision: Voting for #1 after discussing with @RocketD0g

dolugen commented 8 years ago

Following that logic, negative measurement values should also be ignored and not saved.

RocketD0g commented 8 years ago

@dolugen - I hear you, but I don't quite agree with that for the measurements. Small negative values can simply represent typical electronic drift in measurements (think of a weight scale that isn't quite tared properly). Such data can later be gone over for QA/QC, adjusted and be valuable. Also, specifically '-999' is an international value that indicates that the instrument isn't properly reporting data. This can be a valuable signal to know, as well.

In general, I think it is very important to capture and save the information precisely as it is reported, even when the measurements are negative, for posterity. I think it is more debatable on using it for real-time graphing/communication applications.

I think the 0,0 coordinates are a slightly different case in that we have a reasonable expectation that a given country has not set up monitors on Null Island and that those coordinates will never be useful.

dolugen commented 8 years ago

I see, I didn't know negative values were used like that.

RocketD0g commented 8 years ago

Oh, I think it's a super important item to bring up and not at all obvious. As it is, I'm not sure how we should handle small negative values for calculating long-term averages...for instance, if we throw out small negative values, we could potentially be biasing the values too high. If we include them, well, physically not reasonable either. If we call them zero, that's probably inaccurate too - but maybe the best we can do? At any rate, I'm diverting the conversation slightly but this has been on my mind! :P

dolugen commented 8 years ago

Also, specifically '-999' is an international value that indicates that the instrument isn't properly reporting data. This can be a valuable signal to know, as well.

That got me thinking, maybe Null island could also be treated as a deliberate signal as well. Instead of the absence of coordinates, there's something, but the data provider has explicitly set it to null, either the geodata was unavailable at the time, or they are not able to provide.

I don't know if it's an international standard or not, but looks like there're places that use it as a placeholder. From a release note of Natural Earth Data:

WARNING: A troubleshooting country has been added with an Indeterminate sovereignty class called Null Island. It is a fictional, 1 meter square island located off Africa where the equator and prime meridian cross. Being centered at 0,0 (zero latitude, zero longitude) it is useful for flagging geocode failures which are routed to 0,0 by most mapping services.