openaq / openaq-ingestor

1 stars 1 forks source link

Realtime source data cannot ever be colocated #2

Open caparker opened 2 years ago

caparker commented 2 years ago

We are using only the location to match a sensor node

UPDATE tempfetchdata_nodes t
SET sensor_nodes_id = sn.sensor_nodes_id
, added = FALSE
FROM sensor_nodes sn
WHERE t.geom IS NOT NULL
AND st_dwithin(sn.geom, t.geom, .0001)
AND origin='OPENAQ';

UPDATE tempfetchdata_nodes t
SET sensor_nodes_id = sn.sensor_nodes_id
, added = FALSE
FROM sensor_nodes sn
WHERE t.sensor_nodes_id is null
AND t.site_name is not null
AND t.source_name is not null
AND t.site_name = sn.site_name
AND t.source_name=sn.source_name
AND origin='OPENAQ';

Which could lead to issues if two sensors are co-located.

russbiggs commented 2 years ago

I wonder if there is every lat,lng drift, which would also cause some weirdness here. @caparker do you have a recommended alternative to lat,lng?

caparker commented 2 years ago

I think we should use some sort of namespace idea that includes the node/system/sensor to match directly to the sensor, much like they are doing in the second update. But we do have to decide what to do if the names all match but lat/long has changed. I would assume at that point its a new deployment of the same set of instruments.