CityofToronto / bdit_data-sources

Data sources used by the Big Data Innovation Team
https://github.com/orgs/CityofToronto/teams/bigdatainnovationteam
GNU General Public License v3.0
38 stars 8 forks source link

Investigate new Smartmicro sensors on highways #915

Closed gabrielwol closed 5 months ago

gabrielwol commented 6 months ago

Over the last year, 13 out of commission RESCU sensors have been replaced with Smartmicro sensors. Investigate and add to vds.detector_inventory table.

SELECT DISTINCT ON (vds_id) * FROM vds.vdsconfig WHERE detector_id LIKE 'SMARTMICRO - D%' ORDER BY vds_id, start_timestamp DESC'

gabrielwol commented 6 months ago

Some comparison plots between new smartmicro sensors and rescu sensors they replaced:

DS0030DSR: not clear what drives this difference. Both consistently reporting 1 lane of data (ramp). image

DS0045DND: image

DS0045DSD: image

DS0050DND: image

DS0050DSD: image

DS0070DND: image

DS0070DSD: image

DW0050DWR:

DW0060DWR:

DW0070DER: image

DW0080DWR: image

DW0090DWR: image

example query:

SELECT datetime_15min::date,
SUM(count_15min) FILTER (WHERE di.detector_id LIKE 'SMARTMICRO%') AS smartmicro_volume,
SUM(count_15min) FILTER (WHERE di.detector_id NOT LIKE 'SMARTMICRO%') AS pre_smartmicro_volume
FROM vds.counts_15min_div2 AS c15
JOIN vds.detector_inventory AS di ON di.uid = c15.vdsconfig_uid
WHERE di.detector_id LIKE '%DS0050DSD%'
GROUP BY 1 ORDER BY 1;
gabrielwol commented 6 months ago

@A-DUYVESTYN you may be interested in the graphs above