Investigate Australia, Queensland adapter for quality and coverage

[russbiggs]

Per a user email we may be missing some sites and there may be another API for access AQ data from queensland

https://airquality.des.qld.gov.au/v1/stations/PROS/parameters/131/measurements?pagesize=1000&pagenumber=1

This data seems to show only 24 hr averages and AQI (though other sites may have different periods)

For our current data source data seems really spotty. Its not clear if this is an issue with the source or how we are scraping it.

e.g. (note the gaps)

[majesticio]

Ok, I will look into it

[majesticio]

Here are all the unique parameters from all stations from the new Queensland source:	level_0	index	parameter_id	parameter_name	abbreviation	units	1hr_standard	standard	category_id	link	standard_units	station_id
0	0	131	Particle PM2.5	PM2.5	µg/m³	50.0	25.0	5	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-monitoring/trend-graphs\#c	µg/m³ (24hr avg)	abpo
1	1	1	Wind direction	WD	deg	0.0	0.0	2	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-monitoring/meteorology-factors\#windspeed	deg (1hr avg)	ald
2	2	2	Wind speed	WS	m/s	0.0	0.0	2	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-monitoring/meteorology-factors\#windspeed	m/s (1hr avg)	ald
3	3	9	Humidity	Hum	%	0.0	0.0	2	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-monitoring/meteorology-factors\#humidity	% (1hr avg)	ald
4	4	10	Temperature	Temp	°C	100.0	0.0	2	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-monitoring/meteorology-factors\#temperature	deg C (1hr avg)	ald
5	5	27	Rainfall	Rainfall	mm	0.0	0.0	2	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-monitoring/meteorology-factors\#rainfall	mm (1hr total)	ald
6	6	16	Nitrogen dioxide	NO₂	ppm	0.08	0.08	1	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/nitrogen-oxides	ppm (1hr avg)	aru
7	7	3	Ozone	O₃	ppm	0.1	0.065	1	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/ozone	ppm (8hr avg)	aru
8	8	18	Particle PM10	PM10	µg/m³	100.0	50.0	1	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-monitoring/trend-graphs\#b	µg/m³ (24hr avg)	aru
9	21	6	Visibility	Bsp	Mm⁻¹	235.0	235.0	1	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-monitoring/trend-graphs\#a	Mm⁻¹ (1hr avg)	ayr
10	30	13	Solar radiation	Rad	W/m²	0.0	0.0	2	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-monitoring/meteorology-factors\#solar	W/m² (1hr avg)	ayr1
11	32	7	Sulfur dioxide	SO₂	ppm	0.1	0.1	1	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/sulfur-dioxide	ppm (1hr avg)	bar
12	42	160	Hydrogen sulfide (30min)	H₂S	ppb	5.0	108.0	6	https://www\.qld\.gov\.au/environment/pollution/monitoring/air/air-programs/cleanaway-new-chum-odour-issues	ppb (30min avg)	bda
13	99	8	Carbon monoxide	CO	ppm	0.0	9.0	1	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/carbon-monoxide	ppm (8hr avg)	boy
14	124	47	Particles TSP	TSP	µg/m³	250.0	80.0	1	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/samplers	µg/m³ (24hr avg)	can
15	168	20	Benzene	Benz	ppb	0.0	250.0	3	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/toxics\#benzene	ppb (24hr avg)	cgm
16	169	21	Toluene	Tol	ppb	0.0	1000.0	3	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/toxics\#toluene	ppb (24hr avg)	cgm
17	170	22	Xylene	Xyl	ppb	0.0	250.0	3	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/toxics\#xylene	ppb (24hr avg)	cgm
18	171	23	Formaldehyde	Formald	ppb	0.0	40.0	3	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/toxics\#formaldehyde	ppb (24hr avg)	cgm
19	538	52	Lead	Pb	µg/m³	0.0	2.0	4	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/metals\#lead	µg/m³ (24hr avg)	oai
20	539	53	Arsenic	As	µg/m³	0.0	0.3	4	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/metals\#arsenic	µg/m³ (24hr avg)	oai
21	540	54	Cadmium	Cd	µg/m³	0.0	2.0	4	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/metals\#cadmium	µg/m³ (24hr avg)	oai
22	541	55	Copper	Cu	µg/m³	0.0	50.0	4	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/metals\#copper	µg/m³ (24hr avg)	oai
23	542	57	Zinc	Zn	µg/m³	0.0	120.0	4	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/metals\#zinc	µg/m³ (24hr avg)	oai
24	543	41	Nickel	Ni	µg/m³	0.0	0.12	4	https://www\.qld\.gov\.au/environment/pollution/monitoring/air-pollution/metals/\#nickel	µg/m³ (24hr avg)	oai

[majesticio]

It looks like pm10 and pm25 are reported as 24 hour averages, and only so2 and no2 are 1 hour averages. co2 and o3 are 8 hour averages

[majesticio]

It looks like we are already pulling from this source as xml https://apps.des.qld.gov.au/air-quality/xml/feed.php?category=1&region=ALL. I could try to troubleshoot, or it might be better to just redo the adapter using the json data instead of xml

[russbiggs]

Does this explain the spotty coverage though? and does the json endpoint have the "missing" site in question, Proserpine, while xml does not?

[majesticio]

I am not sure about the coverage, the xml is missing Proserpine though (Proserpine is included in the json). It would probably be better to rewrite the adapter

[majesticio]

I am getting a lot of timeouts, both from the API and the xml endpoints. I am guessing this is why the coverage is spotty. Hopefully we can get special access somehow, their public facing server seems to have some traffic issues