search

TerriaJS / terriajs

A library for building rich, web-based geospatial data platforms.
https://terria.io
Apache License 2.0
1.17k stars 362 forks source link

Census data at SA1 #1584

Closed stevage closed 6 years ago

stevage commented 8 years ago

It's important for ParlMap users to be able to access ABS Census data at SA1 level. I vaguely recall having tried to do this previously and found some kind of complication in doing that through the ITT interface. Maybe this will be easier using the SDMX interface?

There is also obviously the issue that SA1 data is big - 55,000 regions for the whole country. So either we should limit by state, or warn the user before fetching it all.

chris-beer commented 8 years ago

Easier, absolutely - it will be as simple as getting data at other levels.

However the problem isn't with the services - it's that we don't publish SA1 level Census data in .Stat for anything other than South Australia at the moment :/

Direct data pack (CSV etc) downloads is an option in the now - perhaps AB and Toby et. al. could do something with the data.gov.au API tool to make that approach work in the short term but yes - you'd be talking a huge amount of files (SA1 region by region). We possibly could arrange something there around getting a complete dump to storage media and sending it over?

Cheers

Chris

Chris Beer Assistant Director (Data Services and APIs) - Dissemination Enterprise Wide Services Statistical Business Transformation Program | Australian Bureau of Statistics +61 2 6252 5614   chris.beer@abs.gov.au     www.abs.gov.au

The Australian Bureau of Statistics acknowledges the traditional custodians of country throughout Australia and recognises their continuing connection to land, waters and community. We pay our respects to them and their cultures, and elders, both past and present

From: Steve Bennett notifications@github.com To: TerriaJS/terriajs terriajs@noreply.github.com, Date: 03/06/2016 12:05 PM Subject: [TerriaJS/terriajs] Census data at SA1 (#1584)

It's important for ParlMap users to be able to access ABS Census data at SA1 level. I vaguely recall having tried to do this previously and found some kind of complication in doing that through the ITT interface. Maybe this will be easier using the SDMX interface?

There is also obviously the issue that SA1 data is big - 55,000 regions for the whole country. So either we should limit by state, or warn the user before fetching it all.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

tobybellwood commented 8 years ago

Our solution for ParlMap was to divide the country up into approx 10k chunks along GCCSA boundaries to be able to use the csv-geo-au for SA1. From memory, the chunks were

Obv we could host the BCP csv files on DGA, but there is considerable merit to having a 'middleman system' to process the query, output the csv for the chosen fields, host it temporarily, and then display in ParlMap (this was on the original roadmap IIRC).

RacingTadpole commented 7 years ago

SA1s are now possible, and where the ABS puts out SDMX-JSON data at the SA1 level (ie. for South Aust), you can see them in the catalog. Does this issue still apply to ParlMap? If not, OK to close now?

AnaBelgun commented 7 years ago

Not really. there is still an open issue on parlmap, but it won't be included as part of the last release of parlmap, due today 18 July. I'll let DPS know and it might be included in a future release, as part of a new project/contract.

kring commented 6 years ago

I'm closing this because I don't think there's anything to do in TerriaJS. SA1s, including all 55k in a single CSV, are now supported.