Support Socrata - Githubissues

TerriaJS / terriajs

A library for building rich, web-based geospatial data platforms.

https://terria.io

Apache License 2.0

1.19k stars 364 forks source link

Support Socrata #708

Closed stevage closed 9 years ago

stevage commented 9 years ago

Requested for City of Melbourne and ACT.

stevage commented 9 years ago

Now, a couple of issues here. Geospatial support in the versions of Socrata running those two platforms is poor. They don't have GeoJSON enabled:

You can get a listing of all datasets like this: https://data.melbourne.vic.gov.au/data.json

Unfortunately the ?type=maps parameter seems to be ignored. (It works for HTML view).

From that listing you can access the raw data, but the columns are inconsistent. For example this one has "Latitude - processed", and "Geocoded" columns ((-37.5, 144.5)). This one has "Latitude" and "GeoCode"

Line and polygon maps generally have a KMZ download, identifiable by mediatype: application/vnd.google-earth.kmz. Example. Zipped shapefile also available.

kgrochow commented 9 years ago

You should talk with Pia and/or Alex at data.gov.au about it if you haven't. They had tried to import from socrata to migrate users and found it unworkable in the end as I remember. They - or Alex with Pia on secondment - may remember more of the details.

stevage commented 9 years ago

@maxious, any comments?

(man Github is great like that...)

maxious commented 9 years ago

Socrata are on github too ;) It would be nice if the data catalogue APIs were better documented especially around filtering and pagination. It would be even better if www.opendatanetwork.com had an API to get all results by jurisdiction (GovPond for Socrata) I don't know any more about it other than Vizicities asked the same question about how to get just spatial data as well so there's demand.

stevage commented 9 years ago

Ok, spoke to Socrata this morning. Where we got to:

confirmed that there's no API way to find spatial datasets
they will work with us to get Socrata users to tag spatial datasets appropriately, so at least we can filter that way
it's also possible to iterate over every dataset, looking for columns where dataTypeName = location. For instance: https://data.melbourne.vic.gov.au/api/views/axts-5eu4.json So I may attempt to script something, but it would need to be a server side process, so doesn't fit well into the Terria model.

There's also this, but not sure if relevant: http://labs.socrata.com/docs/search.html

Seems to allow meta searching across (US?) Socrata instances.

stevage commented 9 years ago

Ah, even better, here's the JSON catalogue: https://data.melbourne.vic.gov.au/api/search/views

That should be enough - look for datasets with "displayType" : "map",

maxious commented 9 years ago

https://data.melbourne.vic.gov.au/api/search/views?limitTo=MAPS :)

stevage commented 9 years ago

Awesome, nice find!

stevage commented 9 years ago

Done: https://github.com/TerriaJS/terriajs/commit/21f7a09b26c9736482237a45f7bfa2ebe1b85e72