Open RacingTadpole opened 8 years ago
@LegoStormtroopr suggested in another channel:
What you are describing is recording all of the structural metadata around particular columns.
We could record all of that metadata around those Data Elements, and then with very minor tweaking NationalMap could then interpret that metadata to understand how to display data on NationalMap.
Let me see if I understand… do you mean we could format the wordy spec at https://github.com/TerriaJS/nationalmap/wiki/csv-geo-au into a machine-readable form, which TerriaJS can then read?
To which @LegoStormtroopr replied
At the least, for the columns yes.
Cool! I’m keen to give it a try (with a side motivation to help me get my head around the concept of metadata). How do I start?
This sort of approach came up in a conversation with ABS geography around the concepts of statistical geography yesterday.
I've always been keen to see the csv-geo-* get legs, but hadn't thought of csv-- !
I'm also interested in whether data packages could be used to carry this extended meaning (possibly generated by future dga?) http://frictionlessdata.io/data-packages/
Just fwiw, "geocsv" is some kind of de facto standard used by Mapbox and some others: https://github.com/mapbox/detect-geocsv
Looks like the crux of that is, first establish that it is csv, then check for these column names:
hasOne(lowerCaseNames, ['wkt', 'geom', 'geometry', 'geojson']) ||
((hasOne(lowerCaseNames, ['x', 'lon', 'lng', 'long']) ||
hasOneThatContains(lowerCaseNames, 'longitude')) &&
(hasOne(lowerCaseNames, ['y', 'lat']) ||
hasOneThatContains(lowerCaseNames, 'latitude')));
@tobybellwood thanks for the data-packages link. That could be useful as another way to tell terriajs how to interpret other files (like our custom "init" json-format does now, but easier to construct). Is that what you meant? I can't see how to use data-packages to define a standard though.
Ok @tobybellwood so I'll just recap how the Data Packages work for everyone's benefit:
All these structures are extensible, and it's ok to include extra attributes not explicitly defined in the spec.
So, what can we do with it?
tableStyle
attributes in those data packages, or support something a bit more data-packagey.I'd go a bit further an suggest that if data.gov.au v2.0 is going to have a metadata registry associated with it, defining and publishing all of the structural metadata would be beneficial for a number of reasons:
What we'd need to accomplish this are services to:
We are now using csv-geo-au for things:
lga_code_2011
in their SDMX-JSON format)I wonder if we should pull out each of these pieces into their own standards, and define
csv-geo-au
in terms of them?