search

GSA / datagov-wptheme

Data.gov WordPress Theme (obsolete)
https://www.data.gov
Other
1.88k stars 411 forks source link

Make field level metadata searchable and link common fields across the catalog #640

Open philipashlock opened 9 years ago

philipashlock commented 9 years ago

We should be able to expose machine readable data dictionaries describing each field or variable in a dataset and index this as part of the catalog in order to more easily discover and understand each variable and provide linkages across disparate datasets.

This could also source metadata from resources linked from the describedBy field such as Swagger API specs and associated JSON Schema files. See https://project-open-data.cio.gov/v1.1/api/

Some precedent with CKAN examples like http://catalogue.data.gov.bc.ca/dataset/mta-mineral-placer-and-coal-titles and related work with Data Packager http://ckan.org/2014/06/09/the-open-knowledge-data-packager/ but ideally this wouldn't require data to be hosted within CKAN (DataStore) in order to provide the field level metadata.

This also relates to https://github.com/project-open-data/project-open-data.github.io/issues/465 and https://github.com/uscensusbureau/citysdk/issues/100

See also http://www.w3.org/blog/data/2015/04/16/csv-on-the-web-seeking-comments-and-implementations/

dportnoy commented 9 years ago

+1 for HealthData.gov. Looking to work together on a solution.

Discussed: http://david.portnoy.us/field-level-data-dictionaries-for-open-data/

philipashlock commented 8 years ago

We'd probably want to build on this extension - https://github.com/cmuphillycapstone/ckanext-dictionary

dportnoy commented 8 years ago

@philipashlock, agreed! I've been talking with the author of this extension on adding this functionality.

philipashlock commented 6 years ago

Just adding this to track more recent discussion and documentation around this as a feature available with CKAN core (#3414) https://stackoverflow.com/questions/47783069/data-dictionary-feature-of-ckan-2-7-x

jalbertbowden commented 6 years ago

ckan next dictionary looks neat. are there any known live examples to poke around on?

philipashlock commented 6 years ago

@jalbertbowden I believe boston is using it, e.g. https://data.boston.gov/dataset/311-service-requests/resource/2968e2c0-d479-49ba-a884-4ef523ada3c0

jalbertbowden commented 6 years ago

that is neat and almost a solution to this thread, i think. html is the perfect format in this situation, the ckan dictionary just needs some microformats/microdata to give it the desired structures. is there a list of examples or desired fields/variables that i can tinker with? should mostly be just a matter of adding attributes and/or elements.