search

openreferral / specification

The Human Services Data Specification - a data exchange format developed by the Open Referral Initiative
https://openreferral.org
Other
117 stars 50 forks source link

Providing a reference datapackage aligned with the specification #233

Closed pmackay closed 12 months ago

pmackay commented 3 years ago

Having previously raised #82, the situation has moved on somewhat 😀

I am working on some tooling ideas that use datapackage.json as input, reading it with the relevant ruby gems.

The version of datapackage.json in this repo is obviously the most uptodate, as this specification project has the most iterative updates. The problem is that using this as input fails trying to read the schema, because tableschema-rb looks for a CSV file based on the path value. The Data Location part of the spec suggests data must be present in some form.

Now there is sample-HSDS-datapackage that includes 2 examples, each with a datapackage.json file and a set of sample CSVs. However, this has not been updated with spec changes.

Options

  1. Update sample-HSDS-datapackage to match the spec.
  2. Add an example folder to this spec and provide a canonical set of template CSVs in it, that mean the specification datapackage.json can be used directly. Or, if the paths should be maintained in a folder at the same level as the datapackage.json, could datapackage.json be moved up into a reference folder (or some other name) that also includes CSVs?
  3. Other ideas?

The 2nd option seems like it would be more likely to result in the template CSV files being maintained, and thus the datapackage.json that defines the spec being directly usable for tooling. But it has other implications.

devinbalkind commented 3 years ago

Option 2 sounds ideal to me.

I think most people experience data as spreadsheets and are comfortable with "templates".

Has anything prevented us from simple putting together a Google Sheets file with all the CSVs and incorporated that into the docs?

Could we close this issue by making such a Google Sheet, downloading each table as a CSV and placing it in the repo?

pmackay commented 3 years ago

@devinbalkind dont the path values need to be routes to text CSV files? I guess I'm not sure if a sheet would work, but it feels like it could introduce format and parsing complexities that are not an issue for text files.

Unless you mean having a sample sheet as well as simple CSV files that are referenced by the datapacakge.json file?

devinbalkind commented 3 years ago

I think I wasn't clear.

My proposal is:

Does that make sense?

pmackay commented 3 years ago

OK that sounds good, the 1st step would be a very useful reference to keep uptodate and the 2nd would help solve this issue.

robredpath commented 3 years ago

Obviously, doing this by hand makes sense for now, but I don't think it'd be hard to automate. I doubt I'll get a chance to do it any time soon, but if others agree, I'll add an issue and see if we can pick it up at some point.

mrshll1001 commented 12 months ago

Closing as the HSDS Schema tools now generates a reference datapackage.json, for not only 3.0 but for profiles as well.