datapackage.json inconsistencies with regard to the official Data Package spec

[xrysanthos]

As I'm currently working with a set of HSDS validation tools, I've come across the following issue.

The official specification for the Data Package schema from Frictionless data, includes a spec for a Table Schema which is being used by their official validator libs.

The validation libraries are using a JSON schema that enforces both primaryKey, foreignKeys.fields and foreignKeys.reference.fields attributes to have values of type array. According to the latest Table Schema spec values can be either of type string or array, but it is also stated that from this version if there is only one value, it should be set as a single element array.

See PR #143 for details.

[timgdavies]

Thanks to @spilio there is now an updated pull request at #145 which aligns the Open Referral datapackage.json with the latest version of the Frictionless Data Table Schema.

This mainly changes the primary and foreign key mentions from strings to arrays, so that where, in datapackage.json, we had:

"primaryKey": "id",
"foreignKeys": [
    {
        "fields": "organization_id",
        "reference": {
            "resource": "organization",
            "fields": "id"
        }
    }
]

we would now have:

"primaryKey": [
  "id"
],
"foreignKeys": [
  {
    "fields": [
      "organization_id"
    ],
    "reference": {
      "resource": "organization",
      "fields": [
        "id"
      ]
    }
  }
]

This has no implications for the structure of HSDS data tables themselves. It is only a change to the underlying schema to make sure Frictionless Data datapackage tools can validate and work with HSDS data.

I'm proposing that this can be merged into '1.1-dev' and then applied as a minor release to the HSDS spec (giving us version 1.1.1).

However, this could affect implementers in two ways:

• (1) If you are generating your own datapackage.json file along with your published data, you would need to update it the datapackage.json template or generation you use, to create 1.1.1 data;

• (2) If you have tools that read the datapackage.json, and are not expecting arrays for primaryKey and foreignKeys[].fields properties, you would need to adapt these tools to deal with 1.1.1

Given the Table Schema spec allows that the target of primaryKey and foreignKeys[].fields could be strings or arrays, and tooling based off the frictionless data specs should in theory already support arrays.

However, if there are implementers who feel they would be affected by this change because their tools rely on the structure of incoming datapackage.json files, and don't support arrays, then we could look at making this a decimal or integer release.

@kinlane @greggish (+all) I'd welcome your feedback on whether this has any impact on the API work, and views on whether we can make this a quick bug-fix or not.

[timgdavies]

This has been merged.