openspending / openspending-migrate

0 stars 1 forks source link

Inferring JSON schema #4

Open danfowler opened 8 years ago

danfowler commented 8 years ago

In the initial pass at this, @trickvi mentioned:

Should we infer the schema for each resource (i.e. should we use the fields.type etc. in DP's schema), it might lead to false positives, I propose we don't use anything except fields.name

In creating the datapackage.json, I'm currently inferring schemas of the generated dataset.csv. With the large clean corpus, this could serve as a good test case for the JTS libraries. However, this also could obviously lead to errors. As an alternative, we can pull the types from the mapping object in the generated dataset.json.