Closed dtemkin-volpe closed 1 month ago
From what I can see, it looks good. As a test, I'd like to run the workflows. Something like:
Test #1: change the ID from text to integer. See what happens to the markdown and generated sql. What field type for ID appears in the markdown (ID to match the json, or would it be integer or test?)
Test #2: change something else. Make sure we have not broken what we have done before
Finally, in our Cambridge Intersection example, double check that the field_type in the config file is consistent with the ID field types in the actual data.
I think the idea was that people could define what types the id
fields would be in their config.csv file, rather than in the json spec files (so the markdown documentation or generated SQL wouldn't change). Because the config.csv file isn't really required, the id
fields are defined as an any type in the spec, and downstream software could then enforce whatever is set in the config file. I enabled the workflow on my end, let me try changing something and make sure everything works still.
Yes, that makes sense. Thank you. Maybe we can create two versions of the template SQLite database, one with INT keys and the other with TEXT keys.
I did a little test, and all seems to be well. Diego, I'm ok with merging if you feel it is ready. Thanks.
Sorry for the big pull request! This request has a few changes, including:
fieldsMatch
entry to allow for end-users to have additional columns (eg. for relevant notes), clarify that every resource is a table, and add$schema
(which replacesprofile
), and changes the enum constraints to use the newcategories
field (this makes the documentation easier to read, but changes the DB's schema from usingVARCHAR
toTEXT
, so feedback on whether I should change this back would be appreciated)frictionless-py
package is updated soon to take advantage of the new features, but for right now, everything is (mostly) backwards compatible.missingValues
field for every table, mainly for CSV processors to recognize that an empty column is a null value.id_type
entry in theconfig
table, that must either be"string"
or"integer"
. This would tell any software using GMNS how to interpret the id fields in other tables.Let me know if there are any issues with this or if anyone wants any changes.