🚀💻 – Reconsider how to repesent the list of schemas within TIDES suite

[e-lo]

User Story

As a developer and user, I want to leverage standardized functionality for listing packages of files and the interplay between them so that it is:

• maintainable and easily usable with existing toolsets (read: not maintained by us), and
• understandable to others leveraging similar standards without having to do a lot of explaining.

Context

• tides.spec.json currently lists all the schemas as frictionless resources, but it is not, in and of itself, conformant to a schema within the realm of frictionless.
• tides.spec.json is currently used in auto-generating the documentation, but this could probably be updated to just pull in all the schema files.
• The tabular-data-package definition requires an additional file be shipped with each set of data (datapackage.json) which defines a set of resources –  which files exist and what schemas they use in addition to metadata.
• For the purposes of defining how a datapackage.json can be structured outside the bounds of a tabular-data-package, Frictionless uses the concept of profiles, which are a definition of the json-schema for datapackage.json.

Solutions

Alternatives

Options so far:

1. do nothing, keep tides.spec.json
2. translate tides.spec.json to a profile (see example of how this was done for camera trap data)
3. Remove tides.spec.json and autogenerate documentation from anything in the /spec folder
4. ?? bueller ??

Additional Context

[botanize]

I'd like to add an option:

4. remove tides.spec.json, it contains no information not already available in the table schemas, autogenerate table on architecture page from *.schema.json files.

e.g.,

spec_summary = []
for s in schema_files:
    schema = read_schema(s)
    spec_summary.append({"Name": schema.get('name').split(".")[0], "Type": schema.get('_table_type'), "Description": schema.get('description')})

spec_summary.sort(key=lambda schema: schema["Name"])

md = "## Files in Specification\nTIDES consists of a package of files as defined in the following table.\n\n"
md += "| Name | Type | Description |\n|-|-|-|\n"
for table_type in TABLE_TYPES:
    for schema in spec_summary:
        if schema["Type"] == table_type:
            md += "| {Name} | {Type} | {Description} |\n".format(**schema)

[e-lo]

I'd like to add an option: remove tides.spec.json, it contains no information not already available in the table schemas, autogenerate table on architecture page from *.schema.json files.

Added this one to consideration.

Thinking points 🤔

1. Are any of the files conditionally required or forbidden if another is present? We'd want to maintain a way to represent this.
2. Are there any other relationships, conditions, or checks that we need to represent "together" as opposed to being able to represent them within individual file schemas?