Array Convention Discussion

[lomky]

A ticket to discuss the conventions surrounding Arrays.

Current Array Conventions. (n/a)

Array Fields:

gcis=# \d+ array
                                                        Table "gcis_metadata.array"
     Column     |        Type         | Modifiers | Storage  | Stats target |                         Description
----------------+---------------------+-----------+----------+--------------+--------------------------------------------------------------
 identifier     | character varying   | not null  | extended |              | A globally unique identifier for this array, such as a UUID.
 rows_in_header | integer             | default 0 | plain    |              | The number of rows in the header of this array.
 rows           | character varying[] |           | extended |              | The data in this array.
Indexes:
    "array_pkey" PRIMARY KEY, btree (identifier)
Check constraints:
    "array_dimensions" CHECK (array_ndims(rows) = 2)
    "ck_array_identifier" CHECK (identifier::text ~ similar_escape('[a-z0-9_-]+'::text, NULL::text))
Referenced by:
    TABLE "array_table_map" CONSTRAINT "array_table_map_array_identifier_fkey" FOREIGN KEY (array_identifier) REFERENCES "array"(identifier) ON UPDATE CASCADE ON DELETE CASCADE
Triggers:
    audit_trigger_row AFTER INSERT OR DELETE OR UPDATE ON "array" FOR EACH ROW EXECUTE PROCEDURE audit.if_modified_func('true')
    audit_trigger_stm AFTER TRUNCATE ON "array" FOR EACH STATEMENT EXECUTE PROCEDURE audit.if_modified_func('true')
    delpub BEFORE DELETE ON "array" FOR EACH ROW EXECUTE PROCEDURE delete_publication()
    updatepub BEFORE UPDATE ON "array" FOR EACH ROW WHEN (new.identifier::text <> old.identifier::text) EXECUTE PROCEDURE update_publication()

Provenance Connections:
none?
activity?

Relationships:

table

[lomky]

Array Conventions

Use Case

Hold the low level content of a Table entity.
Tables may contain multiple arrays.

If a table has multiple arrays, the table caption should briefly explain how the arrays relate.
Tables should note the units in the caption.

They are created whenever a table is created.

Header Values may be cleaned up for proper importing into GCIS.
Data values should not be changed. If they are for any reason, it should be noted in the Table caption.
If the provided raw data has seemingly extraneous data (i.e. totals not displayed in the published work), we reserve judgement on including them on a case by case basis. All data published in the final published Table or Graphic should be included.

We try not to upload CSVs to create arrays, but instead paste in well formatted data.

If the provided raw data is not in a CSV format, we reserve judgement on formatting them on a case by case basis.

Fields

Extant

identifier
Automatically created UUIDs.
In the case of multiple arrays per table, we may identifier the array with short, internally distinct identifiers the show which is which.
See the arrays under the Sea Surface Temp Indicator 2018 Table

rows_in_header
Usually one, indicates how many rows to be ignored as headers, non-data.

rows
The actual data, including headers, if any, stored as a database array.
Avoid special characters, if possible.

Provenance Conventions

prov:wasDerivedFrom

With very rare exceptions, should be used on the Array instead of the Tables.
Most commonly, Arrays are connected to a Dataset via an Activity.

cito:cites

Generally the Table object will be the thing using cito:cites. It is plausible to have a good use case for an array to use cito:cites.

Relationships

table
Arrays should always belong to a Table.
The relationship is generally added through the Table object.
If there are multiple arrays, add them in a sane order.

[lomky]

Metrics

Very Poor identifier

Acceptable identifier + rows_in_header + rows

Connections?

Should have an Activity+Publication provenance connection.
Multiple Arrays connection score should roll up together to their table.
In the case of a Table's Arrays all lacking the connection, we will check for a Table Activity+Publication provenance connection before scoring it to 0. - Same as Figure/Images.