GraphBLAS / binsparse-specification

A cross-platform binary storage format for sparse data, particularly sparse matrices.
https://graphblas.org/binsparse-specification/
BSD 3-Clause "New" or "Revised" License
15 stars 4 forks source link

Consider typing the sparse array itself, rather than the underlying data arrays #24

Closed willow-ahrens closed 1 year ago

willow-ahrens commented 1 year ago

Instead of defining the types of underlying in-memory arrays, e.g. pointers_1_type and indices_0_type, consider defining the types of the in-memory sparse array as a whole. Consider the following spec:

The index_types, position_type, and value_type keys must be present and shall define the data types of all required arrays based on the [[#key_format]]. The types declare the type of the in-memory arrays. While these are often identical to the types used when storing the arrays on disk in the container, the container may choose to store the arrays in another format. For example, uint64 type may be stored as int8 if all the numbers in the array are small enough to fit, but position_type would still list the array as having type uint64.

The index_types shall define the types of in-memory arrays that hold indices. It shall be an array [i_type_0, ..., i_type_(N-1)] where i_type_r is the type of r^th index to the array (regardless of whether the array is row-major or column major).

The position_type shall define the type of in-memory arrays that hold positions.

The value_type shall define the type of in-memory arrays that hold the array elements themselves.

willow-ahrens commented 1 year ago

I'm closing this for now, we can bring this back later if it becomes useful.