Clarify/implement forbiddenness of negative `soma_joinid` in the spec

[johnkerl]

Tracking issue for https://github.com/single-cell-data/SOMA/pull/119#discussion_r1102882763 in the event that feedback needs addressing in a separate PR.

See also https://github.com/single-cell-data/TileDB-SOMA/issues/519.

---

Analysis results in support of https://github.com/single-cell-data/SOMA/pull/124 and https://github.com/single-cell-data/TileDB-SOMA/pull/932:

• Spec:
  • The spec language is almost correct: it says Numeric index types (eg, offset indexing into dense arrays) are specified with int64 type and a domain of [0, 2^63-1). In other words, positive int64 values are used for offset indexing. All this needs is the typo positive to be changed to non-negative: https://github.com/single-cell-data/SOMA/pull/124
  • The spec docstring info could use to make this explicit: also on https://github.com/single-cell-data/SOMA/pull/124
• TileDB-SOMA Python impl:
  • Currently allows negative soma_joinid, contrary to the spec: fixed by https://github.com/single-cell-data/TileDB-SOMA/pull/932
• TileDB-SOMA R impl:
  • Is already correct:
  • https://github.com/single-cell-data/TileDB-SOMA/blob/0.5.0a11/apis/r/R/SOMADataFrame.R#L56-L64
  • https://github.com/single-cell-data/TileDB-SOMA/blob/0.5.0a11/apis/r/R/utils-arrow.R#L130-L142
  • Further proof it's already correct:
  • https://gist.github.com/johnkerl/547d13fb3782fa71cf50c37136e585d9
  • Copied from https://github.com/single-cell-data/TileDB-SOMA/blob/0.5.0a11/apis/r/tests/testthat/test_SOMADataFrame.R#L2
  • Array-schema output added as a comment on that gist reveals domain was written in conformance with the spec: Dim(name='soma_joinid', domain=(0, 9223372036854775807), ...)

[thetorpedodog]

For future consideration:

Since we allow floating-point and string dimensions, it would make sense to me that we should allow users to store data with a domain that goes into the negative numbers in a DataFrame.

To be clear, I do not think this is a change we would need to make right now or even necessarily soon, just a thought for future development. I am fine with saying “no negative numbers” in the current state.

[bkmartinjr]

the "no negative numbers" is a misunderstanding - it was "no negative soma_joinid", a statement that was necessary because soma_joinid is always int64.

So this affects only:

• NDArray dimensions/coordinates
• the soma_joinid of DataFrame (regardless of whether it is an index).

I don't recall that we ever discussed (much less concluded) that negative values could not be indexed for user-defined columns. I feel pretty strongly that we need the ability to index the full domain of any user-defined column.

[johnkerl]

Thanks for the clarify, @bkmartinjr !!