[python] ExperimentDataPipe: configurable `method` (`nd.array`, `scipy.coo`, `scipy.csr`)

[ryan-williams]

Introduce a new method ("np.array") for converting (COO) tiledbsoma.SparseNDArray data to (dense) torch.Tensor.

Comparison vs. existing ("scipy.csr") method:

Code/data here:

• For a fixed "shuffle block" size $B$ rows(131,072, in the figure above):
• For SOMA chunks of size $C$ rows (64, 128, …, 131,072):
• $B/C$ chunks are read from disk, concatenated, and shuffled
• PyTorch batches of size 1,024 are read from the shuffled block, and converted to torch.Tensors.

scipy.csr

Convert arrow.Table to scipy.sparse.csr_matrix (source). This is the current behavior.

np.array

Directly convert arrow.Table to np.array (source).

This method is new here, and seems to offer more speed at the cost of using more memory. It brings SOMA chunks into memory as dense np.arrays.