pydata / sparse

Sparse multi-dimensional arrays for the PyData ecosystem
https://sparse.pydata.org
BSD 3-Clause "New" or "Revised" License
602 stars 126 forks source link

Add SOA level property and change COO format once in LLVM #797

Closed hameerabbasi closed 2 weeks ago

hameerabbasi commented 1 month ago
          Once we update to LLVM 20/nightly builds we will change COO format to SoA (Structure of Arrays) calling convention, as it fixes some issues for us [link](https://discourse.llvm.org/t/passmanager-fails-on-simple-coo-addition-example/81247/2?u=mtsokol).
#COO = #sparse_tensor.encoding<{
    map = (i, j) -> (i : compressed(nonunique), j : singleton(soa)), posWidth = 64, crdWidth = 64
}>
#COO_3D = #sparse_tensor.encoding<{
    map = (i, j, k) -> (i : compressed(nonunique), j : singleton(soa, nonunique), k : singleton(soa)), posWidth = 64, crdWidth = 64
}>
#COO_4D = #sparse_tensor.encoding<{
    map = (i, j, k, l) -> (i : compressed(nonunique), j : singleton(soa, nonunique), k : singleton(soa, nonunique), l : singleton(soa)), posWidth = 64, crdWidth = 64
}>

Then instead of:

fields.append((f"indices_{compressed_counter}", get_nd_memref_descr(2, idx_dtype)))

we would have:

fields.append((f"indices_coo_dim_0_{compressed_counter}", get_nd_memref_descr(1, idx_dtype)))
fields.append((f"indices_coo_dim_1_{compressed_counter}", get_nd_memref_descr(1, idx_dtype)))
coo_dim_counter = 2
...
if LevelFormat.Singleton == level.format:
    fields.append((f"indices_coo_dim_{coo_dim_counter}_{compressed_counter}", get_nd_memref_descr(1, idx_dtype)))
    coo_dim_counter += 1

Does it make sense?

But this will be addressed separately once we can build with nightlies.

_Originally posted by @mtsokol in https://github.com/pydata/sparse/pull/792#discussion_r1810682069_