search

pydata / sparse

Sparse multi-dimensional arrays for the PyData ecosystem
https://sparse.pydata.org
BSD 3-Clause "New" or "Revised" License
581 stars 123 forks source link

MNT: `sparse` import refactor #695

Closed mtsokol closed 1 month ago

mtsokol commented 1 month ago

Hi @hameerabbasi,

This PR refactors sparse importing mechanism. It enforces backend selection at the import time and allows from sparse import COO etc. imports.

github-actions[bot] commented 1 month ago

Test Results

44 tests   - 6 063   31 :white_check_mark:  - 5 954   37s :stopwatch: - 10m 3s  1 suites ±    0   13 :zzz:  -   109   1 files   ±    0    0 :x: ±    0 

Results for commit 2dab3d22. ± Comparison against base commit 38d1a67c.

This pull request removes 6107 and adds 44 tests. Note that renamed tests count towards both. ``` sparse.numba_backend.tests.test_array_function ‑ test_binary[dot-arg_order0] sparse.numba_backend.tests.test_array_function ‑ test_binary[dot-arg_order1] sparse.numba_backend.tests.test_array_function ‑ test_binary[dot-arg_order2] sparse.numba_backend.tests.test_array_function ‑ test_binary[matmul-arg_order0] sparse.numba_backend.tests.test_array_function ‑ test_binary[matmul-arg_order1] sparse.numba_backend.tests.test_array_function ‑ test_binary[matmul-arg_order2] sparse.numba_backend.tests.test_array_function ‑ test_binary[result_type-arg_order0] sparse.numba_backend.tests.test_array_function ‑ test_binary[result_type-arg_order1] sparse.numba_backend.tests.test_array_function ‑ test_binary[result_type-arg_order2] sparse.numba_backend.tests.test_array_function ‑ test_binary[tensordot-arg_order0] … ``` ``` sparse.tests.test_backends ‑ test_asarray[coo-C] sparse.tests.test_backends ‑ test_asarray[coo-F] sparse.tests.test_backends ‑ test_asarray[csc-F] sparse.tests.test_backends ‑ test_asarray[csr-C] sparse.tests.test_backends ‑ test_backends sparse.tests.test_backends ‑ test_finch_lazy_backend sparse.tests.test_backends ‑ test_scikit_learn_dispatch[csc_matrix-csc-F] sparse.tests.test_backends ‑ test_scipy_breadth_first_tree[coo_matrix-coo-F] sparse.tests.test_backends ‑ test_scipy_breadth_first_tree[csc_matrix-csc-F] sparse.tests.test_backends ‑ test_scipy_breadth_first_tree[csr_matrix-csr-C] … ```
This pull request removes 122 skipped tests and adds 13 skipped tests. Note that renamed tests count towards both. ``` sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[f4-axis4-sum-kwargs0] sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[f8-axis4-sum-kwargs0] sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-0-mean-kwargs1] sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-0-sum-kwargs0] sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-1-mean-kwargs1] sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-1-sum-kwargs0] sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-axis4-mean-kwargs1] sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i4-axis4-sum-kwargs0] sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i8-0-mean-kwargs1] sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i8-0-sum-kwargs0] … ``` ``` sparse.tests.test_backends ‑ test_scikit_learn_dispatch[csc_matrix-csc-F] sparse.tests.test_backends ‑ test_scipy_eigs[coo-C] sparse.tests.test_backends ‑ test_scipy_eigs[coo-F] sparse.tests.test_backends ‑ test_scipy_eigs[csc-F] sparse.tests.test_backends ‑ test_scipy_eigs[csr-C] sparse.tests.test_backends ‑ test_scipy_lsqr[coo-C] sparse.tests.test_backends ‑ test_scipy_lsqr[coo-F] sparse.tests.test_backends ‑ test_scipy_lsqr[csc-F] sparse.tests.test_backends ‑ test_scipy_lsqr[csr-C] sparse.tests.test_backends ‑ test_scipy_norm[coo-C] … ```

:recycle: This comment has been updated with latest results.

mtsokol commented 1 month ago

Maybe we can run the test suite twice, once for Finch and once for Numba in CI, without doing the importlib.reload(...) magic? And provide a ./test.sh script?

Done! I run tests twice in the CI with different backends:

- name: Run tests
  run: |
    SPARSE_BACKEND=Numba pytest --pyargs sparse
    SPARSE_BACKEND=Finch pytest --pyargs sparse/tests

For Numba we run all tests. For Finch only the common ones.