pydata / sparse

Sparse multi-dimensional arrays for the PyData ecosystem
https://sparse.pydata.org
BSD 3-Clause "New" or "Revised" License
581 stars 123 forks source link

CI: Add `array_api_tests` CI job #668

Closed mtsokol closed 3 months ago

mtsokol commented 3 months ago

Hi @hameerabbasi,

This PR adds array_api_tests CI job that runs array_api_tests/test_signatures.py test suite against Finch backend.

array-api-skips.txt file contains all currently unimplemented functions.

hameerabbasi commented 3 months ago

Is there a way to ensure that if we add new functionality, we HAVE to remove the corresponding lines from the skips file? Basically treat them as xfail and use this: scrict xfails

github-actions[bot] commented 3 months ago

Test Results

5 923 tests  ±0   5 891 :white_check_mark: +1   6m 52s :stopwatch: - 5m 9s     1 suites ±0      32 :zzz:  - 1      1 files   ±0       0 :x: ±0 

Results for commit 61705357. ± Comparison against base commit 0308ed0b.

This pull request skips 2 and un-skips 3 tests. ``` sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[f4-1-sum-kwargs0] sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i8-None-sum-kwargs0] ``` ``` sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[f8-None-sum-kwargs0] sparse.numba_backend.tests.test_coo ‑ test_reductions_float16[i4-None-sum-kwargs0] sparse.numba_backend.tests.test_coo ‑ test_reductions_float16[i8-None-sum-kwargs0] ```

:recycle: This comment has been updated with latest results.

mtsokol commented 3 months ago

I don't think array-api-tests supports it. Skips file just defines which functions are missing. We did it this way in NumPy and we successfully got to 99% compatibility after a few months. One needs to remember to remove specific lines from this file.