tracking issue: array API standard document

[rgommers]

Timeline (high-level)

• Aug 31: array RFC draft complete enough for final internal review
• Sep 15: array RFC published + array-api repo public

Array API standard document intermediate steps:

Aug 10:

• [x] Broadcasting (status: merged)
• [x] Mutability (status: pending discussion, see https://github.com/data-apis/array-api/issues/24)
• [x] Dtypes and casting (status: merged)
• [x] API signatures: statistical reductions (status: merged)

Aug 17:

• [x] Purpose and scope (status: scope part (gh-22) merged, "how to adopt" (gh-33) under review)
• [x] Type signatures (status: merged)
• [x] Static typing (status: gh-62 for review)
• [x] Indexing (status: under review)
• [x] API signatures: linear algebra (status: first set of functions merged, others like svd and qr TODO)
• [x] API signatures: sorting (status: under review)
• [x] API signatures: searching (status: merged)
• [x] Accuracy (status: under review)
• [x] Methodology: data on existing API design & usage (status: under review)

Aug 24:

• [x] ~Array duck typing~
• [ ] API signatures: array creation (status: gh-35 merged, a few functions (mainly meshgrid) to be added)
• [x] API signatures: array manipulation (status: under review)
• [x] API signatures: set functions (status: under review)
• [x] API signatures: logical functions (status: merged)
• [x] C API (status: WIP content in issue gh-51)
• [x] Use cases (status: merged)
• [x] Assumptions (status: merged)
• [x] API signatures: math, non element-wise

Aug 31:

• [x] Array object (status: gh-53 for review)
• [x] Future API standard evolution (status: under review, see gh-37; for the actual process see https://github.com/data-apis/workgroup/issues/22)
• [x] Parallelism (status: gh-61 for review)
• [x] Data interchange mechanisms (see gh-34)
• [x] Verification - test suite (status: merged)
• [x] API: constants (status: merged)
• [x] versioning (status: under review, see gh-37)

Added later:

• [x] Python operators, and equivalent dunder methods and library functions/methods (see gh-9)
• [ ] Device support (see gh-39)

[rgommers]

Current status:

There are 11 open PRs that are either (close to) ready for merging or that need input on specific choices to be made. I'll comment only on the ones that need action:

• ~gh-25 (set functions): needs a decision on sort order for unique. In particular input from TensorFlow needed (@alextp).~ update: merged, opened gh-40 for the TODO
• gh-19 (sort functions): needs update from @kgryte on stable sorting decision.
• ~gh-22 (scope section): needs an update from @rgommers for the TBD about device support.~
• ~gh-33 ("how to adopt the API" section): needs review~
• ~gh-34 (data interchange / DLPack): needs review~
• ~gh-29 (accuracy): needs completing, and input on how to specify accuracy for what's outside of the IEEE-754 spec.~
• ~gh-18 (tooling / methodology): needs a bit more detail on array-api-comparison and how the data produced by both tools is used for decision making.~

Still TODO:

• [ ] second set of linalg functions
• [x] second set of array creation & manipulation functions
• [x] constants
• [x] PRs for topics for which content is being discussed in an issue:
  • [x] mutability (see gh-66)
  • [x] static typing (partly addressed in section Function and method signatures)
  • [x] parallelism (see gh-4)
  • [x] future API standard evolution & versioning
• [ ] Topics that we don't yet have content for:
  • [x] Indexing (likely complex)
  • [x] Array object (should be straightforward)
  • [x] Purpose (take from announcement blog post)
  • [x] C API (see gh-51)
• [ ] Check what's in array libraries but not in the standard - did we leave out anything we shouldn't have? --> see gh-43

[rgommers]

A text search for "TODO" on master:

``` $ grin -C2 TODO ./API_specification/index.rst: 28 - .. _api-versioning: 29 - 30 : TODO: add `__array_api_version__` (or similar) version here, and the way to 31 + retrieve it - see https://github.com/data-apis/workgroup/issues/24 ./API_specification/set_functions.md: 37 - - If `True`, the function must sort the unique elements in ascending order before returning as output. If `False`, the function must sort the unique elements in the same order that they occur in `x`. Default: `False`. 38 - 39 : _TODO: sort order needs discussion. See [gh-40](https://github.com/data-apis/array-api/issues/40)_ 40 + 41 + #### Returns ./design_topics/data_interchange.md: 57 - 58 - 59 : TODO: design an appropriate Python API for DLPACK (`to_dlpack` followed by `from_dlpack` is a little clunky, we'd like it to work more like the buffer protocol does on CPU, with a single constructor function). 60 + 61 : TODO: specify the expected behaviour with copy/view/move/shared-memory semantics in detail. 62 + 63 + ./design_topics/device_support.md: 3 - # Device support 4 - 5 : TODO. See https://github.com/data-apis/array-api/issues/39 6 + ./future_API_evolution.md: 6 - the process documented at: 7 - 8 : TODO: link to specific document in https://github.com/data-apis/governance once 9 + it is available. 10 + ./purpose_and_scope.md: 42 - 43 - The concrete set of functionality that is in scope for this version of the 44 : standard is shown in this diagram (_TODO: update after deciding on how optional 45 + extensions are dealt with_): 46 + ./verification_test_suite.md: 5 - ## Measuring conformance 6 - 7 : TODO: explain how to use the test suite to measure and report on the level of 8 + (non-)conformance. ```

EDIT: some of these are already decided on and can easily be fixed up. The important ones that need significant work are:

• DLPACK: design a new Python API, specify move/copy/shared-mem semantics better
• device support
• how to deal with optional extensions

EDIT 2: down to:

$ grin -C2 TODO
./API_specification/set_functions.md:
   37 -     -   If `True`, the function must sort the unique elements in ascending order before returning as output. If `False`, the function must sort the unique elements in the same order that they occur in `x`. Default: `False`.
   38 - 
   39 :         _TODO: sort order needs discussion. See [gh-40](https://github.com/data-apis/array-api/issues/40)_
   40 + 
   41 + #### Returns
./design_topics/data_interchange.md:
   57 - 
   58 - 
   59 : TODO: design an appropriate Python API for DLPACK (`to_dlpack` followed by `from_dlpack` is a little clunky, we'd like it to work more like the buffer protocol does on CPU, with a single constructor function).
   60 + 
   61 : TODO: specify the expected behaviour with copy/view/move/shared-memory semantics in detail.
   62 + 
   63 + 
./design_topics/device_support.md:
    3 - # Device support
    4 - 
    5 : TODO. See https://github.com/data-apis/array-api/issues/39
    6 + 
./purpose_and_scope.md:
   42 - 
   43 - The concrete set of functionality that is in scope for this version of the
   44 : standard is shown in this diagram (_TODO: update after deciding on how optional
   45 + extensions are dealt with_):
   46 + 

[rgommers]

I think we're good here, all PRs are merged and there are TODO comments for the things that are left to do. No need for this tracking issue anymore.