Change memory object USM allocation ownership, and make execution asynchronous

[oleksandr-pavlyk]

This is PR changes the size and content of the struct behind dpctl.memory._Memory object.

This is backwards incompatible change, and would require rebuilding of downstream projects with native extensions (cython, or pybind11) that use dpctl.

The _Memory object no longer explicitly frees USM allocations it made, but delegates this job to a smart pointer. This allows host_task jobs that ensure deferment of USM deallocation till after offloaded tasks that operate on it complete execution to do their job by capturing the smart pointer in the callable passed to the host_task instead needing to rely on Python reference counting, and thus need to acquire GIL in that callable.

Furthermore, the PR introduces mechanism of ensuring sequential order amongst tasks implementing offloading Python API, such as dpctl.tensor implementing array API compliant tensor library.

One consequence of this change is that timing of dpctl.tensor functions must synchronize the execution queue before taking the final timestamp reading.

• [x] Have you provided a meaningful PR description?
• [ ] Have you added a test, reproducer or referred to an issue with a reproducer?
• [x] Have you tested your changes locally for CPU and GPU devices?
• [x] Have you made sure that new changes do not introduce compiler warnings?
• [x] Have you checked performance impact of proposed changes?
• [ ] If this PR is a work in progress, are you opening the PR as a draft?

[github-actions[bot]]

Deleted rendered PR docs from intelpython.github.com/dpctl, latest should be updated shortly. :crossed_fingers:

[github-actions[bot]]

Array API standard conformance tests for dpctl=0.18.0dev0=py310h15de555_50 ran successfully. Passed: 890 Failed: 11 Skipped: 91

[coveralls]

coverage: 88.057% (+0.1%) from 87.911% when pulling f4e3b6f1b635ec1f0fb83b3146cc57055b4f3aff on memory-work into c6f5f790b963a8984d3a7760cc9358aa4efe3d65 on master.

[github-actions[bot]]

Array API standard conformance tests for dpctl=0.18.0dev0=py310h15de555_51 ran successfully. Passed: 890 Failed: 11 Skipped: 91

[github-actions[bot]]

Array API standard conformance tests for dpctl=0.18.0dev0=py310h15de555_51 ran successfully. Passed: 890 Failed: 11 Skipped: 91

[github-actions[bot]]

Array API standard conformance tests for dpctl=0.18.0dev0=py310h15de555_53 ran successfully. Passed: 890 Failed: 11 Skipped: 91

[github-actions[bot]]

Array API standard conformance tests for dpctl=0.18.0dev0=py310h15de555_55 ran successfully. Passed: 890 Failed: 11 Skipped: 91

[oleksandr-pavlyk]

What's missing from this PR that must be addressed before merging:

• [x] dpctl_capi documentation must be expanded with new CAPI added, one for usm_ndarray, and one for usm_memory
• [ ] Execution model explanation in documentation must now reflect that execution is asynchronous
• [x] Examples of dpctl for pybind11 and cython must be updated.

[oleksandr-pavlyk]

I suggest we expand the documentation with explaining the asynchronous execution in a separate PR.

The important change to realize is that execution of tensor operations is done asynchronously, so to get a proper timing values, one must use queue.wait() before taking the final time stamp reading.

Submission preserves sequential ordering of operations as executed by Python interpreter.

[github-actions[bot]]

Array API standard conformance tests for dpctl=0.18.0dev0=py310h15de555_57 ran successfully. Passed: 890 Failed: 11 Skipped: 91

[github-actions[bot]]

Array API standard conformance tests for dpctl=0.18.0dev0=py310h15de555_57 ran successfully. Passed: 890 Failed: 11 Skipped: 91

[github-actions[bot]]

Array API standard conformance tests for dpctl=0.18.0dev0=py310h15de555_58 ran successfully. Passed: 890 Failed: 11 Skipped: 91

[github-actions[bot]]

Array API standard conformance tests for dpctl=0.18.0dev0=py310h15de555_59 ran successfully. Passed: 890 Failed: 11 Skipped: 91

[github-actions[bot]]

Array API standard conformance tests for dpctl=0.18.0dev0=py310h15de555_60 ran successfully. Passed: 889 Failed: 12 Skipped: 91