Investigate different Python interfaces and pick one

[unzvfu]

Consider the various different possible ways to make cuda-fixnum accessible within Python. Describe requirements of such an interface (e.g. easy integration with NumPy).

[unzvfu]

Analysis of candidate libraries:

• (NumPy: Doesn't support GPUs, but data will often arrive as a NumPy array, so we need to play nicely with it.)
• Numba: Good candidate.
  • Supports JIT compilation of functions operating on Numba arrays.
    • easy to extend cuda-fixnum in client code
  • Good integration with NumPy
  • Automatically serialises functions for distributed computing; plays well with Dask.
  • Ongoing work to support the GOAI DataFrame API
• PyTorch:
  • Easy integration with PyTorch's ML functionality.
  • Need to write C++ in terms of the ATen library's at::Tensor type.
  • There is at least one example of someone managing to make PyTorch and Numba play nicely together.
• ~Theano~: No longer maintained; developers recommend PyTorch instead.
• ~Dask~: Too high-level, like Spark for Python; requires plugging in a lower-level NumPy-compatible array such as CuPy or (maybe) Numba.
• ~PyGPU~:
  • Interface to libgpuarray, which is part of Theano.
  • Appears maintenance stopped with Theano.
• ~CuPy~:
  • Good NumPy compatibility.
  • Very clean interface.
  • But GPU kernels are expressed as raw strings. :nauseated_face:
• ~PyCuda~:
  • Good NumPy compatibility.
  • But GPU kernels are expressed as raw strings. :nauseated_face:

[unzvfu]

Ideally we'd write stuff like:

A = array([a, b, c], dtype=modnum(modulus))
B = array([d, e, f], dtype=modnum(modulus))
C = A + B

and the arrays would be moved to/located on the device, and the addition would be modulo the modulus specified in the type. Probably we'll have to do something slightly less elegant though. For example, to do the above in NumPy, I believe we'd have to subclass ndarray and also subclass the relevant ufuncs. There doesn't appear to be a way to simply specify a new dtype which redirects the ufunc calls.