Closed saulshanabrook closed 6 years ago
When running the tests, only test_boxes_unboxes
passes. Other tests fail with stopping Python with error message:
LLVM ERROR: Program used external function 'ndt_as_ndarray' which could not be resolved!
SOLUTION: ndt_as_ndarray
is defined in libndtypes
, adding the following code to libndtypes.py fixes the LLVM ERROR:
import llvmlite
llvmlite.binding.load_library_permanently('libndtypes.so')
All tests pass now.
I would like to give a try to replace
ndt_t = ir.ArrayType(char, sizes.SIZEOF_NDT_T)
with
ndt_t = ir.IntType(sizes.SIZEOF_NDT_T)
so that ndt_t
would become single value type for llvm.
RESULT: It works, almost, I could retrieve the value of ndim (using integer binary operations) and print it inside njit function, but then python segfaults. Another problem is that I could not figure out how to reference a integer value needed for passing arguments to ndt_as_ndarray
.
Getting struct values from a wide integer requires two binary operations (and_ and ashr, it could be optimized to one when using trunc) between wide integers. In the case of ndt_ndarray_t the bitwidth is 3088, and I am not sure if this can be faster or slower compared to using bitcast approach.
SOLUTION: ndt_as_ndarray is defined in libndtypes, adding the following code to libndtypes.py fixes the LLVM ERROR:
Good call. I added this but also loading 'libndtypes.dylib'
on mac, so it works for me. I also added CI, so that we can catch these errors more easily.
so that ndt_t would become single value type for llvm.
Is it that you would think this would generate more performant code? If we switched, I think it would be ndt_t = ir.IntType(8 * sizes.SIZEOF_NDT_T)
not ndt_t = ir.IntType(sizes.SIZEOF_NDT_T)
, since IntType
takes the number of bits and sizeof
returns the number of char
s/bytes.
Getting struct values from a wide integer requires two binary operations (and_ and ashr, it could be optimized to one when using trunc) between wide integers.
When I think of implementing it this way, I wouldn't ever operate on the integers themselves, just the pointers. So if we have a pointer to an ndt_t
, we just offset that pointer by the right amount of bytes and then call that a pointer to an ndt_ndarray_t
.
I think it would be ndt_t = ir.IntType(8 * sizes.SIZEOF_NDT_T)
Yes, that is correct. This also resolved python segfaulting. Thanks!
Do you know how to get a reference of a value using IR? This is needed in order to use IntegerModel and passing the values to ndt_as_ndarray.
Btw, when using PrimitiveModel, ir.ArrayType and ir.IntType are equivalent performance-wise.
In order to get ndim, for instance, it seems wasteful to create a new ndarray object.
In order to get ndim, for instance, it seems wasteful to create a new ndarray object.
Yes you are totally right about that. I am updating it now to just grab the ndim off the original ndt_t
object instead of creating the ndarray
. I missed that it had it on there.
OK I fixed ndim to not create the ndarray struct. I also split the files into directories to hopefully make it easier to navigate the code.
Do you know how to get a reference of a value using IR? This is needed in order to use IntegerModel and passing the values to ndt_as_ndarray.
I don't think this is possible. src:
A pointer points to memory. If not stored, a value may live in a register. So you really need the alloca.
This addresses #1 and #2 by demonstrating a way to structure this repo, that separates high level/exposed python lowering from low level/lib* lowering.
The aim was to allow users to
@jit
a function and use the same Python API that they are used to for the plures projects. But, we also expose another lower level API, that we use internally (and users could use as well), to implement higher level functions. This mirrors the underlying C header files. The goal here is to keep us sane, by mirroring the existing names and functions in the C libraries.CONST_NDT
as casting fnNdt_FromType
as casting fnCreate C function to create ndt_array_texpose as python function create_ndt_array_tCreate C function to get shape of ndt_array_texpose as python attribute of NdtArrayTypeshape
attribute ofPyNdtType