JIT improvements

[pavanky]

place holder issue to track JIT improvements.

Convert to JIT:

• [ ] Reductions
• [ ] range and iota
• [x] select

New functions:

• [x] clamp

Behavior changes:

• [x] Compiling custom kernels
• [x] Better heuristics

CPU Specific changes:

• [x] Improve performance of TNJ
• [ ] Implement JIT using LLVM

CUDA changes

• [x] Move to libnvrtc
• [x] Improve nvvm performance
• [x] Fix launch configuration

[pavanky]

@arrayfire/core-devel any other suggestions we can think of ?

[ghost]

Is replace in JIT? Can you add batch mode to it (and select) too?

[pavanky]

@gaika Not sure if replace can be JIT'd but select can definitely be including batch mode.

Although both of them support "batch" mode right now also.

[ghost]

Not sure how batch is supported, because when I pass arrays with different dimensions I get:

ERROR: ArrayFire Error (203) : Invalid input size
In function af_err af_select(void**, af_array, af_array, af_array)
In file src/api/c/select.cpp:54
Invalid dimension for argument 1
Expected: cdims[i] == std::min(adims[i], bdims[i])

[pavanky]

@gaika can you tell me what the dimensions are for a, b, and c?

[ghost]

For example:

julia> select(a, b, c)
ERROR: ArrayFire Error (203) : Invalid input size
In function af_err af_select(void**, af_array, af_array, af_array)
In file src/api/c/select.cpp:46
Invalid dimension for argument 1
Expected: cinfo.ndims() == std::min(ainfo.ndims(), binfo.ndims())

Stacktrace:
 [1] _error(::UInt32) at /home/joe/.julia/v0.6/ArrayFire/src/util.jl:44
 [2] select(::ArrayFire.AFArray{Bool,2}, ::ArrayFire.AFArray{Int64,1}, ::ArrayFire.AFArray{Int64,2}) at /home/joe/.julia/v0.6/ArrayFir
e/src/wrap.jl:1068                                                                                                                   

julia> a
ArrayFire.AFArray{Bool,2}
1×3 Array{Bool,2}:
 true  false  true

julia> b
ArrayFire.AFArray{Int64,1}
1-element Array{Int64,1}:
 0

julia> c
ArrayFire.AFArray{Int64,2}
2×3 Array{Int64,2}:
 4  5  6
 7  8  9

[pavanky]

@gaika Please don't use a 1x1 array, they are highly inefficient compared to af::constant(val, dims);

On the note of batching, I see that the conditional arrays need to match one of the input dimensions for now. This can be fixed.