tgujar
commented
7 months ago Since option 1 doesnt incur the cost of JIT compilation maybe this is the better approach in terms of performance. My current plan is to arrange the types in increasing order of register usage and split the large switch in type_dispatcher into reasonable chunks. We activate a chunk using branching at CPU, and dispatching the appropriate compile time cond based on type. What do you think about this? This does tie the implementation of the comparator/hasher for a type to the granularity and split of the chunks, but maybe this is okay? Essentially, the compile time conditional would be like has_nested_column but more granular. we can abstract out the CPU side branching for creating the compile time cond into a function which can be used in places where we need to construct a device row hasher/comparator.
if constexpr(cond) {
switch(...) {
...
}
}
. // repeat for chunks
.
.
davidwendt
sleeepyjack
vyasr
Is your feature request related to a problem? Please describe. cuco insert kernel has poor occupancy due to high register usage during hash table build operation executed by cuDF. If I disable some of the code paths for complex types(commenting out dict, string, list, struct, decimal) in https://github.com/rapidsai/cudf/blob/434df44d9fe1c94e8047bcc37266ae663eae8a8d/cpp/include/cudf/utilities/type_dispatcher.hpp#L456 the type dispatcher, then the register usage per thread drops from 75 -> 46 and leads to a significant occupancy bump. It seems that the insert kernel has to pay the cost of high register usage even for simpler types since the compiler has to account for all code paths.
I did some experiments by disabling different subsets of types, list has types I disable -> register count for insert kernel
Here is the speedup I see on mixed semi join kernel by improving occupancy for int32 keys obtained by disabling complex types
Describe the solution you'd like Improve occupancy by disabling codepaths for complex types.
Describe alternatives you've considered
Additional context Add any other context, code examples, or references to existing implementations about the feature request here.