Closed vgokhale closed 8 months ago
The upstream kernel had some optimizations, specifically for splitting the causal mask across two calls so only one needs to incur the warp divergence penalty. Performance details here.
The upstream kernel had some optimizations, specifically for splitting the causal mask across two calls so only one needs to incur the warp divergence penalty. Performance details here.