Currently, to protect from misaligned addresses on device, every allocation is padded to 8 bytes in the pool allocator. This would mitigate a lot of the device memory bandwidth benefits from running in single precision. A potential fix could be to only pad those allocations that are not multiples of nproma, as these are for all intents and purposes guaranteed to be multiples of 8.
Currently, to protect from misaligned addresses on device, every allocation is padded to 8 bytes in the pool allocator. This would mitigate a lot of the device memory bandwidth benefits from running in single precision. A potential fix could be to only pad those allocations that are not multiples of nproma, as these are for all intents and purposes guaranteed to be multiples of 8.