kose-y
commented
1 year ago Thanks for the report. How many threads were used?
I wrote matrix-vector multiplication three years ago, and there could be room for improvement. I will look into the GX part. Something that sounds counterintuitive to me is that if multiple threads work on the same tile, there could be some cache conflict, but that doesn't seem to be the case here. Let's check these parts in other environments.
When I read the code today, I see things I could have done differently. For example, recursive tiling strategies would have made code maintenance easier than the current fixed-size block approach (at the cost of slightly worse performance). The machinery based on Tullio.jl it is in the OpenADMIXTURE package's loop.jl file.
biona001
BrendonChau
For stochastic trace estimation, we require being able to evaluate terms like $\mathbf{G}^\top \mathbf{x}$ and $\mathbf{G}\mathbf{x}$ efficiently, where $\mathbf{G}$ is a SnpArray. I was looking into
SnpLinAlgand noticed that I was having alot of memory allocations, presumably due to use of JuliaBase.@threadsand pointer allocation from the tiling procedure. However, it turns out that only multithreading within a tile evaluation rather than multithreading over several tiles yields a substantial performance improvement.First the section defining the functions
Gt_mul_x!,Gt_mul_x_tiled!,G_mul_x!, andG_mul_x_tiled!And setting up data for simulation:
First, we can look at $\mathbf{G}^\top \mathbf{x}$,
then $\mathbf{Gx}$,
And finally, looking at the same operations using
SnpLinAlgFor $\mathbf{G}^\top \mathbf{x}$, tiling provides no improvement over just using
@turbo thread=true, when using SnpLinAlg, there is substantial overhead from using Base threads and tiled views. Whereas for $\mathbf{Gx}$ there is an over 5x performance improvement from using sequential tiling with limiting multithreading to a single tile computation. Even better, by not parallelizing over tiles, we avoid all heap allocations.I suspect we might see a similar improvement with $\mathbf{G}^\top \mathbf{X}_1$ and $\mathbf{GX}_2$ where $\mathbf{X}_1$ and $\mathbf{X}_2$ are matrices, but implementation of that might be very tedious dealing with tile remainder chunks.
I did these simulations on my M1 Max Macbook, please let me know if the improvement is comparable for other hardware. It is possible that for something like a 32-core Xeon processor on a server we would see different behavior.