TACO does not run parallel

[mtghorbani]

Hello,

I have written MTTKRP, TTM, SpMV, and Tensor Hadamard Product THP kernels using the TACO library. In different versions of my code, I have used different data layouts for input and output tensors (from fully dense to fully sparse). The problem is I cannot get TACO running on multi-thread for any of my kernels and any of the data layouts (including the fully dense one). I checked it both by timing the computation (and single and 4 threads give me the same performance number) and checking the number of threads for the process running the code (which in all scenarios gives me 1).

My operating system is Ubuntu 22.04 LTS. I have built the version of TACO on the master branch with cmake -DCMAKE_BUILD_TYPE=Release -DOPENMP=ON .. command. I have also set the export TACO_CFLAGS=-O3 -fopenmp -ffast-math -std=c99 -march=native -mtune=native -ftree-vectorize I use Clang version 18 and the flags -std=c++17 -O3 -fopenmp -ffast-math -march=native -mtune=native -ftree-vectorize to compile the code. I have also run export OMP_NUM_THREADS=4 and export OMP_SCHEDULE="static".

This is a code snippet of MTTKRP and my timing:

long time = 0, start, end;
IndexVar i, j, k, l;
A(i,j) = B(i,k,l) * D(l,j) * C(k,j);
A.compile();
A.assemble();
start = duration_cast<microseconds>(system_clock::now().time_since_epoch()).count();
A.compute();
end = duration_cast<microseconds>(system_clock::now().time_since_epoch()).count();
time += end - start;
cout << time;

Initializations for the tensors have been done before the mentioned code.

I would appreciate it if you could help me with this problem.

[tongzhou80]

Hi, did you check the generated "compute" code? Is the outermost loop annotated with "#pragma parallel for"?