sandbox search execute failed

[TUMSchieben]

Hi there, when I tried to run iree-llvm-sandbox, I got the following error:

python3 mmperf.py build/matmul results/haswell -sandbox -benchmark_path=/home/mmperf/benchmark_sizes/bert_large_matmul.txt Latest symlink path is: /home/mmperf/results/haswell/latest Latest results path is: /home/mmperf/results/haswell/2022-05-16_11-47-53-448597 Linux System Detected.. looking for /proc/cpuinfo

############################################################### Compile-time problem size {'m': 512, 'n': 1024, 'k': 1024} Runtime problem size {'m': 512, 'n': 1024, 'k': 1024} Problem types [<class 'numpy.float32'>, <class 'numpy.float32'>, <class 'numpy.float32'>]

Compilation expert SingleTiling2DPeel xxxxxxxxxx: Dialect: Einsum spec: km,kn->mn Compile time 0.06579804420471191 cannot dump ExecutionEngine object code to file: object cache is disabled xxxxxxxxxx : 100 iters time on 1 threads

 slowest          p1         p10         p25         p50         p75         p90         p99     fastest        unit

---

 2.7e-02     2.7e-02     2.6e-02     2.6e-02     2.5e-02     2.5e-02     2.5e-02     2.5e-02     2.5e-02     seconds
   40.15       40.42       41.60       42.08       42.28       42.40       42.62       43.06       43.06    GFlops/s
    0.39        0.39        0.41        0.41        0.41        0.41        0.42        0.42        0.42       GBs/s

Run time 2.809998035430908 /home/mmperf/external/iree-llvm-sandbox/python/examples/core/harness.py:94: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. self.data = self.data.append(product)

Compilation expert SingleTiling3DPeel xxxxxxxxxx: Dialect: Einsum spec: km,kn->mn Compile time 0.16719818115234375 cannot dump ExecutionEngine object code to file: object cache is disabled xxxxxxxxxx : 100 iters time on 1 threads

 slowest          p1         p10         p25         p50         p75         p90         p99     fastest        unit

---

 2.9e-02     2.8e-02     2.7e-02     2.7e-02     2.7e-02     2.7e-02     2.5e-02     2.5e-02     2.5e-02     seconds
   36.62       38.25       39.40       39.94       40.10       40.48       42.35       42.93       42.93    GFlops/s
    0.36        0.37        0.38        0.39        0.39        0.40        0.41        0.42        0.42       GBs/s

Run time 3.0525176525115967 /home/mmperf/external/iree-llvm-sandbox/python/examples/core/harness.py:94: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. self.data = self.data.append(product)

Compilation expert SingleTiling3DPad xxxxxxxxxx: Dialect: Einsum spec: km,kn->mn Compile time 0.19397234916687012 Segmentation fault (core dumped) Traceback (most recent call last): File "mmperf.py", line 520, in sys.exit(main(sys.argv)) File "mmperf.py", line 368, in main sandbox_sizes, sandbox_speeds = sandbox_perf(file_path, args.num_iters) File "mmperf.py", line 266, in sandbox_perf result = subprocess.run(cmd, shell=True, check=True, cwd=dst_f) File "/usr/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'python -m python.examples.matmul.iree_sandbox_matmul -matrix_path /home/mmperf/benchmark_sizes/bert_large_matmul.txt -n_iters 100' returned non-zero exit status 139.

[yzhang93]

@TUMSchieben I've tested this python mmperf.py ./build_sandbox/matmul results -sandbox -benchmark_path=benchmark_sizes/benchmark_all_sizes.txt and found some large sizes failed with segment fault. I tried to run the same sizes in original iree-llvm-sandbox (https://github.com/google/iree-llvm-sandbox/tree/d868c9df3ccc1cf0de0b62c22d4e4a4d70f6bc89), and found the same problem. However, the submodule has not been updated for almost 2 month, and the problem must have been fixed and updated in the ToM of iree-llvm-sandbox. I would suggest you to try with their ToM codes. At the same time, we'll try to update the submodule on our side, but may need some time to finish.

[idoOwhd]

I try this python mmperf.py ./build_sandbox/matmul results -sandbox -benchmark_path=benchmark_sizes/benchmark_all_sizes.txt, but an AssertionError comes up, but I've no idea to solve it: