search

buddy-compiler / buddy-benchmark

Benchmark Framework for Buddy Projects
Apache License 2.0
46 stars 39 forks source link

[OSPP] MatMul and Conv2d optimization benchmark. #43

Closed WLFJ closed 2 years ago

WLFJ commented 2 years ago

Test Platform Infomation

Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz
Run on (16 X 2500 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 36608 KiB (x1)

Common Benchmark Bootstrap

In root of buddy-benchmark, run these command to only build OpBenchmark:

cd build
rm -rf bin
rm -rf benchmarks 
cmake -G Ninja .. \
    -DOP_OPTIMIZATION_BENCHMARKS=ON \
    -DVECTORIZATION_BENCHMARKS=OFF \
    -DOpenCV_DIR=/YOUR/OPENCV/BUILD/DIR \
    -DDEEP_LEARNING_BENCHMARKS=OFF \
    -DBUDDY_OPT_BUILD_DIR=/YOUR/BUDDY_OPT/BUILD/DIR/ \
    -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
    -DCMAKE_BUILD_TYPE=Debug
    # -DBUDDY_OPT_ATTR=avx512f \
cd ..
ninja -C build

MatMul Benchmark Result

Running ./build/bin/gemm-benchmark | python3 scripts/gflops.py

BM_GEMM means my optimized result and BM_OPENCV_GEMM means OpenCV's implementation:

2022-09-16T10:22:42+08:00
Running ./build/bin/gemm-benchmark
Run on (16 X 2500 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 36608 KiB (x1)
Load Average: 0.14, 0.11, 0.04
BM_GEMM          64              553538.4591475971
BM_GEMM          128             65398.170986576835
BM_GEMM          192             20453.796914273567
BM_GEMM          256             7259.422550877849
BM_GEMM          320             4425.801223300597
BM_GEMM          384             2445.6269631862415
BM_GEMM          448             1517.6112513245014
BM_GEMM          512             656.6318096699417
BM_GEMM          576             713.0500382399725
BM_GEMM          640             513.8042782220483
BM_GEMM          704             399.50232364077203
BM_GEMM          768             179.27180559842645
BM_GEMM          832             236.91637368059855
BM_GEMM          896             195.1725279286418
BM_GEMM          960             168.36350050383183
BM_GEMM          1024            48.06790476467977
BM_GEMM          1088            122.77465160054176
BM_GEMM          1152            97.43677262618962
BM_GEMM          1216            78.04578604837354
BM_GEMM          1280            49.49216409035463
BM_GEMM          1344            60.184090291471385
BM_GEMM          1408            38.509812660864725
BM_GEMM          1472            38.90747680496042
BM_GEMM          1536            28.991043221682833
BM_GEMM          1600            38.976016592059814
BM_GEMM          1664            37.99762681161418
BM_GEMM          1728            20.022802526841666
BM_GEMM          1792            16.536948054130665
BM_GEMM          1856            19.24368720162082
BM_GEMM          1920            18.903745403351568
BM_GEMM          1984            19.837965027295297
BM_GEMM          2048            12.22523707197647
BM_OPENCV_GEMM           64              130048.1349219794
BM_OPENCV_GEMM           128             15361.50155214498
BM_OPENCV_GEMM           192             4582.319342086562
BM_OPENCV_GEMM           256             1860.7300695371612
BM_OPENCV_GEMM           320             877.6974007035395
BM_OPENCV_GEMM           384             493.81378686255556
BM_OPENCV_GEMM           448             301.2306977860141
BM_OPENCV_GEMM           512             98.20550977576413
BM_OPENCV_GEMM           576             139.52611443413645
BM_OPENCV_GEMM           640             104.13489342613329
BM_OPENCV_GEMM           704             77.68314969149178
BM_OPENCV_GEMM           768             33.54486997269415
BM_OPENCV_GEMM           832             42.60413044461639
BM_OPENCV_GEMM           896             33.924426632610334
BM_OPENCV_GEMM           960             25.28015580042007
BM_OPENCV_GEMM           1024            9.820765763749563
BM_OPENCV_GEMM           1088            15.988910399432237
BM_OPENCV_GEMM           1152            16.12311889263529
BM_OPENCV_GEMM           1216            15.48935532846007
BM_OPENCV_GEMM           1280            5.118412785943296
BM_OPENCV_GEMM           1344            7.113887944011761
BM_OPENCV_GEMM           1408            6.623986646649414
BM_OPENCV_GEMM           1472            6.477157608819943
BM_OPENCV_GEMM           1536            4.280514124205863
BM_OPENCV_GEMM           1600            6.127263601585574
BM_OPENCV_GEMM           1664            5.633894140118237
BM_OPENCV_GEMM           1728            5.854860137873324
BM_OPENCV_GEMM           1792            5.5199058465802215
BM_OPENCV_GEMM           1856            9.284927480299219
BM_OPENCV_GEMM           1920            9.287915489124313
BM_OPENCV_GEMM           1984            9.341679179380826
BM_OPENCV_GEMM           2048            5.764199108684365

Correctness

Coming soon ...

Conv2d(Conv2dNchwFchw) Benchmark Result

Running ./build/bin/conv2d-benchmark | python3 scripts/gflops.py

BM_CONV means my optimized result and BM_CONV_ORG means MLIR linalg lower to affine result: