search

iree-org / iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.
http://iree.dev/
Apache License 2.0
2.47k stars 548 forks source link

Integrate llvm/llvm-project@27ac46e6bea2 #17662

Closed antiagainst closed 5 days ago

antiagainst commented 1 week ago

Updated to llvm/llvm-project@27ac46e6bea2

Updated to openxla/stablehlo@dd48ec5

Updated to llvm/torch-mlir@77d7f64

github-actions[bot] commented 1 week ago

Abbreviated Benchmark Summary

@ commit b918d15fd1fa968cb5c401910f67edd7cd702c58 (vs. base f4279657ef8da12d07f068a37cbd93986edb47d8)

Data-Tiling Comparison Table

Click to show | Name | No-DT (baseline) | DT-Only | DT-UK | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | BertForMaskedLMTF(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[30-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [216.588 (1.0X)](https://perf.iree.dev/serie?IREE?cb3631222b94571a286e32c3aa1e56c021aba1b7f3d82ffd2400ea07d9dfcc3f) | [136.228 (1.6X)](https://perf.iree.dev/serie?IREE?d66ec4d9a0391604c1305ba1621a84e2716900edcfd07b9e910ae2289ae0c06b) | [107.958 (2.0X)](https://perf.iree.dev/serie?IREE?9b3354efe105e56bf9f9ae18ede492ce77a01e5cfcec56eed66b21795d8d8944) | | BertLargeTF(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[30-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [742.909 (1.0X)](https://perf.iree.dev/serie?IREE?cbf78199705b14cb7332489c7415445f1b0e4189ec7a91b28232ff0e037fd7e2) | [273.101 (2.7X)](https://perf.iree.dev/serie?IREE?c508d0d4027c9043adeaa376315af41ed2752cc3a420db026e4c7a65ac7e72f5) | [222.530 (3.3X)](https://perf.iree.dev/serie?IREE?4c7b547fdbbf99b5d399e31f743a40249f0cd2aec451b9c0e6b2222fbf87bf4f) | | DeepLabV3\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [31.949 (1.0X)](https://perf.iree.dev/serie?IREE?015af8c7c74743569726f8fecf3c5af66eb516b1e4c27b9c53444e5eb68254f9) | [36.997 (0.9X)](https://perf.iree.dev/serie?IREE?743cead81b4f18c046391c264faee8fcfec712e0893eb4bd1821c243bbd6a277) | [30.029 (1.1X)](https://perf.iree.dev/serie?IREE?7237c7cbf5353280472161050ccb803bd6237ac656eab0604d5cc610d73ef778) | | DeepLabV3\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [6.924 (1.0X)](https://perf.iree.dev/serie?IREE?882a01b5adfe6cf932e3cacf39a21659d97c6d680a7c3aacbef5298958c13078) | [9.291 (0.7X)](https://perf.iree.dev/serie?IREE?05be03bfe23f01dd96435741a073aafbc0fe170c0e55ab9a7e93d341caef6d4c) | [8.488 (0.8X)](https://perf.iree.dev/serie?IREE?60ebe003ad32386572a7515583e00883b11209d13c62d6907be645492557aa71) | | EfficientNetV2STF(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [273.836 (1.0X)](https://perf.iree.dev/serie?IREE?d14bc72f848279de26aba8bd86bb530767acc4ca769356ab548258db49c44555) | [258.584 (1.1X)](https://perf.iree.dev/serie?IREE?f4f6998a005b11abfc62d171cb7ccecc66ff0adf1250f6141cf77384c8c2710e) | [229.107 (1.2X)](https://perf.iree.dev/serie?IREE?ce7eec0c36a5fda73313a06da87ff315e0307cd6d2962d167e7e641eea50604c) | | EfficientNetV2STF(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [35.882 (1.0X)](https://perf.iree.dev/serie?IREE?e7eb7934128cdfa74ffd4b1a5435fb595b313cfb7057fd458caccf04037346ac) | [36.161 (1.0X)](https://perf.iree.dev/serie?IREE?6896e6df27205cbb6da22c1e01edb20cf99114096afb3070dfcb3cb4e0cffddc) | [34.048 (1.1X)](https://perf.iree.dev/serie?IREE?d38b4a4e1e86311faf6d3a7dcd6a8b8ce8ec305456e4a79e599104dd31e97909) | | EfficientNet\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [26.875 (1.0X)](https://perf.iree.dev/serie?IREE?480a2fe9ab9bd9ade098ff3c5fa0fd61a93c787c99329a1cdcecac6e5d708558) | [51.654 (0.5X)](https://perf.iree.dev/serie?IREE?77ec219632508714b65bf89df1697c34e6d348f3676369c7be150427db27ff69) | [13.073 (2.1X)](https://perf.iree.dev/serie?IREE?423824abc1ed6574ed1315b6c6432366edefbec9704c4b524d6daa9c7f18bf0a) | | EfficientNet\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [5.810 (1.0X)](https://perf.iree.dev/serie?IREE?e94f7cad9035a9a3f3f6dc8ca0fb4ecc25339cf0f4a153c842b95ec00dc66f7f) | [10.966 (0.5X)](https://perf.iree.dev/serie?IREE?c9799ba2e06e7ff1b35ce0afede535fdd00fde8a013b354b441b96cd5f2d746a) | [5.011 (1.2X)](https://perf.iree.dev/serie?IREE?579b8550840595f0dc5a89acbb574ebf022c1581132b82e56139df142953c820) | | GPT2\_117M\_TF\_1X1XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [70.099 (1.0X)](https://perf.iree.dev/serie?IREE?0be99f368751e55d1ce96e0d44819c3ba3a69c12c040048a67344f516f69873e) | [39.009 (1.8X)](https://perf.iree.dev/serie?IREE?0d2a0cc3f7b9ce99e9b440c2a15e1dfc308d6068c16b2f194cd80d662a8631c4) | [39.880 (1.8X)](https://perf.iree.dev/serie?IREE?ce26c2ff64d5511aea1d19f13a17363995cdcf8c88d01097da455525abaf9efe) | | GPT2\_117M\_TF\_1X1XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [9.132 (1.0X)](https://perf.iree.dev/serie?IREE?b04574805bfe322d9ce4e3c40a974d1429196fb3d08ede92ba8f45a74c81a773) | [8.426 (1.1X)](https://perf.iree.dev/serie?IREE?164beca383be6e4b92a1416648eb85b869f90e001dc92f34337e320deb56dac4) | [8.427 (1.1X)](https://perf.iree.dev/serie?IREE?9c569e155e55577bd706c41591db729c6ee388ecd7a466a21d3716dde38575a9) | | GPT2\_117M\_TF\_1X4XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [88.627 (1.0X)](https://perf.iree.dev/serie?IREE?212726872c6a041363a7346217805fde6a21e1953d006a279cb748ca865a95aa) | [42.185 (2.1X)](https://perf.iree.dev/serie?IREE?0e5bcd7b2ff684bb944169754aaa292c7620f9745c8bc7eece25a857ca2790e9) | [41.799 (2.1X)](https://perf.iree.dev/serie?IREE?b56af01b0b5512c28b180552134e3e2701a068586e8a1a08bb307e0a1e42d656) | | GPT2\_117M\_TF\_1X4XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [11.043 (1.0X)](https://perf.iree.dev/serie?IREE?230baee287330f520a0576d6bcdd8df7a714059bdab8d1308b6655269aea2e13) | [8.942 (1.2X)](https://perf.iree.dev/serie?IREE?b1bb91ac3f0d19f8fdacf27867699785b0bf4e497539fd5dc0a560a966a1923b) | [8.844 (1.2X)](https://perf.iree.dev/serie?IREE?dd29ae6a7fad89ad7309c00d1b60ed6314eccd88402f49433e41f857d415a428) | | MiniLML12H384Uncased(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [79.368 (1.0X)](https://perf.iree.dev/serie?IREE?e076babcf92c08d76f05c53bec9bcf823f3855b6280c2c74465ed25bb2bb2bd7) | [78.974 (1.0X)](https://perf.iree.dev/serie?IREE?3cec4327f6d6f3c03c2875c991931aae6d9c570246aaa2d6c77d3645df416418) | [57.119 (1.4X)](https://perf.iree.dev/serie?IREE?3da49d74eed3cd740c69a6a2a97f3ff7e54710ea66c083670042256b2648ddcf) | | MiniLML12H384Uncased(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [12.297 (1.0X)](https://perf.iree.dev/serie?IREE?254aa396e6ccfbf529973e678cf3d88722dacec4e44b58aa1fcc65993e875f0d) | [15.543 (0.8X)](https://perf.iree.dev/serie?IREE?519d29906fc7bc5c26b0ab8daa3110b4eef7b9b31e59024fc3b68a15a1d28b66) | [13.807 (0.9X)](https://perf.iree.dev/serie?IREE?52c4c346a22d0b8ff2fec9701d9bb1aa75423140f5db580ad6da29213aba0d59) | | MobileBertSquad\_fp16(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [180.604 (1.0X)](https://perf.iree.dev/serie?IREE?746443fef718b98d7449c0b2d1733195479afa32e50ae726e8f695cc48611f57) | [249.701 (0.7X)](https://perf.iree.dev/serie?IREE?c2d8e1f9fedf1ccec89bee8f43b61d7d50685dc10988d9c581d1aee016db6f37) | [185.516 (1.0X)](https://perf.iree.dev/serie?IREE?b528e469bfd43258750e70a724bf02eeb157173782b5a5a8912ae036e3ffce58) | | MobileBertSquad\_fp16(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [34.365 (1.0X)](https://perf.iree.dev/serie?IREE?2ef61bde12ad45388014562af6d14a98a83069ff322eaf91293186d8d5ea4bb9) | [65.402 (0.5X)](https://perf.iree.dev/serie?IREE?34ee2e2ba09888f818674020eb64983f8cbeafea086acf8752ce205163a34360) | [61.207 (0.6X)](https://perf.iree.dev/serie?IREE?6c820fd574f08948bddbf14fa5075d1dce2a0191d677cbfedf58fa6b9ddbf9a3) | | MobileBertSquad\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [181.452 (1.0X)](https://perf.iree.dev/serie?IREE?51473638a07429e21bf4b4fdfdb47201bbdff46edc0134cab2d589abc65a4ed6) | [258.734 (0.7X)](https://perf.iree.dev/serie?IREE?ea034f9eba3e0451d6f8d85c4ba75edf05f523679641e59388d0e95a07501426) | [190.092 (1.0X)](https://perf.iree.dev/serie?IREE?4d92c9901b7c73d8e02e63adfdcdf63ef0fb529360a908f93b888dee1c3f9c31) | | MobileBertSquad\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [34.127 (1.0X)](https://perf.iree.dev/serie?IREE?6037970c8a3f46a533e6d0c2db581a2cda6d827709bb23562085b36cf30d5921) | [66.019 (0.5X)](https://perf.iree.dev/serie?IREE?db137427179ede345175477fd30bbdec92e81f9f2db1de00ac270ed4d66c99d7) | [61.293 (0.6X)](https://perf.iree.dev/serie?IREE?d7d25a8c838db8d5859a25187d8fedc23de97e0280b1a85e12e9348f411c0c8e) | | MobileBertSquad\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [490.488 (1.0X)](https://perf.iree.dev/serie?IREE?5b81ba0c3d0db49f11e4c7e51f4138a723c72445c4d1b7d6d441d5a02bbf700a) | [1069.969 (0.5X)](https://perf.iree.dev/serie?IREE?88f76fdb84efab06d138c459614e51abb1a9993921fb78948857eef0f9783f1c) | [214.015 (2.3X)](https://perf.iree.dev/serie?IREE?7001a4f2a5e52aa034f802096f625e278fc10b92cd85653335c3a7c5110492c7) | | MobileBertSquad\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [66.354 (1.0X)](https://perf.iree.dev/serie?IREE?a2bd1c8e875ac8dcd218641e73102249a16c011c38d3775d52d9dd8a9ba324f4) | [132.469 (0.5X)](https://perf.iree.dev/serie?IREE?b780a99e26116acaa224bdfbb951ea06ffbdac06aebd015fddb15eefc7bfb7f7) | [62.066 (1.1X)](https://perf.iree.dev/serie?IREE?6c3eebd478ce05568e03b90fffbaabf0ae95774046d9f492ee53b8e34a6b692d) | | MobileNetV1\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [24.524 (1.0X)](https://perf.iree.dev/serie?IREE?1622e274d5ac570e18826aaec62f223c538583eb2f76e771d24eb2f7785954aa) | [22.923 (1.1X)](https://perf.iree.dev/serie?IREE?38a1120d11abb07ae2e44af50d73bb4dc3ab9d9d6d54dca0b35a1ea74816d4d8) | [18.116 (1.4X)](https://perf.iree.dev/serie?IREE?6600e5c77f343f3727788ac55712340db67660453f0d5b2a78f8a2f00bffa9f2) | | MobileNetV1\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [4.975 (1.0X)](https://perf.iree.dev/serie?IREE?002cd64f66606ef48d9568103412f709d494fbea040a6879b069436ccc106733) | [5.314 (0.9X)](https://perf.iree.dev/serie?IREE?94ff42f10cd47d6a4d48b44dc5e1b51917f104cf16125306ae1cf694c9c8c2d4) | [4.534 (1.1X)](https://perf.iree.dev/serie?IREE?14e8174454310c9b24812dca661319c7b8e78a1175003f56abe8cfa7e7bb9cb9) | | MobileNetV2\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [11.844 (1.0X)](https://perf.iree.dev/serie?IREE?48cac7cf7dea690dd7d8e8669fd5d6f65d1f20c0de1710dc381cf15533354bed) | [15.349 (0.8X)](https://perf.iree.dev/serie?IREE?dcaca9ab1e951e5901176700acd3f87249435eb5f86e050db7acf5e4e956a2b3) | [11.374 (1.0X)](https://perf.iree.dev/serie?IREE?6272e089c33b7c5333b6188b6f61fbb15e7b6a0e9fcd9d54b3b7271cd730e0da) | | MobileNetV2\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [3.728 (1.0X)](https://perf.iree.dev/serie?IREE?9accf20747a0a52c6c6b7da7433c9e9cdf68a813ec6589b781ecb7791a836e34) | [5.376 (0.7X)](https://perf.iree.dev/serie?IREE?97b8402709c23827e768130bf99bb0413af8fadf7b876518e6699988391dc240) | [4.886 (0.8X)](https://perf.iree.dev/serie?IREE?ce780c2ab7c9b837611b5e1dcdbce18e7563fb9d9137e68b5a50bd917a54f83d) | | MobileNetV2\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [21.595 (1.0X)](https://perf.iree.dev/serie?IREE?23e7ffd476616a14cc5b0cabe27332ff71fec9cdc22801b675f8e6349c498814) | [42.746 (0.5X)](https://perf.iree.dev/serie?IREE?f3bd5dc9496be7b61f002132e487918ba87270871a67421c66dc07c54db67a82) | [11.864 (1.8X)](https://perf.iree.dev/serie?IREE?10f2428bc7da79d6d0f23d87caa4cb20ba55d968736b64c6a47c3041be10f641) | | MobileNetV2\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [5.843 (1.0X)](https://perf.iree.dev/serie?IREE?c196cfd95d87ddeb4cb008e055ec417dd805617dd204295c17856ca0f9e0863c) | [9.586 (0.6X)](https://perf.iree.dev/serie?IREE?dccff3d9f2ac906d4abb62647e9720ba736d806e320811f1f038a48b0111486a) | [5.403 (1.1X)](https://perf.iree.dev/serie?IREE?5b41fd88f5fa3c217d024908b57237037d8851b0cba869fb142270cb2fd17ff1) | | MobileNetV3Small\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ c2-standard-60[cpu] | [2.771 (1.0X)](https://perf.iree.dev/serie?IREE?fd46a78e4032c5fa09644bcda90d0d8b73e9196fb89e2458db2838ddf5fd4c16) | [3.335 (0.8X)](https://perf.iree.dev/serie?IREE?3643fcd78839344f79b2abe8185a335331670b2280b32bcb9abef8eb38da1811) | [2.719 (1.0X)](https://perf.iree.dev/serie?IREE?485da7a706b6c0940ef45626ec12ab149da295cc6a3c0a2c63e5a15a952580b4) | | MobileNetV3Small\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [2.848 (1.0X)](https://perf.iree.dev/serie?IREE?069f6917e401e63c9e50c548c70cc699385e6f6908517eb6c79c96e597bf96d7) | [3.462 (0.8X)](https://perf.iree.dev/serie?IREE?d28571a1e8bf10881978bb14a2ec3bc8b98b13759989ef2dc9cd4e0098cbad4d) | [2.824 (1.0X)](https://perf.iree.dev/serie?IREE?c27738e97498c969076d1a2a693322821dd104dbcf7ba6e129ba893584bb0dfd) | | MobileSSD\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [35.119 (1.0X)](https://perf.iree.dev/serie?IREE?0aac8a2a5c45ed0ed35dcd65338a5a414c6beefcdbb0fbb4f299b42d41b639e1) | [39.143 (0.9X)](https://perf.iree.dev/serie?IREE?5220973fd48229657591ff1dcb25fc65ff226cdc18fc8ba4a3f63f97ae0721fb) | [31.758 (1.1X)](https://perf.iree.dev/serie?IREE?d6bfea70085e57a372f18983ddd9f7598b084dc4aac07754c80e4f4f5c4fb407) | | MobileSSD\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [8.480 (1.0X)](https://perf.iree.dev/serie?IREE?4af168ed94d96166f35b8264e160ca1e85a3c6ef3faa08284f447a5613f6ce39) | [10.922 (0.8X)](https://perf.iree.dev/serie?IREE?f6a67dc645b0ccbe999fd99bbb14fc092f22c4037d94e0883b5bbd6a934a3785) | [9.806 (0.9X)](https://perf.iree.dev/serie?IREE?ec20addfc5f284c92b739d0eaf245af0027627de593635539a86709332ae5acf) | | PersonDetect\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ c2-standard-60[cpu] | [0.698 (1.0X)](https://perf.iree.dev/serie?IREE?da589d3a658ddcc4dacaab64c8c7253bab3b0b90fbd35158ba58ed883266d5dc) | [1.300 (0.5X)](https://perf.iree.dev/serie?IREE?16eaa3be7a8f8e5e7c4a9fa0843fe07e74afee4e52bead60aaf5df3c38b308aa) | [0.573 (1.2X)](https://perf.iree.dev/serie?IREE?3283ddd7c21e5db8eea573c2f94ae318c5baa6bf3d9340ba157573937e7b6632) | | PersonDetect\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [0.766 (1.0X)](https://perf.iree.dev/serie?IREE?77dd6dcff77b2053dbc4cbafc7ca36f8ee5aabdc138b5808830908b037014cc3) | [1.378 (0.6X)](https://perf.iree.dev/serie?IREE?b4c044d54badc2cbccba6ca5e7a34c87662a80ee5a82342965f093c79b502923) | [0.632 (1.2X)](https://perf.iree.dev/serie?IREE?8d8fd2fbd7901ece93ffa5e47c460dd793c4489b5751a15bb0c3e1b8d82073db) | | PoseNet\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [17.591 (1.0X)](https://perf.iree.dev/serie?IREE?0d4e114d66ae2e078076cc40fca5e6af76232c3936effb92d33e23f76f26ede8) | [24.207 (0.7X)](https://perf.iree.dev/serie?IREE?240bed9536f23aa37693e758c317d4edcc131bc420736c2ab4712829c33909a3) | [18.919 (0.9X)](https://perf.iree.dev/serie?IREE?51181aae886260ff3c24d829e8bf9e3a892aa93305321c1012476aace79f9e65) | | PoseNet\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [4.113 (1.0X)](https://perf.iree.dev/serie?IREE?a2ebf5883d38f358868199609143debdbb2947b6e0ab6c5b03802cb813022f9f) | [5.902 (0.7X)](https://perf.iree.dev/serie?IREE?922fc7aad073c4a1cf509939a4c85e346b1623e6f0d1741fcdfaab4507f4c446) | [5.123 (0.8X)](https://perf.iree.dev/serie?IREE?1e0197113e1bab228898b4e76067c7c8dcd0faf2b0cf5af9dbb227491de894e4) | | matmul\_256x256x2048\_i8\_i4\_i32\_tile\_config\_default(linalg) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ c2-standard-60[cpu] | [7.593 (1.0X)](https://perf.iree.dev/serie?IREE?641b82f32c47ecd4d02c8c82926118acfce0f530e8728e04a1d593a2876847d2) | [7.552 (1.0X)](https://perf.iree.dev/serie?IREE?572f89c5a8137fcf5856f5d94835535008403ed036c5bd53042ca6df4dddcc1d) | [7.579 (1.0X)](https://perf.iree.dev/serie?IREE?c3a0b8c64c6406c9e4a46d537f2acd4ed2b9f6c191387830c5fcb215cd91d9d0) | | DeepLabV3\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [48.870 (1.0X)](https://perf.iree.dev/serie?IREE?95281c38b844a3b0ea1964e9634e7a8e2b40025936e3402ff2902be01dbd31b7) | [85.362 (0.6X)](https://perf.iree.dev/serie?IREE?5bbd93c48aea71957fe659dc2fd4645dee8557e76e580f43395b4ec33f73e7c1) | [43.950 (1.1X)](https://perf.iree.dev/serie?IREE?f17944b7339d0d84be14cd71d31c10b495df98114d5af917259df75540551fa4) | | DeepLabV3\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [50.161 (1.0X)](https://perf.iree.dev/serie?IREE?4cc57db28e42e4b50f3d234a99faee5e7d48ac787d70f106ed2260e4160f27fc) | [86.173 (0.6X)](https://perf.iree.dev/serie?IREE?a3b21d18d3fa90747450ec5a36386400b6f48c5ab5e5393e05138bb5ef5daadb) | [44.522 (1.1X)](https://perf.iree.dev/serie?IREE?d44c3fbc39f410214516a4c591f879e0ac9454b33a970ff63953fc00f2ec465b) | | DeepLabV3\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [30.148 (1.0X)](https://perf.iree.dev/serie?IREE?ed4f76526e499d8e959237456899cc74fa4bab29674b0ba083c5ce38edc61fab) | [50.222 (0.6X)](https://perf.iree.dev/serie?IREE?9f1f23ea9084f82c09b20fa3b41f7622b374fbcd47917db5876e323302efd0b2) | [27.713 (1.1X)](https://perf.iree.dev/serie?IREE?5343c96ad4bb05804680ca8a51d26bc1ffc4e1d16348e923b4ea234ceb6f94b4) | | GPT2\_117M\_TF\_1X1XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [91.988 (1.0X)](https://perf.iree.dev/serie?IREE?712d1d8286ecd1d7d66c2f4426924cff01be3c71d3512d1f675fc3560487113b) | [21.605 (4.3X)](https://perf.iree.dev/serie?IREE?3aa9b99cc5acee247d6642af968bf90ba7d1a0627d8cd3c573f0e7a2522bf0ef) | [21.298 (4.3X)](https://perf.iree.dev/serie?IREE?d43fc641fce6a72ff3fe58571f3c55e36e65ef7fc868f197554cdd9a5a451015) | | GPT2\_117M\_TF\_1X1XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [92.809 (1.0X)](https://perf.iree.dev/serie?IREE?71eae757691075543390b054227af100cfbb850c70094713e12f2c48c2f7db07) | [22.112 (4.2X)](https://perf.iree.dev/serie?IREE?462fad0e00922d12af0343ce5929213b26c49c383fd5c69dc08c9ca5145c0293) | [21.786 (4.3X)](https://perf.iree.dev/serie?IREE?3b12e9908a7263dea59779315d80b3b215f17a287e84b6cb3a73ac2b5faa1d0f) | | GPT2\_117M\_TF\_1X1XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [52.257 (1.0X)](https://perf.iree.dev/serie?IREE?a20f9c8cbe11916179b5a347f4e60d1c4e37519719e1aeeface855fe7fc4740f) | [21.971 (2.4X)](https://perf.iree.dev/serie?IREE?fc795521dc3d23e4e8665c02b111d361e4da9dd54e38885995e100d4e0063231) | [21.738 (2.4X)](https://perf.iree.dev/serie?IREE?f64e2e4991de95b0282191703bcc5eade1188cbc1dc5012fe7a377d7300e0954) | | GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [134.269 (1.0X)](https://perf.iree.dev/serie?IREE?8f03b8167746d6dfe9237cf890831c5521ab5169b0892d660c2f817c5f579223) | [27.732 (4.8X)](https://perf.iree.dev/serie?IREE?2d8906fd07aa37db29f8d4959028469dd3d384df8312351839cc51868e0932aa) | [27.705 (4.8X)](https://perf.iree.dev/serie?IREE?8ae7cfed6678287118515c19784beaae637b5bfa1a259ee0c40d0ae15de02f32) | | GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [134.984 (1.0X)](https://perf.iree.dev/serie?IREE?ad83887341f3e360b8a4be6c5683e012c82aeb10d65482cfde8e842bc144a48e) | [30.162 (4.5X)](https://perf.iree.dev/serie?IREE?458172413fb7700f4a84ae51402399781a76ad89398e6b6d0ac20ab605f1fd1d) | [29.443 (4.6X)](https://perf.iree.dev/serie?IREE?d9f100bcdbbfe35bada2541180c89460cc12b0e8a17c3c0126af94dd3e194f04) | | GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [75.121 (1.0X)](https://perf.iree.dev/serie?IREE?38f7c3ef079798f2116c2bdff47240a6a261b066b99b8d12fe8e7da255c0e1f3) | [26.746 (2.8X)](https://perf.iree.dev/serie?IREE?9f19b85505311dadfe8c5211a8a50f75b77b35bc3f0c5031be7c3d7a9d1581f7) | [26.845 (2.8X)](https://perf.iree.dev/serie?IREE?10216d6baf8d3e228a42f5849a691954104e8fc91e514be1c63736ef737f59d5) | | MobileBertSquad\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [709.751 (1.0X)](https://perf.iree.dev/serie?IREE?dc2023c6113c87aad59f2b49214ab2995b32c7ba040b314e890ea2ec7081f90b) | [449.043 (1.6X)](https://perf.iree.dev/serie?IREE?2eef43089ae6d4ec5425dd2e53110a2b2627cc9a8a85dc8dd295033ed7177572) | [349.952 (2.0X)](https://perf.iree.dev/serie?IREE?d4572856894af9013e311991e4371c81498ee30b1fc90ee840632d1a3a512193) | | MobileBertSquad\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [710.270 (1.0X)](https://perf.iree.dev/serie?IREE?ab3be2a007f3201e419112cd2bf753bbbe4e15431946411433a61ab0e34cdfca) | [464.069 (1.5X)](https://perf.iree.dev/serie?IREE?8c78d47df130263e959b534abeb7045f86dc5b359cea445abd85ac2769efa6ee) | [358.349 (2.0X)](https://perf.iree.dev/serie?IREE?cca718432d630f48a03660753dbea3c60120aea2692fab0fccf6a4928be7a247) | | MobileBertSquad\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [398.723 (1.0X)](https://perf.iree.dev/serie?IREE?c1b5a77b70decd14b1d3268ad2a167631422bff95c8f8c126dc7a876bd3c0632) | [276.855 (1.4X)](https://perf.iree.dev/serie?IREE?80b58632ef882cdaec07c38dddc972ce9363cc435ecba22530b109479b4a59b8) | [217.366 (1.8X)](https://perf.iree.dev/serie?IREE?04f958179d9bc04eca09f2ad518a3cb494931445f23fcc2791b2d9fcee5cf1bc) | | MobileBertSquad\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [1115.003 (1.0X)](https://perf.iree.dev/serie?IREE?df6786c3bd20d93e1230f8b59212221a7e9de0eefdc39ac2f7192b76047d2803) | [1069.568 (1.0X)](https://perf.iree.dev/serie?IREE?33e5e79843a382c4a07d443902369a96b08b2d274fe00db77c6f510838ee6820) | [304.249 (3.7X)](https://perf.iree.dev/serie?IREE?5a9829035177db026ff3371238afa1f319a3b715e22ea7d1670c8fab8c243d94) | | MobileBertSquad\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [1118.333 (1.0X)](https://perf.iree.dev/serie?IREE?106ccd69f92add8c01ecfa00b551ae901a5e9864595601ff75e090a03c97dc49) | [1074.236 (1.0X)](https://perf.iree.dev/serie?IREE?caa6101a583b18f027528852c89e896d5ff42383141c5063f664ed6b245f6943) | [307.661 (3.6X)](https://perf.iree.dev/serie?IREE?dbaa3dbc7fba073c6e934eb505c4679fe625bbfdeb7e2316960c659f6eb8b2e6) | | MobileBertSquad\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [580.417 (1.0X)](https://perf.iree.dev/serie?IREE?8156b68796001f010990cd4da026415ee8875a0ccf609258df8b27c1cd5ed71e) | [584.437 (1.0X)](https://perf.iree.dev/serie?IREE?84a126639a5fa770b83493257cd4f61a235bb813a1213baef295d4c07157ae86) | [182.949 (3.2X)](https://perf.iree.dev/serie?IREE?9cff4a1b1873b4cda93168fc674fdc046c7b6640f81283cae57c871afbbe216d) | | Vit\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [2099.719 (1.0X)](https://perf.iree.dev/serie?IREE?1bb02b9cb5407a193c5ad68d57ba004d6694ae1b9f3b4af974af7197f30f9082) | [1856.552 (1.1X)](https://perf.iree.dev/serie?IREE?a57eb6fb2cf6fbf61f9141e63784cc356153766042d0890e6caee565eadcbf21) | [302.392 (6.9X)](https://perf.iree.dev/serie?IREE?3263426782173c417a4205ee460ccf4acb939c53397da8ae06f8ebf3f7228f87) | | Vit\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [2102.297 (1.0X)](https://perf.iree.dev/serie?IREE?5f2fe9c7dc19b8dda9300eb881b22481951e7c8f9aaaef2923bf31cea6b4d812) | [1881.012 (1.1X)](https://perf.iree.dev/serie?IREE?c3116c8939132a965cb9aaf18bd682f915bfcd780109926e151083b122722e52) | [301.545 (7.0X)](https://perf.iree.dev/serie?IREE?893537d80a1d230ac7751f901899b02d29ad6a179afa59b16b16e712a2fab297) | | Vit\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [1133.840 (1.0X)](https://perf.iree.dev/serie?IREE?8c2249d8f9c199d56ae43e1d4d6b288194fa1e3b31914cc88d210deccad3d351) | [1077.407 (1.1X)](https://perf.iree.dev/serie?IREE?d9648731b2f404580bedd33800f06352af624b5bf0e19530224040279061154f) | [179.527 (6.3X)](https://perf.iree.dev/serie?IREE?6c111c114ceccecfdeb1b3608ec4701ac6c62fa29abfa7270a6737f92c94cb0b) | | matmul\_256x256x2048\_i8\_i4\_i32\_tile\_config\_default(linalg) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [12.382 (1.0X)](https://perf.iree.dev/serie?IREE?a694805fd2aa24f7bb3464e817ade1eda09588928e5b168947eef7e6b5ac8dee) | [14.433 (0.9X)](https://perf.iree.dev/serie?IREE?e8065d1ddf99eb564720a2efd4c3b45e2553b6b34f71a857501a49c39c1d8f82) | [1.304 (9.5X)](https://perf.iree.dev/serie?IREE?fe0a953188f398da446a84e74ad069d4029568c0a02709b84bef8922533bb14a) |

Regressed Latencies 🚩

Benchmark Name Average Latency (ms) Median Latency (ms) Latency Standard Deviation (ms)
matmul\_2562x2561x2561\_f32t\_f32t\_f32t\_tile\_config\_default(linalg) [cuda-sm\_80-linux\_gnu-cuda][ukernel,matmul] cuda(none)[full-inference,default-flags] with default @ a2-highgpu-1g[gpu] 1.534 (vs. 1.368, 12.15%↑) 1.534 0.001
matmul\_123x2561x2561\_f32t\_f32t\_f32t\_tile\_config\_default(linalg) [cuda-sm\_80-linux\_gnu-cuda][ukernel,matmul] cuda(none)[full-inference,default-flags] with default @ a2-highgpu-1g[gpu] 0.222 (vs. 0.200, 11.16%↑) 0.222 0.000
MobileBertSquad\_int8(tflite) [arm-valhall-vulkan\_android31-vulkan\_spirv][default-flags] vulkan(none)[full-inference,default-flags] with default @ pixel-6-pro[gpu] 94.955 (vs. 86.395, 9.91%↑) 95.940 2.322

[Top 3 out of 4 results showed]

Improved Latencies 🎉

Benchmark Name Average Latency (ms) Median Latency (ms) Latency Standard Deviation (ms)
matmul\_3456x1024x2048\_f32t\_tile\_config\_default(linalg) [cuda-sm\_80-linux\_gnu-cuda][ukernel,matmul] cuda(none)[full-inference,default-flags] with default @ a2-highgpu-1g[gpu] 0.130 (vs. 0.166, 21.53%↓) 0.130 0.000
MobileBertSquad\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu][experimental-flags,dt-only] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] 1069.568 (vs. 1222.156, 12.49%↓) 1070.319 4.894
MobileBertSquad\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu][experimental-flags,dt-only] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] 584.437 (vs. 652.467, 10.43%↓) 588.813 12.434

[Top 3 out of 21 results showed]

Improved Total Dispatch Sizes 🎉

Benchmark Name Total Dispatch Size (bytes)
GPT2\_117M\_TF\_1X1XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu][experimental-flags,dt-only,compile-stats] 11392 (vs. 12864, 11.44%↓)
GPT2\_117M\_TF\_1X1XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu][default-flags,dt-uk,compile-stats] 11280 (vs. 12336, 8.56%↓)
GPT2\_117M\_TF\_1X4XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu][experimental-flags,dt-only,compile-stats] 18224 (vs. 19328, 5.71%↓)

[Top 3 out of 6 results showed]

Regressed Stream IR Dispatch Count (# of cmd.dispatch ops) 🚩

Benchmark Name Stream IR Dispatch Count (# of cmd.dispatch ops)
GPT2\_117M\_TF\_1X4XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu][experimental-flags,dt-only,compile-stats] 330 (vs. 318, 3.77%↑)
GPT2\_117M\_TF\_1X4XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu][default-flags,dt-uk,compile-stats] 330 (vs. 318, 3.77%↑)
GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu][default-flags,dt-uk,compile-stats] 330 (vs. 318, 3.77%↑)

[Top 3 out of 10 results showed]

Improved Stream IR Dispatch Count (# of cmd.dispatch ops) 🎉

Benchmark Name Stream IR Dispatch Count (# of cmd.dispatch ops)
GPT2\_117M\_TF\_1X1XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu][experimental-flags,dt-only,compile-stats] 355 (vs. 367, 3.27%↓)
GPT2\_117M\_TF\_1X1XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu][default-flags,dt-uk,compile-stats] 355 (vs. 367, 3.27%↓)
GPT2\_117M\_TF\_1X1XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu][default-flags,dt-uk,compile-stats] 355 (vs. 367, 3.27%↓)

[Top 3 out of 6 results showed]

For more information:

Source Workflow Run

qedawkins commented 1 week ago

This seems like a fairly likely candidate for the source of dispatch count changes: https://github.com/iree-org/llvm-project/commit/7ef83f5561b34ca07fdef23ca2b3c01c583dbbf5

Especially because the changes are being observed in data tiling enabled benchmarks. cc @Max191

nirvedhmeshram commented 5 days ago

We need to look at the regressions in number of dispatches. I can help (but not today).

@MaheshRavishankar are you blocking the integrate for this or would you look at it in a follow up since Quinn has explained the possible reason for the difference?

MaheshRavishankar commented 5 days ago

We need to look at the regressions in number of dispatches. I can help (but not today).

@MaheshRavishankar are you blocking the integrate for this or would you look at it in a follow up since Quinn has explained the possible reason for the difference?

Could you try reverting that locally to see if that is the issue. Then we can decide what to do next

nirvedhmeshram commented 5 days ago

We need to look at the regressions in number of dispatches. I can help (but not today).

@MaheshRavishankar are you blocking the integrate for this or would you look at it in a follow up since Quinn has explained the possible reason for the difference?

Could you try reverting that locally to see if that is the issue. Then we can decide what to do next

@MaheshRavishankar PTAL at the benchmark comment now, the bot has edited it and it seems the dispatch number regression is gone with the revert.