search

iree-org / iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.
http://iree.dev/
Apache License 2.0
2.47k stars 551 forks source link

Start LLVM integrate integrates/llvm-20240618 #17690

Closed nirvedhmeshram closed 1 week ago

github-actions[bot] commented 1 week ago

Abbreviated Benchmark Summary

@ commit b056699587b42ca5956ec4ea21c939eb67aacb0a (vs. base 1f954b2ba1222a3153bd8c084bf0010abdaad459)

Data-Tiling Comparison Table

Click to show | Name | No-DT (baseline) | DT-Only | DT-UK | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | BertForMaskedLMTF(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[30-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [213.058 (1.0X)](https://perf.iree.dev/serie?IREE?cb3631222b94571a286e32c3aa1e56c021aba1b7f3d82ffd2400ea07d9dfcc3f) | [133.223 (1.6X)](https://perf.iree.dev/serie?IREE?d66ec4d9a0391604c1305ba1621a84e2716900edcfd07b9e910ae2289ae0c06b) | [104.240 (2.0X)](https://perf.iree.dev/serie?IREE?9b3354efe105e56bf9f9ae18ede492ce77a01e5cfcec56eed66b21795d8d8944) | | BertLargeTF(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[30-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [799.419 (1.0X)](https://perf.iree.dev/serie?IREE?cbf78199705b14cb7332489c7415445f1b0e4189ec7a91b28232ff0e037fd7e2) | [269.342 (3.0X)](https://perf.iree.dev/serie?IREE?c508d0d4027c9043adeaa376315af41ed2752cc3a420db026e4c7a65ac7e72f5) | [223.769 (3.6X)](https://perf.iree.dev/serie?IREE?4c7b547fdbbf99b5d399e31f743a40249f0cd2aec451b9c0e6b2222fbf87bf4f) | | DeepLabV3\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [32.027 (1.0X)](https://perf.iree.dev/serie?IREE?015af8c7c74743569726f8fecf3c5af66eb516b1e4c27b9c53444e5eb68254f9) | [36.774 (0.9X)](https://perf.iree.dev/serie?IREE?743cead81b4f18c046391c264faee8fcfec712e0893eb4bd1821c243bbd6a277) | [29.996 (1.1X)](https://perf.iree.dev/serie?IREE?7237c7cbf5353280472161050ccb803bd6237ac656eab0604d5cc610d73ef778) | | DeepLabV3\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [7.059 (1.0X)](https://perf.iree.dev/serie?IREE?882a01b5adfe6cf932e3cacf39a21659d97c6d680a7c3aacbef5298958c13078) | [9.277 (0.8X)](https://perf.iree.dev/serie?IREE?05be03bfe23f01dd96435741a073aafbc0fe170c0e55ab9a7e93d341caef6d4c) | [8.545 (0.8X)](https://perf.iree.dev/serie?IREE?60ebe003ad32386572a7515583e00883b11209d13c62d6907be645492557aa71) | | EfficientNetV2STF(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [274.173 (1.0X)](https://perf.iree.dev/serie?IREE?d14bc72f848279de26aba8bd86bb530767acc4ca769356ab548258db49c44555) | [259.453 (1.1X)](https://perf.iree.dev/serie?IREE?f4f6998a005b11abfc62d171cb7ccecc66ff0adf1250f6141cf77384c8c2710e) | [228.318 (1.2X)](https://perf.iree.dev/serie?IREE?ce7eec0c36a5fda73313a06da87ff315e0307cd6d2962d167e7e641eea50604c) | | EfficientNetV2STF(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [35.696 (1.0X)](https://perf.iree.dev/serie?IREE?e7eb7934128cdfa74ffd4b1a5435fb595b313cfb7057fd458caccf04037346ac) | [36.008 (1.0X)](https://perf.iree.dev/serie?IREE?6896e6df27205cbb6da22c1e01edb20cf99114096afb3070dfcb3cb4e0cffddc) | [34.294 (1.0X)](https://perf.iree.dev/serie?IREE?d38b4a4e1e86311faf6d3a7dcd6a8b8ce8ec305456e4a79e599104dd31e97909) | | EfficientNet\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [26.955 (1.0X)](https://perf.iree.dev/serie?IREE?480a2fe9ab9bd9ade098ff3c5fa0fd61a93c787c99329a1cdcecac6e5d708558) | [51.660 (0.5X)](https://perf.iree.dev/serie?IREE?77ec219632508714b65bf89df1697c34e6d348f3676369c7be150427db27ff69) | [13.044 (2.1X)](https://perf.iree.dev/serie?IREE?423824abc1ed6574ed1315b6c6432366edefbec9704c4b524d6daa9c7f18bf0a) | | EfficientNet\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [5.825 (1.0X)](https://perf.iree.dev/serie?IREE?e94f7cad9035a9a3f3f6dc8ca0fb4ecc25339cf0f4a153c842b95ec00dc66f7f) | [10.946 (0.5X)](https://perf.iree.dev/serie?IREE?c9799ba2e06e7ff1b35ce0afede535fdd00fde8a013b354b441b96cd5f2d746a) | [5.039 (1.2X)](https://perf.iree.dev/serie?IREE?579b8550840595f0dc5a89acbb574ebf022c1581132b82e56139df142953c820) | | GPT2\_117M\_TF\_1X1XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [70.100 (1.0X)](https://perf.iree.dev/serie?IREE?0be99f368751e55d1ce96e0d44819c3ba3a69c12c040048a67344f516f69873e) | [39.920 (1.8X)](https://perf.iree.dev/serie?IREE?0d2a0cc3f7b9ce99e9b440c2a15e1dfc308d6068c16b2f194cd80d662a8631c4) | [37.110 (1.9X)](https://perf.iree.dev/serie?IREE?ce26c2ff64d5511aea1d19f13a17363995cdcf8c88d01097da455525abaf9efe) | | GPT2\_117M\_TF\_1X1XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [9.184 (1.0X)](https://perf.iree.dev/serie?IREE?b04574805bfe322d9ce4e3c40a974d1429196fb3d08ede92ba8f45a74c81a773) | [8.790 (1.0X)](https://perf.iree.dev/serie?IREE?164beca383be6e4b92a1416648eb85b869f90e001dc92f34337e320deb56dac4) | [8.819 (1.0X)](https://perf.iree.dev/serie?IREE?9c569e155e55577bd706c41591db729c6ee388ecd7a466a21d3716dde38575a9) | | GPT2\_117M\_TF\_1X4XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [86.044 (1.0X)](https://perf.iree.dev/serie?IREE?212726872c6a041363a7346217805fde6a21e1953d006a279cb748ca865a95aa) | [42.339 (2.0X)](https://perf.iree.dev/serie?IREE?0e5bcd7b2ff684bb944169754aaa292c7620f9745c8bc7eece25a857ca2790e9) | [38.398 (2.2X)](https://perf.iree.dev/serie?IREE?b56af01b0b5512c28b180552134e3e2701a068586e8a1a08bb307e0a1e42d656) | | GPT2\_117M\_TF\_1X4XI32(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [11.121 (1.0X)](https://perf.iree.dev/serie?IREE?230baee287330f520a0576d6bcdd8df7a714059bdab8d1308b6655269aea2e13) | [8.701 (1.3X)](https://perf.iree.dev/serie?IREE?b1bb91ac3f0d19f8fdacf27867699785b0bf4e497539fd5dc0a560a966a1923b) | [8.652 (1.3X)](https://perf.iree.dev/serie?IREE?dd29ae6a7fad89ad7309c00d1b60ed6314eccd88402f49433e41f857d415a428) | | MiniLML12H384Uncased(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [79.508 (1.0X)](https://perf.iree.dev/serie?IREE?e076babcf92c08d76f05c53bec9bcf823f3855b6280c2c74465ed25bb2bb2bd7) | [77.125 (1.0X)](https://perf.iree.dev/serie?IREE?3cec4327f6d6f3c03c2875c991931aae6d9c570246aaa2d6c77d3645df416418) | [55.921 (1.4X)](https://perf.iree.dev/serie?IREE?3da49d74eed3cd740c69a6a2a97f3ff7e54710ea66c083670042256b2648ddcf) | | MiniLML12H384Uncased(stablehlo) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [12.338 (1.0X)](https://perf.iree.dev/serie?IREE?254aa396e6ccfbf529973e678cf3d88722dacec4e44b58aa1fcc65993e875f0d) | [14.319 (0.9X)](https://perf.iree.dev/serie?IREE?519d29906fc7bc5c26b0ab8daa3110b4eef7b9b31e59024fc3b68a15a1d28b66) | [13.169 (0.9X)](https://perf.iree.dev/serie?IREE?52c4c346a22d0b8ff2fec9701d9bb1aa75423140f5db580ad6da29213aba0d59) | | MobileBertSquad\_fp16(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [180.383 (1.0X)](https://perf.iree.dev/serie?IREE?746443fef718b98d7449c0b2d1733195479afa32e50ae726e8f695cc48611f57) | [247.400 (0.7X)](https://perf.iree.dev/serie?IREE?c2d8e1f9fedf1ccec89bee8f43b61d7d50685dc10988d9c581d1aee016db6f37) | [185.039 (1.0X)](https://perf.iree.dev/serie?IREE?b528e469bfd43258750e70a724bf02eeb157173782b5a5a8912ae036e3ffce58) | | MobileBertSquad\_fp16(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [34.284 (1.0X)](https://perf.iree.dev/serie?IREE?2ef61bde12ad45388014562af6d14a98a83069ff322eaf91293186d8d5ea4bb9) | [64.972 (0.5X)](https://perf.iree.dev/serie?IREE?34ee2e2ba09888f818674020eb64983f8cbeafea086acf8752ce205163a34360) | [60.827 (0.6X)](https://perf.iree.dev/serie?IREE?6c820fd574f08948bddbf14fa5075d1dce2a0191d677cbfedf58fa6b9ddbf9a3) | | MobileBertSquad\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [180.789 (1.0X)](https://perf.iree.dev/serie?IREE?51473638a07429e21bf4b4fdfdb47201bbdff46edc0134cab2d589abc65a4ed6) | [251.641 (0.7X)](https://perf.iree.dev/serie?IREE?ea034f9eba3e0451d6f8d85c4ba75edf05f523679641e59388d0e95a07501426) | [189.259 (1.0X)](https://perf.iree.dev/serie?IREE?4d92c9901b7c73d8e02e63adfdcdf63ef0fb529360a908f93b888dee1c3f9c31) | | MobileBertSquad\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [34.332 (1.0X)](https://perf.iree.dev/serie?IREE?6037970c8a3f46a533e6d0c2db581a2cda6d827709bb23562085b36cf30d5921) | [65.330 (0.5X)](https://perf.iree.dev/serie?IREE?db137427179ede345175477fd30bbdec92e81f9f2db1de00ac270ed4d66c99d7) | [61.452 (0.6X)](https://perf.iree.dev/serie?IREE?d7d25a8c838db8d5859a25187d8fedc23de97e0280b1a85e12e9348f411c0c8e) | | MobileBertSquad\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [490.336 (1.0X)](https://perf.iree.dev/serie?IREE?5b81ba0c3d0db49f11e4c7e51f4138a723c72445c4d1b7d6d441d5a02bbf700a) | [1068.335 (0.5X)](https://perf.iree.dev/serie?IREE?88f76fdb84efab06d138c459614e51abb1a9993921fb78948857eef0f9783f1c) | [214.429 (2.3X)](https://perf.iree.dev/serie?IREE?7001a4f2a5e52aa034f802096f625e278fc10b92cd85653335c3a7c5110492c7) | | MobileBertSquad\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[15-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [66.447 (1.0X)](https://perf.iree.dev/serie?IREE?a2bd1c8e875ac8dcd218641e73102249a16c011c38d3775d52d9dd8a9ba324f4) | [131.642 (0.5X)](https://perf.iree.dev/serie?IREE?b780a99e26116acaa224bdfbb951ea06ffbdac06aebd015fddb15eefc7bfb7f7) | [62.439 (1.1X)](https://perf.iree.dev/serie?IREE?6c3eebd478ce05568e03b90fffbaabf0ae95774046d9f492ee53b8e34a6b692d) | | MobileNetV1\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [25.185 (1.0X)](https://perf.iree.dev/serie?IREE?1622e274d5ac570e18826aaec62f223c538583eb2f76e771d24eb2f7785954aa) | [23.167 (1.1X)](https://perf.iree.dev/serie?IREE?38a1120d11abb07ae2e44af50d73bb4dc3ab9d9d6d54dca0b35a1ea74816d4d8) | [17.750 (1.4X)](https://perf.iree.dev/serie?IREE?6600e5c77f343f3727788ac55712340db67660453f0d5b2a78f8a2f00bffa9f2) | | MobileNetV1\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [4.802 (1.0X)](https://perf.iree.dev/serie?IREE?002cd64f66606ef48d9568103412f709d494fbea040a6879b069436ccc106733) | [5.308 (0.9X)](https://perf.iree.dev/serie?IREE?94ff42f10cd47d6a4d48b44dc5e1b51917f104cf16125306ae1cf694c9c8c2d4) | [4.541 (1.1X)](https://perf.iree.dev/serie?IREE?14e8174454310c9b24812dca661319c7b8e78a1175003f56abe8cfa7e7bb9cb9) | | MobileNetV2\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [12.312 (1.0X)](https://perf.iree.dev/serie?IREE?48cac7cf7dea690dd7d8e8669fd5d6f65d1f20c0de1710dc381cf15533354bed) | [15.493 (0.8X)](https://perf.iree.dev/serie?IREE?dcaca9ab1e951e5901176700acd3f87249435eb5f86e050db7acf5e4e956a2b3) | [11.317 (1.1X)](https://perf.iree.dev/serie?IREE?6272e089c33b7c5333b6188b6f61fbb15e7b6a0e9fcd9d54b3b7271cd730e0da) | | MobileNetV2\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [3.711 (1.0X)](https://perf.iree.dev/serie?IREE?9accf20747a0a52c6c6b7da7433c9e9cdf68a813ec6589b781ecb7791a836e34) | [5.332 (0.7X)](https://perf.iree.dev/serie?IREE?97b8402709c23827e768130bf99bb0413af8fadf7b876518e6699988391dc240) | [4.916 (0.8X)](https://perf.iree.dev/serie?IREE?ce780c2ab7c9b837611b5e1dcdbce18e7563fb9d9137e68b5a50bd917a54f83d) | | MobileNetV2\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [21.570 (1.0X)](https://perf.iree.dev/serie?IREE?23e7ffd476616a14cc5b0cabe27332ff71fec9cdc22801b675f8e6349c498814) | [42.827 (0.5X)](https://perf.iree.dev/serie?IREE?f3bd5dc9496be7b61f002132e487918ba87270871a67421c66dc07c54db67a82) | [11.833 (1.8X)](https://perf.iree.dev/serie?IREE?10f2428bc7da79d6d0f23d87caa4cb20ba55d968736b64c6a47c3041be10f641) | | MobileNetV2\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [5.891 (1.0X)](https://perf.iree.dev/serie?IREE?c196cfd95d87ddeb4cb008e055ec417dd805617dd204295c17856ca0f9e0863c) | [9.507 (0.6X)](https://perf.iree.dev/serie?IREE?dccff3d9f2ac906d4abb62647e9720ba736d806e320811f1f038a48b0111486a) | [5.394 (1.1X)](https://perf.iree.dev/serie?IREE?5b41fd88f5fa3c217d024908b57237037d8851b0cba869fb142270cb2fd17ff1) | | MobileNetV3Small\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ c2-standard-60[cpu] | [2.792 (1.0X)](https://perf.iree.dev/serie?IREE?fd46a78e4032c5fa09644bcda90d0d8b73e9196fb89e2458db2838ddf5fd4c16) | [3.334 (0.8X)](https://perf.iree.dev/serie?IREE?3643fcd78839344f79b2abe8185a335331670b2280b32bcb9abef8eb38da1811) | [2.716 (1.0X)](https://perf.iree.dev/serie?IREE?485da7a706b6c0940ef45626ec12ab149da295cc6a3c0a2c63e5a15a952580b4) | | MobileNetV3Small\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [2.930 (1.0X)](https://perf.iree.dev/serie?IREE?069f6917e401e63c9e50c548c70cc699385e6f6908517eb6c79c96e597bf96d7) | [3.517 (0.8X)](https://perf.iree.dev/serie?IREE?d28571a1e8bf10881978bb14a2ec3bc8b98b13759989ef2dc9cd4e0098cbad4d) | [2.800 (1.0X)](https://perf.iree.dev/serie?IREE?c27738e97498c969076d1a2a693322821dd104dbcf7ba6e129ba893584bb0dfd) | | MobileSSD\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [34.705 (1.0X)](https://perf.iree.dev/serie?IREE?0aac8a2a5c45ed0ed35dcd65338a5a414c6beefcdbb0fbb4f299b42d41b639e1) | [39.577 (0.9X)](https://perf.iree.dev/serie?IREE?5220973fd48229657591ff1dcb25fc65ff226cdc18fc8ba4a3f63f97ae0721fb) | [31.118 (1.1X)](https://perf.iree.dev/serie?IREE?d6bfea70085e57a372f18983ddd9f7598b084dc4aac07754c80e4f4f5c4fb407) | | MobileSSD\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [8.603 (1.0X)](https://perf.iree.dev/serie?IREE?4af168ed94d96166f35b8264e160ca1e85a3c6ef3faa08284f447a5613f6ce39) | [10.877 (0.8X)](https://perf.iree.dev/serie?IREE?f6a67dc645b0ccbe999fd99bbb14fc092f22c4037d94e0883b5bbd6a934a3785) | [9.841 (0.9X)](https://perf.iree.dev/serie?IREE?ec20addfc5f284c92b739d0eaf245af0027627de593635539a86709332ae5acf) | | PersonDetect\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ c2-standard-60[cpu] | [0.698 (1.0X)](https://perf.iree.dev/serie?IREE?da589d3a658ddcc4dacaab64c8c7253bab3b0b90fbd35158ba58ed883266d5dc) | [1.293 (0.5X)](https://perf.iree.dev/serie?IREE?16eaa3be7a8f8e5e7c4a9fa0843fe07e74afee4e52bead60aaf5df3c38b308aa) | [0.569 (1.2X)](https://perf.iree.dev/serie?IREE?3283ddd7c21e5db8eea573c2f94ae318c5baa6bf3d9340ba157573937e7b6632) | | PersonDetect\_int8(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [0.771 (1.0X)](https://perf.iree.dev/serie?IREE?77dd6dcff77b2053dbc4cbafc7ca36f8ee5aabdc138b5808830908b037014cc3) | [1.378 (0.6X)](https://perf.iree.dev/serie?IREE?b4c044d54badc2cbccba6ca5e7a34c87662a80ee5a82342965f093c79b502923) | [0.631 (1.2X)](https://perf.iree.dev/serie?IREE?8d8fd2fbd7901ece93ffa5e47c460dd793c4489b5751a15bb0c3e1b8d82073db) | | PoseNet\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [17.853 (1.0X)](https://perf.iree.dev/serie?IREE?0d4e114d66ae2e078076cc40fca5e6af76232c3936effb92d33e23f76f26ede8) | [24.354 (0.7X)](https://perf.iree.dev/serie?IREE?240bed9536f23aa37693e758c317d4edcc131bc420736c2ab4712829c33909a3) | [18.968 (0.9X)](https://perf.iree.dev/serie?IREE?51181aae886260ff3c24d829e8bf9e3a892aa93305321c1012476aace79f9e65) | | PoseNet\_fp32(tflite) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_task(embedded\_elf)[8-thread,full-inference,default-flags] with default @ c2-standard-60[cpu] | [4.119 (1.0X)](https://perf.iree.dev/serie?IREE?a2ebf5883d38f358868199609143debdbb2947b6e0ab6c5b03802cb813022f9f) | [5.909 (0.7X)](https://perf.iree.dev/serie?IREE?922fc7aad073c4a1cf509939a4c85e346b1623e6f0d1741fcdfaab4507f4c446) | [5.170 (0.8X)](https://perf.iree.dev/serie?IREE?1e0197113e1bab228898b4e76067c7c8dcd0faf2b0cf5af9dbb227491de894e4) | | matmul\_256x256x2048\_i8\_i4\_i32\_tile\_config\_default(linalg) [x86\_64-cascadelake-linux\_gnu-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ c2-standard-60[cpu] | [7.598 (1.0X)](https://perf.iree.dev/serie?IREE?641b82f32c47ecd4d02c8c82926118acfce0f530e8728e04a1d593a2876847d2) | [7.579 (1.0X)](https://perf.iree.dev/serie?IREE?572f89c5a8137fcf5856f5d94835535008403ed036c5bd53042ca6df4dddcc1d) | [7.581 (1.0X)](https://perf.iree.dev/serie?IREE?c3a0b8c64c6406c9e4a46d537f2acd4ed2b9f6c191387830c5fcb215cd91d9d0) | | DeepLabV3\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [48.159 (1.0X)](https://perf.iree.dev/serie?IREE?95281c38b844a3b0ea1964e9634e7a8e2b40025936e3402ff2902be01dbd31b7) | [83.911 (0.6X)](https://perf.iree.dev/serie?IREE?5bbd93c48aea71957fe659dc2fd4645dee8557e76e580f43395b4ec33f73e7c1) | [44.036 (1.1X)](https://perf.iree.dev/serie?IREE?f17944b7339d0d84be14cd71d31c10b495df98114d5af917259df75540551fa4) | | DeepLabV3\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [50.218 (1.0X)](https://perf.iree.dev/serie?IREE?4cc57db28e42e4b50f3d234a99faee5e7d48ac787d70f106ed2260e4160f27fc) | [85.307 (0.6X)](https://perf.iree.dev/serie?IREE?a3b21d18d3fa90747450ec5a36386400b6f48c5ab5e5393e05138bb5ef5daadb) | [44.046 (1.1X)](https://perf.iree.dev/serie?IREE?d44c3fbc39f410214516a4c591f879e0ac9454b33a970ff63953fc00f2ec465b) | | DeepLabV3\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [30.108 (1.0X)](https://perf.iree.dev/serie?IREE?ed4f76526e499d8e959237456899cc74fa4bab29674b0ba083c5ce38edc61fab) | [50.024 (0.6X)](https://perf.iree.dev/serie?IREE?9f1f23ea9084f82c09b20fa3b41f7622b374fbcd47917db5876e323302efd0b2) | [27.630 (1.1X)](https://perf.iree.dev/serie?IREE?5343c96ad4bb05804680ca8a51d26bc1ffc4e1d16348e923b4ea234ceb6f94b4) | | GPT2\_117M\_TF\_1X1XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [91.346 (1.0X)](https://perf.iree.dev/serie?IREE?712d1d8286ecd1d7d66c2f4426924cff01be3c71d3512d1f675fc3560487113b) | [22.282 (4.1X)](https://perf.iree.dev/serie?IREE?3aa9b99cc5acee247d6642af968bf90ba7d1a0627d8cd3c573f0e7a2522bf0ef) | [22.550 (4.1X)](https://perf.iree.dev/serie?IREE?d43fc641fce6a72ff3fe58571f3c55e36e65ef7fc868f197554cdd9a5a451015) | | GPT2\_117M\_TF\_1X1XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [92.610 (1.0X)](https://perf.iree.dev/serie?IREE?71eae757691075543390b054227af100cfbb850c70094713e12f2c48c2f7db07) | [21.996 (4.2X)](https://perf.iree.dev/serie?IREE?462fad0e00922d12af0343ce5929213b26c49c383fd5c69dc08c9ca5145c0293) | [22.050 (4.2X)](https://perf.iree.dev/serie?IREE?3b12e9908a7263dea59779315d80b3b215f17a287e84b6cb3a73ac2b5faa1d0f) | | GPT2\_117M\_TF\_1X1XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [51.844 (1.0X)](https://perf.iree.dev/serie?IREE?a20f9c8cbe11916179b5a347f4e60d1c4e37519719e1aeeface855fe7fc4740f) | [21.947 (2.4X)](https://perf.iree.dev/serie?IREE?fc795521dc3d23e4e8665c02b111d361e4da9dd54e38885995e100d4e0063231) | [21.976 (2.4X)](https://perf.iree.dev/serie?IREE?f64e2e4991de95b0282191703bcc5eade1188cbc1dc5012fe7a377d7300e0954) | | GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [140.318 (1.0X)](https://perf.iree.dev/serie?IREE?8f03b8167746d6dfe9237cf890831c5521ab5169b0892d660c2f817c5f579223) | [28.274 (5.0X)](https://perf.iree.dev/serie?IREE?2d8906fd07aa37db29f8d4959028469dd3d384df8312351839cc51868e0932aa) | [27.140 (5.2X)](https://perf.iree.dev/serie?IREE?8ae7cfed6678287118515c19784beaae637b5bfa1a259ee0c40d0ae15de02f32) | | GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [123.177 (1.0X)](https://perf.iree.dev/serie?IREE?ad83887341f3e360b8a4be6c5683e012c82aeb10d65482cfde8e842bc144a48e) | [29.904 (4.1X)](https://perf.iree.dev/serie?IREE?458172413fb7700f4a84ae51402399781a76ad89398e6b6d0ac20ab605f1fd1d) | [29.204 (4.2X)](https://perf.iree.dev/serie?IREE?d9f100bcdbbfe35bada2541180c89460cc12b0e8a17c3c0126af94dd3e194f04) | | GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [76.381 (1.0X)](https://perf.iree.dev/serie?IREE?38f7c3ef079798f2116c2bdff47240a6a261b066b99b8d12fe8e7da255c0e1f3) | [27.090 (2.8X)](https://perf.iree.dev/serie?IREE?9f19b85505311dadfe8c5211a8a50f75b77b35bc3f0c5031be7c3d7a9d1581f7) | [26.131 (2.9X)](https://perf.iree.dev/serie?IREE?10216d6baf8d3e228a42f5849a691954104e8fc91e514be1c63736ef737f59d5) | | MobileBertSquad\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [693.803 (1.0X)](https://perf.iree.dev/serie?IREE?dc2023c6113c87aad59f2b49214ab2995b32c7ba040b314e890ea2ec7081f90b) | [456.211 (1.5X)](https://perf.iree.dev/serie?IREE?2eef43089ae6d4ec5425dd2e53110a2b2627cc9a8a85dc8dd295033ed7177572) | [354.862 (2.0X)](https://perf.iree.dev/serie?IREE?d4572856894af9013e311991e4371c81498ee30b1fc90ee840632d1a3a512193) | | MobileBertSquad\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [699.711 (1.0X)](https://perf.iree.dev/serie?IREE?ab3be2a007f3201e419112cd2bf753bbbe4e15431946411433a61ab0e34cdfca) | [456.048 (1.5X)](https://perf.iree.dev/serie?IREE?8c78d47df130263e959b534abeb7045f86dc5b359cea445abd85ac2769efa6ee) | [356.451 (2.0X)](https://perf.iree.dev/serie?IREE?cca718432d630f48a03660753dbea3c60120aea2692fab0fccf6a4928be7a247) | | MobileBertSquad\_fp32(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [392.858 (1.0X)](https://perf.iree.dev/serie?IREE?c1b5a77b70decd14b1d3268ad2a167631422bff95c8f8c126dc7a876bd3c0632) | [273.925 (1.4X)](https://perf.iree.dev/serie?IREE?80b58632ef882cdaec07c38dddc972ce9363cc435ecba22530b109479b4a59b8) | [215.603 (1.8X)](https://perf.iree.dev/serie?IREE?04f958179d9bc04eca09f2ad518a3cb494931445f23fcc2791b2d9fcee5cf1bc) | | MobileBertSquad\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [1116.736 (1.0X)](https://perf.iree.dev/serie?IREE?df6786c3bd20d93e1230f8b59212221a7e9de0eefdc39ac2f7192b76047d2803) | [700.881 (1.6X)](https://perf.iree.dev/serie?IREE?33e5e79843a382c4a07d443902369a96b08b2d274fe00db77c6f510838ee6820) | [304.209 (3.7X)](https://perf.iree.dev/serie?IREE?5a9829035177db026ff3371238afa1f319a3b715e22ea7d1670c8fab8c243d94) | | MobileBertSquad\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [1116.426 (1.0X)](https://perf.iree.dev/serie?IREE?106ccd69f92add8c01ecfa00b551ae901a5e9864595601ff75e090a03c97dc49) | [690.901 (1.6X)](https://perf.iree.dev/serie?IREE?caa6101a583b18f027528852c89e896d5ff42383141c5063f664ed6b245f6943) | [306.575 (3.6X)](https://perf.iree.dev/serie?IREE?dbaa3dbc7fba073c6e934eb505c4679fe625bbfdeb7e2316960c659f6eb8b2e6) | | MobileBertSquad\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [574.021 (1.0X)](https://perf.iree.dev/serie?IREE?8156b68796001f010990cd4da026415ee8875a0ccf609258df8b27c1cd5ed71e) | [379.817 (1.5X)](https://perf.iree.dev/serie?IREE?84a126639a5fa770b83493257cd4f61a235bb813a1213baef295d4c07157ae86) | [180.198 (3.2X)](https://perf.iree.dev/serie?IREE?9cff4a1b1873b4cda93168fc674fdc046c7b6640f81283cae57c871afbbe216d) | | Vit\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [2104.560 (1.0X)](https://perf.iree.dev/serie?IREE?1bb02b9cb5407a193c5ad68d57ba004d6694ae1b9f3b4af974af7197f30f9082) | [1102.476 (1.9X)](https://perf.iree.dev/serie?IREE?a57eb6fb2cf6fbf61f9141e63784cc356153766042d0890e6caee565eadcbf21) | [302.309 (7.0X)](https://perf.iree.dev/serie?IREE?3263426782173c417a4205ee460ccf4acb939c53397da8ae06f8ebf3f7228f87) | | Vit\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [2106.298 (1.0X)](https://perf.iree.dev/serie?IREE?5f2fe9c7dc19b8dda9300eb881b22481951e7c8f9aaaef2923bf31cea6b4d812) | [1101.930 (1.9X)](https://perf.iree.dev/serie?IREE?c3116c8939132a965cb9aaf18bd682f915bfcd780109926e151083b122722e52) | [306.795 (6.9X)](https://perf.iree.dev/serie?IREE?893537d80a1d230ac7751f901899b02d29ad6a179afa59b16b16e712a2fab297) | | Vit\_int8(tflite) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] | [1125.408 (1.0X)](https://perf.iree.dev/serie?IREE?8c2249d8f9c199d56ae43e1d4d6b288194fa1e3b31914cc88d210deccad3d351) | [616.219 (1.8X)](https://perf.iree.dev/serie?IREE?d9648731b2f404580bedd33800f06352af624b5bf0e19530224040279061154f) | [184.748 (6.1X)](https://perf.iree.dev/serie?IREE?6c111c114ceccecfdeb1b3608ec4701ac6c62fa29abfa7270a6737f92c94cb0b) | | matmul\_256x256x2048\_i8\_i4\_i32\_tile\_config\_default(linalg) [armv8.2-a-generic-linux\_android29-llvm\_cpu] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] | [12.148 (1.0X)](https://perf.iree.dev/serie?IREE?a694805fd2aa24f7bb3464e817ade1eda09588928e5b168947eef7e6b5ac8dee) | [10.041 (1.2X)](https://perf.iree.dev/serie?IREE?e8065d1ddf99eb564720a2efd4c3b45e2553b6b34f71a857501a49c39c1d8f82) | [1.325 (9.2X)](https://perf.iree.dev/serie?IREE?fe0a953188f398da446a84e74ad069d4029568c0a02709b84bef8922533bb14a) |

Regressed Latencies 🚩

Benchmark Name Average Latency (ms) Median Latency (ms) Latency Standard Deviation (ms)
GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu][experimental-flags,no-dt] local\_sync(embedded\_elf)[full-inference,default-flags] with default @ pixel-6-pro[big-cores] 140.318 (vs. 125.543, 11.77%↑) 140.314 0.255
GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu][experimental-flags,no-dt] local\_task(embedded\_elf)[2-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] 76.381 (vs. 71.942, 6.17%↑) 76.365 0.189

Improved Latencies 🎉

Benchmark Name Average Latency (ms) Median Latency (ms) Latency Standard Deviation (ms)
GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu][experimental-flags,no-dt] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] 123.177 (vs. 137.687, 10.54%↓) 123.103 0.410
GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu][default-flags,dt-uk] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] 29.204 (vs. 31.525, 7.36%↓) 29.286 0.563
GPT2\_117M\_TF\_1X4XI32(stablehlo) [armv8.2-a-generic-linux\_android29-llvm\_cpu][experimental-flags,dt-only] local\_task(embedded\_elf)[1-thread,full-inference,system-scheduling] with default @ pixel-6-pro[big-cores] 29.904 (vs. 31.897, 6.25%↓) 29.717 0.804

[Top 3 out of 10 results showed]

No improved or regressed compilation metrics 🏖️

For more information:

Source Workflow Run