openxla / xla

A machine learning compiler for GPUs, CPUs, and ML accelerators
Apache License 2.0
2.41k stars 361 forks source link

[xla:ffi] Use lazy decoding for AnyBuffer #14319

Closed copybara-service[bot] closed 5 days ago

copybara-service[bot] commented 1 week ago

[xla:ffi] Use lazy decoding for AnyBuffer

Make external AnyBuffer consistent with internal one.

name old cpu/op new cpu/op delta BM_AnyBufferArgX1 12.8ns ± 9% 11.9ns ±16% -7.35% (p=0.000 n=80+80) BM_AnyBufferArgX4 19.5ns ± 6% 13.1ns ±12% -32.89% (p=0.000 n=80+79) BM_BufferArgX1 13.1ns ± 6% 13.1ns ± 6% ~ (p=0.616 n=80+79) BM_BufferArgX4 20.3ns ± 7% 20.0ns ± 6% -1.81% (p=0.000 n=80+79) BM_BufferArgX8 41.5ns ± 4% 36.6ns ± 6% -11.78% (p=0.000 n=79+80) BM_TupleOfI32Attrs 66.2ns ± 2% 66.2ns ± 1% ~ (p=0.250 n=62+59)