search

openxla / community

Stores documents and resources used by the OpenXLA developer community
Apache License 2.0
105 stars 23 forks source link

[RFC] Common Benchmark Suite #77

Closed pzread closed 1 year ago

pzread commented 1 year ago

This is the subsequent RFC of the high-level benchmarking strategy for OpenXLA #75 (in review).

It proposes to create a benchmark suite to fulfill the benchmarking requirements in the strategy.

The temporary links to the high-level strategy RFC will be updated once #75 is merged.

pzread commented 1 year ago

Will someone be able to help add suitable reviewers? 🙂 (maybe include @jpienaar @GMNGeoffrey @julianwa @mariewhite @reichlfl).

pzread commented 1 year ago

Kindly ping for visibility : ) Also tag @powderluv who might be interested

joker-eph commented 1 year ago

Nice work, looking forward to this! :)

I just left a few nit comments mostly, thanks.

pzread commented 1 year ago

Thanks for all the reviews and I think we don't have a major objection to this idea 🙂 I added a section in the bottom to list the remaining open questions and will track them in the new benchmark repository.

pzread commented 1 year ago

Hi @theadactyl Would you be able to help merge this RFC and create the repository openxla-benchmark as the next step?

GMNGeoffrey commented 1 year ago

I can merge. Will look into the new repo