Add circuit benchmark script

qiboteam / qibojit-benchmarks

Benchmark code for qibojit performance accessment

Apache License 2.0

2 stars 3 forks source link

Add circuit benchmark script #1

Closed stavros11 closed 3 years ago

stavros11 commented 3 years ago

Adds script that benchmarks performance of various qibo backends (qibojit, qibotf, etc.) for circuit simulation. The scripts are based on the benchmark from the qibo repository which builds the circuit using the qibo API.

TODO:

[ ] Expand the list of available circuits (for example based on table 1 of the HyQuas paper)
[ ] Include a README with documentation on:
- [x] how to run benchmarks
- [ ] how to read/plot the log files
- [x] how to install each qibo backend
[ ] Add bash scripts that execute the benchmarks on CPU and GPU (perhaps customized for specific machines).
[x] Add tests.
[x] Optional: Change print messages to a custom logger (similar to what we have in qibo).
[ ] Optional: Add a benchmark that calls the custom operators of qibotf/qibojit directly, without building qibo circuits. This will give an estimate of qibo's overhead.

I will edit this post with a complete list of the circuits we would like in order to track progress.

scarrazza commented 3 years ago

@stavros11 many thanks for this. I believe that we can start sharing this repo with the nvidia team, please have a look at #2.

stavros11 commented 3 years ago

@stavros11 many thanks for this. I believe that we can start sharing this repo with the nvidia team, please have a look at #2.

Although this is still under development, it is possible to run benchmarks at least for the existing circuits (QFT, example variational circuit and specific gate benchmarks) so it should be okay to share. Yesterday I made some modifications to improve logging and printing of messages, added tests (not complete) and I just pushed some example bash scripts that I used to generate the QFT tables I posted above.

In terms of core code and logging functionality I believe this is almost complete. In the next days I will try to expand the list of circuits and the tests.

scarrazza commented 3 years ago

@stavros11 thank you very much, looks pretty good. Maybe we can merge this PR and open an issue to keep track of the other points?

stavros11 commented 3 years ago

@stavros11 thank you very much, looks pretty good. Maybe we can merge this PR and open an issue to keep track of the other points?

Thanks for reviewing. Sure, we can merge this now and expand the available circuits in a new PR.

I would open an issue for circuits and a separate one for the non-qibo benchmarks in case we want to test this case too.

scarrazza commented 3 years ago

Perfect, lets go for it.

stavros11 commented 3 years ago

I opened some issues for the points I summarized in the first post which are not yet implemented here. @scarrazza, if you prefer we can merge this now and I will address these issues in different PRs.

scarrazza commented 3 years ago

Perfect, thank you very much.