search

FZJ-PGI-12 / QAOA.jl

Apache License 2.0
13 stars 5 forks source link

Add benchmark scripts for reproducibility #29

Closed babreu-ncsa closed 1 year ago

babreu-ncsa commented 1 year ago

The Benchmark section of README.md shows a performance comparison between QAOA.jl and PennyLane. Although the data is in the assets folder, having the PennyLane script would be essential for direct reproducibility. Could you please add the script that was used for benchmarking? https://github.com/openjournals/joss-reviews/issues/5364

timbode commented 1 year ago

The script is actually here: https://github.com/FZJ-PGI-12/QAOA.jl/blob/master/notebooks/benchmark.ipynb

I will again make sure it's working out of the box now.

timbode commented 1 year ago

@babreu-ncsa The notebook has been updated here: https://github.com/FZJ-PGI-12/QAOA.jl/blob/29-add-benchmark-scripts-for-reproducibility/notebooks/benchmark.ipynb

It should now be in a state where you just have to do Run all to reproduce the plot. Note, however, that the case with p=5 and n=20 will take something like 3 hours for PennyLane (on a M1 MacBook Pro, that is). So you might want to change the nrange to something more economical.

Please let me know if you get results which are roughly in the same ball park. I got slightly different ones on my new machine with a different Julia version, which is to be expected though, I guess. If you give me green light, I'll do the merge.

Thanks!

timbode commented 1 year ago

I'll do the merge anyway now, should be fine.

babreu-ncsa commented 1 year ago

@timbode thank you, it looks good to me!