qiboteam / qibojit-benchmarks

Benchmark code for qibojit performance accessment
Apache License 2.0
2 stars 3 forks source link

Add adiabatic evolution benchmark #35

Closed stavros11 closed 2 years ago

stavros11 commented 2 years ago

Following the last comment in #26, this adds a script for qibo adiabatic evolution benchmarks. It is using the H = t TFIM + (1 - t) X Hamiltonian as in the original paper. Note that for time evolution we can have scaling plots as a function of the number of qubits or the time step dt.

Here are some first results from the qibomachine:

evolution_scaling_total_dry_time.pdf evolution_scaling_total_simulation_time.pdf

I don't understand the noise (the machine is not being used) and why CPU is faster than GPU, as this was not the case in the original paper.

mlazzarin commented 2 years ago

Thanks! Concerning the plots, how many qubits did you use?

stavros11 commented 2 years ago

Thanks! Concerning the plots, how many qubits did you use?

This is for 10 qubits, so normally I would expect the CPU to be faster. However, I am confused with the figure we used in the original paper (Fig. 8 in arXiv:2009.01845) which, at least according to the text, is also for 10 qubits and the GPU appears faster.

Here are some plots vs nqubits using the script from this PR: evolution_scaling_nqubits_total_dry_time.pdf evolution_scaling_nqubits_total_simulation_time.pdf

These have a more expected scaling, excluding cuquantum which is a bit off for small circuits but appears to be slightly better than cupy for large, in contrast to what we found in circuit benchmarks. Btw, all this is using the Trotter decomposition so it is equivalent to simulating a circuit of unitary gates, because this is how this evolution is coded in qibo. This might explain why cuquantum has better scaling.

stavros11 commented 2 years ago

@scarrazza here are the updated plots from the last post with the CPU lines completed

evolution_scaling_nqubits_total_dry_time.pdf evolution_scaling_nqubits_total_simulation_time.pdf

Something I just noticed and is not clear from the plots is that the GPU lines go up to 29 qubits instead of 30, as in circuit simulation. This shouldn't be the case, since evolution with Trotter decomposition should be evaluated through a circuit. I will try to check if there is something that allocates additional GPU memory and the 30 qubit run fails.

scarrazza commented 2 years ago

@stavros11 shall we merge this?

stavros11 commented 2 years ago

@stavros11 shall we merge this?

Yes, it would be good to merge. The evolution benchmark script works ok. I'm not sure if the bash scripts are ideal and in one-to-one correspondance with the plots we are finally having on the paper, but I will revise all bash scripts to match the paper in #37 so I can fix this there. It would be good to merge so that I can update #37 with this.

scarrazza commented 2 years ago

Good, thanks!