Investigate Gradient Descent

[NicholasSynesi]

[NicholasSynesi]

@as2457 So this is my latest attempt for gradient descent, using the best model for eta I could find. It's no longer a linear regression as I found there was too much noise early on if I began too high, and not enough movement if I began with a smaller value. Now the 1 - overlap is decreasing after ~ 10 iterations which is better in that regard, however even over 40,000 iterations it still hasn't converged. What do you think the best course of action is here, run more iterations, or something else?

[AdamSmith-physics]

@NicholasSynesi This is looking pretty good. Maybe you could keep the learning rate a little bit higher at later iterations?

How does this compare with the polar method? Maybe you could plot this again with all curves the same colour, then add the curves from the polar decomposition in another colour. This would be a good way to compare the methods.

It could be that we need to go for longer still. If this is the case, this is not such a problem since we do the optimisation once and then can test a lot of things. An outlook for your thesis would then be that the optimisation problem is hard and needs further investigation.

[AdamSmith-physics]

In the end, because of the time limitations, we might just have to go with the best we're able to achieve, even if we can't get it to converge. This is still an approximation to the state, just not globally optimal.

[NicholasSynesi]

So I've tried to slightly tune eta and it doesn't look too bad, but I can't seem to get it to converge within 30-40k sweeps. I don't think the HPC is working correctly at the moment so it's slightly tricker to get results but when everything is fixed there I think my best bet is to let it run for as many sweeps as I can to get convergence

[LucaYoy]

Ya Imma do to 80k for both polar and gd with my linear decrease in eta

[AdamSmith-physics]

Honestly, this looks great! Getting 10^(-7) for the infidelity is much better than before!

Go big for the number of iterations, do a few runs and then select the best one as the final optimised state (even if it looks like there is more that could be done). Then you should compare GD with Polar and then pick the best method for going forward.

[LucaYoy]

@as2457 @NicholasSynesi Some updates for me.

First, I run N=8 M=5 sweeps = 80000 to see if I can get it to converge, This is the result. It took HPC approx 30 min per optimisation so everything was done in 1 hour.

Next I run N=14 M=5 sweeps= 6000, this is what I got. Took HPC about 3 hours per optimisation so a total of 3 hours for everything using parallelism fully

im using constant eta=0.3; this model looked good for N=8 M=5 sweeps=5000 on my PC however we can clearly see it is not good when going to sweeps=80000. Conclusion: Still working on eta tuning

[NicholasSynesi]

Do you wanna try the eta tuning I use and see if that gets anywhere? I can’t run anything on the HPC until tonight so you can play around with it if you’re still stuck

[LucaYoy]

Yes I have one more idea of a linear decrease and then ill try yours

[AdamSmith-physics]

I would also check your code for bugs if you are getting different results on your laptop and HPC. Make sure you try run the HPC Python script on your laptop to test it first.

[LucaYoy]

So like this is now for 0.3 --> 0.05 linearly until sweeps//4 then constant

Run on HPC sweeps=80000

Run on my PC sweeps=5000

Now using @NicholasSynesi method for eta sweeps=80000 on HPC

Also we have N=14 M=5 sweeps=60000 using Nich method for eta:

[LucaYoy]

Found a mistake I think now above should be fine

[AdamSmith-physics]

What was the mistake? Glad you sorted it

[LucaYoy]

It was just getting confused between different parameters. Im updating the above post as I gather more data. @NicholasSynesi your eta method for N=14 M=5 sweeps=60000 is now plotted in above comment

[LucaYoy]

New Data from me

I did the comparison tests again however using polar method and sweeps=100000

Comparing Models fixed N=14 M=5

Comparing qubits 8,10,12,14 fixed M=5

Comparing layers 3,4,5,6,7 fixed N=14 M=5

Seems like for some of them im converging whereas for other not really. Also notice the pattern for comparing qubits plots. It seems like for qb 10,12,14 im starting to conv but for qb 8 im not which is counter intuitive... or those are just local minima...