Add SPSA optimization method

[ankit27kh]

The Simultaneous Perturbation Stochastic Approximation (SPSA) optimisation method is a faster optimisation method.

If the number of terms being optimized is p, then the finite-difference method takes 2p measurements of the objective function at each iteration (to form one gradient approximation), while SPSA takes only two measurements

It is also naturally suited for noisy measurements. Thus, it will be useful when simulating noisy systems.

The theory (and implementation) for SPSA is:

• Described in James C. Spall, “An Overview of the Simultaneous Perturbation Method for Efficient Optimization”, 1998;

Furthermore, it is implemented:

• In the noisyopt package, specifically see the source code here;
• In Qiskit, see the SPSA optimizer documentation.

More information: https://www.jhuapl.edu/SPSA/

[mtthss]

Sounds like a nice contribution, do you want to take a stab at it?

[ankit27kh]

@mtthss if no one is working on it, I would like to try. Can you provide me with some general points before I start, like which files to update, what to take care of etc., as I haven't contributed to optax before.

[lockwo]

Is there any updates on this? I have previously worked with SPSA in TF (https://github.com/tensorflow/quantum/pull/653) and would be interested in working on this but don't want to do redundant labor.

[ankit27kh]

Hi @lockwo, are you still interested? If you can implement SPSA, it'll be of great help!

[fabianp]

@ankit27kh : since there hasn't been activity for this in a year, I think it's safe for you to take over.

if you end up contributing this example, please do so to the contrib/ directory. Thanks!