Create benchmark suite

[MilesCranmer]

This creates a simple benchmark for catching performance regressions on small, tightly controlled problems. To kick things off I added the multivariate first-order optimizers including Adam, AdaMax, BFGS, LBFGS, NGMRES, ConjugateGradient, GradientDescent, and MomentumGradientDescent.

The specific benchmark I added is fitting the Gabor function (2 parameters) to a noisy dataset.

It's compatible with any library supporting BenchmarkTools.jl. You can run the benchmark with:

using Optim
using Pkg
Pkg.activate("benchmark")
include("benchmark/benchmarks.jl")

results = run(SUITE)
@show results

I also add a GitHub action to run AirspeedVelocity.jl on this benchmark for any new PR. It will automatically print out the performance and load time comparison of master in a GitHub comment on the PR.

The current benchmarks across revision history are as follows:

	benchmarks	v1.9.2	v1.8.0	v1.7.6
multivariate/solvers/first_order/AdaMax	0.212 ± 0.00058 ms	0.212 ± 0.00067 ms
multivariate/solvers/first_order/Adam	0.213 ± 0.00058 ms	0.212 ± 0.00054 ms
multivariate/solvers/first_order/BFGS	0.0845 ± 0.00029 ms	0.0843 ± 0.00029 ms	0.0844 ± 0.00025 ms	0.0871 ± 0.00033 ms
multivariate/solvers/first_order/ConjugateGradient	0.273 ± 0.00096 ms	0.273 ± 0.0017 ms	0.274 ± 0.00093 ms	0.291 ± 0.0012 ms
multivariate/solvers/first_order/GradientDescent	0.0842 ± 0.00029 ms	0.0841 ± 0.00029 ms	0.0841 ± 0.00033 ms	0.0868 ± 0.00029 ms
multivariate/solvers/first_order/LBFGS	0.0853 ± 0.00033 ms	0.0851 ± 0.00033 ms	0.085 ± 0.00033 ms	0.088 ± 0.00033 ms
multivariate/solvers/first_order/MomentumGradientDescent	0.0842 ± 0.00033 ms	0.0841 ± 0.00042 ms	0.0842 ± 0.00033 ms	0.0868 ± 0.00029 ms
multivariate/solvers/first_order/NGMRES	0.74 ± 0.0034 ms	0.741 ± 0.0037 ms	0.742 ± 0.0026 ms	0.777 ± 0.0031 ms
time_to_load	0.981 ± 0.009 s	0.99 ± 0.0028 s	0.98 ± 0.02 s	0.364 ± 0.0065 s

Meaning there are no immediate performance regressions. However the load time has clearly increased from v1.7 to v1.8 which #1081 will fix.

---

To create this table above, run the following in bash:

# Install AirspeedVelocity:
julia -e 'using Pkg; pkg"add AirspeedVelocity"; pkg"build AirspeedVelocity"'

# Run benchmark over commit history
benchpkg Optim --rev=benchmarks,v1.9.2,v1.8.0,v1.7.6 -s benchmark/benchmarks.jl --exeflags='--threads=4' --path=.

# Generate the table:
benchpkgtable Optim --rev=benchmarks,v1.9.2,v1.8.0,v1.7.6

We can also get the memory:

	benchmarks	v1.9.2	v1.8.0	v1.7.6
multivariate/solvers/first_order/AdaMax	0.064 k allocs: 2.62 kB	0.064 k allocs: 2.62 kB
multivariate/solvers/first_order/Adam	0.064 k allocs: 2.62 kB	0.064 k allocs: 2.62 kB
multivariate/solvers/first_order/BFGS	0.061 k allocs: 3.44 kB	0.061 k allocs: 3.44 kB	0.061 k allocs: 3.44 kB	0.066 k allocs: 3.83 kB
multivariate/solvers/first_order/ConjugateGradient	0.207 k allocs: 9.09 kB	0.207 k allocs: 9.09 kB	0.207 k allocs: 9.09 kB	0.23 k allocs: 10.9 kB
multivariate/solvers/first_order/GradientDescent	0.054 k allocs: 2.92 kB	0.054 k allocs: 2.92 kB	0.054 k allocs: 2.92 kB	0.059 k allocs: 3.31 kB
multivariate/solvers/first_order/LBFGS	0.083 k allocs: 5.48 kB	0.083 k allocs: 5.48 kB	0.083 k allocs: 5.48 kB	0.088 k allocs: 5.88 kB
multivariate/solvers/first_order/MomentumGradientDescent	0.055 k allocs: 3 kB	0.055 k allocs: 3 kB	0.055 k allocs: 3 kB	0.06 k allocs: 3.39 kB
multivariate/solvers/first_order/NGMRES	0.646 k allocs: 0.0444 MB	0.646 k allocs: 0.0444 MB	0.646 k allocs: 0.0444 MB	0.719 k allocs: 0.05 MB
time_to_load	0.19 k allocs: 17.1 kB	0.19 k allocs: 17.1 kB	0.19 k allocs: 17.1 kB	0.19 k allocs: 17.1 kB

with --mode=memory in the benchpkgtable command. If you want I can put that table in the GitHub action as well?

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.79%. Comparing base (78ab1f4) to head (005eb7f).

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #1084 +/- ## ========================================== - Coverage 84.85% 84.79% -0.06% ========================================== Files 46 46 Lines 3480 3480 ========================================== - Hits 2953 2951 -2 - Misses 527 529 +2 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[pkofod]

Very useful, thanks!