Performance 2 - Linear Models

[andresrossb]

benchmarking of linear models

[cortner]

done a bit of rewriting of rrule_evaluate. very promising. But still need to insert some final files before multiple properties works again as expected. Basically I want to take the opportunity of not only optimize the codes but also remove all the duplicate codes in the evaluator.

[cortner]

P.S.: there was another nasty performance bug in rrule_evaluate so we are definitely recovering signifant performance optimizations. Not quite the factor 3 of evaluate that I was hoping but definitely in the factor 4-5 range.

[cortner]

Andres - I had to change the test_multiprop.jl tests a bit. Would be worth it if you take a closer look there in case this propagates into your ACEflux. Basically, I now define contract(a::Number, b::AbstractVector) = a * b), which conflicts with how you've used it. But this is really important to allow a generic use-case of the adjoints...

[cortner]

@andresrossb should be ready for you to play :)

[cortner]

@andresrossb ... I'm trying to catch up again. What left to do here, just revisit the benchmark suite, and make sure it runs ok? At least on our AMD EPYC server?

[cortner]

I think

Pgroup["adjoint_EVAL_D1"] = BenchmarkGroup()

should be replaced and _D1 no longer exists, there should now be a single adjoint_EVAL_D.

[cortner]

Just asking the question for now, let's discuss: should the following be in ACE or elsewhere? E.g. ACEfit.jl?

#zygote calls
Pgroup["site_energy"] = BenchmarkGroup()
Pgroup["Zygote_grad_params"] = BenchmarkGroup()
Pgroup["Zygote_grad_config"] = BenchmarkGroup()
Pgroup["Zygote_grad_params_config"] = BenchmarkGroup()

#loss functions
Pgroup["eval_full_loss"] = BenchmarkGroup()
Pgroup["eval_energy_loss"] = BenchmarkGroup()
Pgroup["der_full_loss"] = BenchmarkGroup()
Pgroup["der_energy_loss"] = BenchmarkGroup()

[cortner]

maybe keep it to 1, 2, 4.

Pnumprops = [1, 2, 3, 4] 

[cortner]

I believe this is from my basis benchmarks?

# degrees = Dict(2 => [7, 12, 17],
#                3 => [7, 11, 15],
#                4 => [7, 10, 13], 
#                5 => [7, 9, 11] )

I'd like to reduce this a bit. How about

# degrees = Dict(2 => [10, 17],
#                3 => [9, 15],
#                5 => [7, 11] )

or even less than that .. otherwise the benchmark reports get too long.

[andresrossb]

Just asking the question for now, let's discuss: should the following be in ACE or elsewhere? E.g. ACEfit.jl?

#zygote calls
Pgroup["site_energy"] = BenchmarkGroup()
Pgroup["Zygote_grad_params"] = BenchmarkGroup()
Pgroup["Zygote_grad_config"] = BenchmarkGroup()
Pgroup["Zygote_grad_params_config"] = BenchmarkGroup()

#loss functions
Pgroup["eval_full_loss"] = BenchmarkGroup()
Pgroup["eval_energy_loss"] = BenchmarkGroup()
Pgroup["der_full_loss"] = BenchmarkGroup()
Pgroup["der_energy_loss"] = BenchmarkGroup()

gradient(m->sum(sum(gradient(x->site_energy(m,x), cfg)[1]).rr), LM) is not running on my end. There is a contract with a Chainrules.ZeroTangent() for some reason I don't yet understand. This is when running Pgroup["Zygote_grad_params_config"] = BenchmarkGroup()

[andresrossb]

Did bm_linear.jl run for you?

[cortner]

Didn’t try yet

[cortner]

I've made some changes to the benchmarks along the lines suggested above. The benchmarks now run and I think this represents a good first benchmark suite. If you are happy with those changes then we can merge. Happy to discuss though.

[cortner]

@andresrossb - when you have a moment can you let me know whether you are happy to merge this or wanted to do something else to finish this PR? Thank you.