API++, CMA-ES integration, efficiency, formatting.

[wbrickner]

Hello!

I'm a little unsure if the library works now, or if the CMA-ES stuff is correct.

Major changes:

• Restructured the public API to be "modern", idiomatic rust, nudging us towards releasable package.
• • The typed builder pattern prevents any API misuse by making it impossible, and assembly of large structs is costless.
• • API feels elegant to me but this is a matter of taste, would like to hear your feedback. The CMA-ES options are a bit of a disaster as I don't really understand the domain very well.
• Improved the efficiency of a few algorithms.
• Added population-level parallelism on the side of eant2.
• Uniform code formatting

Remaining work (in my view):

Required:

• Perfect the public API
• Fix any integration issues between eant2 and cmaes packages
• Ensure the thing actually works™ (fix mutation bugs I saw mentioned elsewhere)
• Update documentation to reflect the various changes.

Nice to have:

• Use object pools to amortize allocations, reducing overhead and improving CPU cache locality.
• Merge modifications to cme that allow us to further reduce buffer copying inside eant2
• Improve runtime complexity of deduplication algorithm, etc + parallelize additional steps which are currently serial.
• Tune the usage of parallelism to ensure no significant overhead cost is incurred (rayon can sometimes have significant startup costs depending on usage patterns, fine for now)
• It would be really powerful (and good for my use-case!) to be able to tune opaque parameters alongside network parameters. A contrived example is a network which controls the motion of a robotic arm to catch an object, like in the original EANT paper. What is the optimal length of the forearm, the fingers, and the interplay therein? Who knows! Adding options for an opaque set of parameters to be co-optimized by CMA-ES would be really useful.

Thank you for all your help, I hope the changes I've made are valuable and not too intrusive!

[pengowen123]

Thanks for the PR! It looks good overall, but I will be able to take a closer look and leave actual review comments tomorrow.

[pengowen123]

The API looks good so far, although the CMA-ES termination criteria parts will need some changes. The main thing to note here is that CMA-ES is practically guaranteed to terminate eventually. Therefore, the main reasons to configure its termination criteria are:

• as problem-dependent parameters (e.g., the target function value),
• to bound iteration to prevent excessive running time (e.g., max generations or max function evals), and
• if the default values for the other termination criteria are interfering with solving a problem properly by causing early termination.

The first case is generally useful and should be exposed in the API.

The second case also makes sense to be configurable by the user because it allows them to decide how much computing resources to allocate to CMA-ES to optimize each network. If they don't mind the solution being suboptimal, a lower value can be set for these termination criteria. But if they only care about getting a good solution and don't mind it taking longer to run, they can set the values for these termination criteria to None to disable them completely (this is probably a good default).

However, the third case is both rare and difficult to detect automatically. Therefore, configuration of most of the termination criteria is unnecessary. A good set to start with would be: FunTarget, MaxFunctionEvals, and MaxGenerations, with the latter two being optional and disabled by default. More can always be added later should they turn out to be useful.

(also, it seems you added some ignored files on accident, .DS_Store and Cargo.lock)

[pengowen123]

With this PR and the remaining mutation bugs fixed, the library should be functional. You've already listed the important work to do after that, and I'll also add that the library could use more thorough tests so we can be sure it works and continues to work properly.

Extra parameters seems like an interesting feature. Do you have a specific case in mind where you might use this (aside from the robotic arm example)?

As for performance, I would expect rayon's performance overhead to be negligible in comparison with everything else. It will only happen once per generation, while many CMA-ES runs will be performed in that same time. The rest of the changes can be made after the API is finished and tests are added. Also, I have some ideas for optimization in cmaes that should significantly improve the performance of eant2 as well. Additionally, I intend on redesigning cge to make most of eant2's work much cleaner/more performant.

[wbrickner]

Extra parameters seems like an interesting feature. Do you have a specific case in mind where you might use this (aside from the robotic arm example)?

Yes, many! Big fan of inverse design, so I've tried using other neuroevolutionary libraries in Rust. I'm often in a situation where I have left a design open ended (physical part, control parameters for an input filter(s), etc), and am stuck trying to optimize a separate set of parameters that control the design along with the neural network component. This is usually really hard, not possible, or in the naive case really slow (outer loop auxiliary, inner loop neural).

One concern I have is that finding optimal auxiliary parameters increases the complexity of the CMA-ES search, and we need to be careful on how we pass on those parameters and how we estimate their optimal search range, and if we ever are to compare the two sets of parameters on the basis of similarity, how should the auxiliary parameters change that analysis. I think it might work well enough to just tack them on the end of the network params & let CMA-ES optimize the fitness function blindly.

The FitnessFunction trait will need to be modified to support this, as we need to provide the network and a slice to the auxiliary parameters. Perhaps:

trait FitnessFunction {
  fn fitness(&self, network: &mut Network) -> f64 { todo!(); };

  fn with_auxiliary(&self, network: &mut Network, auxiliary: &[f64]) -> f64 {
    self.fitness(network)
  }
}

The user who isn't interested in the extra flexibility simply opts out by implementing fitness (they don't need to know about the existence of with_auxiliary), those who need it implement with_auxiliary (which is what we call). One downside is it's not a static error to have left fitness unimplemented, but it is an immediate runtime panic.

Another approach to hide the existence of the auxiliary parameters from those who don't want them would be:

pub trait FitnessFunction {
  fn fitness(&self, network: &mut Network) -> f64;
}

pub trait AuxiliaryFitnessFunction {
  fn fitness(&self, network: &mut Network, aux: &[f64]) -> f64;
}

trait InternalFitnessFunction {
  fn fitness(&self, network: &mut Network, aux: &[f64]) -> f64;
}

impl<T: FitnessFunction> InternalFitnessFunction for T {
  fn fitness(&self, network: &mut Network, _: &[f64]) -> f64 {
    self.fitness(network)
  }
}

impl<T: AuxiliaryFitnessFunction> InternalFitnessFunction for T {
  fn fitness(&self, network: &mut Network, aux: &[f64]) -> f64 {
    self.fitness(network, aux)
  }
}

[pengowen123]

That should be easy to support then. The auxiliary parameters can simply be optimized in the main CMA-ES runs alongside the main parameters and passed to FitnessFunction::fitness as an extra argument. This shouldn't add too much to the run time if the number of auxiliary parameters is small because the dimension already includes every connection weight, so it might only increase from 30 -> 33 for example. For the design, I think just tacking another argument onto the existing method and not adding any new methods will be best here:

trait FitnessFunction {
  fn fitness(&self, network: &mut Network, aux: &[f64]) -> f64;
}

The aux slice would simply be empty if auxiliary parameters aren't enabled and can be ignored by writing _: &[f64] if it's unused. If an extra method was added instead, then FitnessFunction::fitness could not be implemented for fitness functions that require auxiliary parameters, forcing the user to write a dummy implementation. Using a different trait would add extra complexity to the API because the auxiliary parameters should ideally be a runtime parameter, and selecting between two trait implementations at runtime requires an extra method for maximum usability (this is the approach used in the cmaes crate for selecting between serial/parallel objective function execution, but in that case it is necessary because the difference is more fundamental).

The main issues are then:

• How should the initial mean and step size be determined for the auxiliary parameters?
• It sounds like some use cases would require boundaries/constraints on the parameters (e.g., a robotic arm cannot have a negative length), but cmaes currently does not support this feature.

Similarity/identicality are determined only by the structure of the networks in question, and I wouldn't expect auxiliary parameters to change that. Presumably, the optimal auxiliary parameters for two structurally similar networks should be similar in the same way that their optimal weights are expected to be, so EANT2's default behavior can be kept here unless testing suggests otherwise.

[wbrickner]

Okay, I think the main points are resolved. Ready to merge?

[pengowen123]

Alright, everything looks good. Thanks again for your hard work!

Everything else can be implemented in followup PRs. I think the mutation bug is the only thing left to fix before the library at least functions. A non-trivial example problem will need to be created to observe it, as the problem was that invalid networks were being constructed after several EANT2 generations.