Multi trust regions

[khurram-ghani]

Related issue(s)/PRs: None Related branch: victor/trustregions

Summary

This PR adds support for multi-trust-regions acquisition rule; based-on and refactored from branch victor/trustregions.

The main classes are:

• CollectionSearchSpace
  • A new abstract class representing search spaces made up of a collection of sub-spaces.
• TaggedMultiSearchSpace
  • A concrete implementation of CollectionSearchSpace for collections where all sub-spaces have the same dimensionality. This is the class used for current trust-region rules.
• TaggedProductSearchSpace
  • The existing product search space class now inherits from CollectionSearchSpace.
• UpdatableTrustRegion
  • A new abstract class representing search spaces that can be updated or reinitialised.
• BatchTrustRegion
  • A new abstract class for trust region acquisition rules.
• SingleObjectiveTrustRegionBox
  • A concrete implementation of UpdatableTrustRegion for the algorithm implemented in victor/trustregions.
• BatchTrustRegionBox
  • A concrete implementation of BatchTrustRegion for the algorithm implemented in victor/trustregions.

The intention is that whenever a new trust region algorithm is implemented, in most cases, it would be sufficient to simply sub-class UpdatableTrustRegion and BatchTrustRegion.

Question: since SingleObjectiveTrustRegionBox/BatchTrustRegionBox is a specific trust-regions implementation, should it be moved to either a separate file or perhaps to a notebook? Is it generally useful? Answer: yes these are useful and should be kept with the rules for now.

An example notebook will be added in a follow-on PR.

Fully backwards compatible: yes

PR checklist

• [X] The quality checks are all passing
• [X] The bug case / new feature is covered by tests
• [X] Any new features are well-documented (in docstrings or notebooks)

[khurram-ghani]

trying to make up my mind about updatable class - are there efficiency gains with that?

@hstojic There aren't computation efficiency gains. This is about good design patterns (encapsulation); what is easy for developers to understand and maintain. In my opinion, having a separate update-able class makes it clear what it does and keeps most of the code self contained. Only when something applies to multiple regions (e.g. maybe_reinitialise_subspaces), it exists in the rule iteself.