surrogate model

[aklawonn]

Implements the code structure for using surrogate models (meta models, response surfaces) with probeye. This structure is up for discussion, so please feel free to comment and propose changes. In order to check the currently intended use of a surrogate model, check out this integration test. When finished, this fixes #78.

[atulag0711]

Thanks Alex. The integration example looks nice and makes sense to me. However, fit method in probeye context would be difficult. I will complete my implementation, push here and then we can discuss.

[merijnpepijndebakker]

Thanks for this. In my opinion the structure you propose makes a lot of sense, i.e. consider the surrogate model as a separate class and as model object. One remark is that we also have different ways to sample new datapoints to generate the surrogate. How would you connect the surrogate with the sampling methods?

[joergfunger]

I would think the sampling could be done separately. So we would either have a sample_from_prior functionality, or any other sampler that takes as input a parameter estimation problem and returns the samples. Then these samples would be passed to the metamodel, which would then evalute the samples and build the metamodel. You could certainly perform the computation of the outputs (so the forward model output that is to be metamodeled) as a separate independent step, however doing it inside the metamodel allows for an adaptation (so if the metamodel requires new samples, they could just be added whenever the model is evaluated).

[JanKoune]

Hi, thank you for the feedback. It may be best to provide a brief description of the changes and the intended layout of the sampling and surrogating interface before addressing your individual comments:

1. Surrogating and sampling are now separated by defining the SurrogateBase and SamplerBase classes, from which the user is expected to derive their own implementation.
2. Both base classes should be as simple as possible for flexibility to allow for different approaches to surrogating and sampling. E.g. in a simple case we may want to draw LHS samples from the prior and manually fit a surrogate model which we then use to define a forward model, or in a more complex case we may want to pass the surrogate model to the sampler to perform iterative fitting and sampling.
3. The main use of the SurrogateBase class is to copy the interface of a given forward model, and provide a template for surrogate models by implementing "empty" methods that are typically useful for surrogating (e.g. fit and predict).
4. The SamplerBase is meant to read the inverse problem and forward model definitions and extract the information that is needed by the sampler (e.g. parameter priors and bounds, and input/output shapes). Indeed it would be best to do the surrogating for each forward model and not for the entire inverse problem. This could be enforced by making the forward model an input argument to SamplerBase, ensuring that a sampler is tied to a specific forward model.
5. Most of the functionality for getting the problem and forward model information is not yet implemented in the base class. I will try to get it working asap.
6. The base classes are now decoupled from harlow.

[JanKoune]

Thank you for the suggestions @danielandresarcones. Some notes:

1. What is the benefit of creating surrogating and sampling wrapper base classes plus an interface class? Why not use the __init__ method of SamplerBase to obtain all the information needed for surrogating and sampling from InverseProblem and ForwardModel?
2. Ideally, with the current structure of SamplerBase and SurrogateBase the user would not need any knowledge of the InverseProblem and ForwardModel internals since extracting the necessary information would be taken care of in the __init__ of SamplerBase, although this is not yet implemented.
3. I like the idea of having the interface also act as the forward model. However, it seems that there is a lot of processing of the forward model that happens internally in probeye and could potentially cause problems (e.g. the issue with deepcopy) if the forward model becomes too complex. It could also be that this is fine, but I tried this approach initially and encountered some issues which is why I opted to keep SurrogateBase as light as possible.

[danielandresarcones]

1. My idea behind it is to separate functions and allow an easier integration with external generic samplers/surrogates, unless we decide to go always through harlow. Using the __init__ from SamplerBase means that the interactions between surrogate and sampler are controled by the sampler, and that this one will have to store the information regarding datasets, formats and parameters, as well as implementing the extra utilities that we may need. If we want to use a surrogate with a previously generated dataset with no sampling at all, having to define a sampler seems counterintuitive. In the same way, derived samplers shouldn't be able to modify the utilities concerning the ForwardModel, such as comparing the output of the surrogate model or generating the covariance matrices.
2. The derived class would not have to modify it, but it will still be passed through the __init__and accessible, as well as having to generate a copy of it. I consider encapsulating the sampler and surrogate from the inverse and forward problems the safest option.
3. That is a good point, although it is something to consider.

[JanKoune]

I don't see anything in the current implementation tying us to Harlow or necessitating the use of a sampler to fit a surrogate model to an existing dataset. It seems to me that the two approaches are mostly functionally equivalent, with the main differences being:

• Having a separate interface class vs. having SamplerBase act as the interface
• Defining the forward model as part of the interface vs. defining it as part of the surrogate model

I do not have a strong preference for either approach.