Dimension reduced surrogate construction for parametric PDE maps
___ ___ ___
/__/\ ___ / /\ / /\ ___
\ \:\ / /\ / /::\ / /::\ /__/|
\__\:\ / /:/ / /:/\:\ / /:/\:\ | |:|
___ / /::\ /__/::\ / /:/~/:/ / /:/~/:/ | |:|
/__/\ /:/\:\ \__\/\:\__ /__/:/ /:/ /__/:/ /:/ __|__|:|
\ \:\/:/__\/ \ \:\/\ \ \:\/:/ \ \:\/:/ /__/::::\
\ \::/ \__\::/ \ \::/ \ \::/ ~\~~\:\
\ \:\ /__/:/ \ \:\ \ \:\ \ \:\
\ \:\ \__\/ \ \:\ \ \:\ \__\/
\__\/ \__\/ \__\/
___ ___ ___
/ /\ / /\ /__/\
/ /:/_ / /::\ _\_ \:\
/ /:/ /\ ___ ___ / /:/\:\ /__/\ \:\
/ /:/ /:/ /__/\ / /\ / /:/ \:\ _\_ \:\ \:\
/__/:/ /:/ \ \:\ / /:/ /__/:/ \__\:\ /__/\ \:\ \:\
\ \:\/:/ \ \:\ /:/ \ \:\ / /:/ \ \:\ \:\/:/
\ \::/ \ \:\/:/ \ \:\ /:/ \ \:\ \::/
\ \:\ \ \::/ \ \:\/:/ \ \:\/:/
\ \:\ \__\/ \ \::/ \ \::/
\__\/ \__\/ \__\/
This code is used to construct dimension reduced neural network surrogates for parametric mappings governed by PDEs
hIPPYlib
is used for the data generation and construction of model based projectors
Tensorflow
and keras
are used for the construction of neural network surrogates
hessianlearn
is used for second order optimization of keras neural network models.
hIPPYflow
implements software infrastructure for input and output dimension reduction strategies for parametric mappings governed by PDEs. Given a parametric PDE Variational Problem implemented in hIPPYlib
(using FEniCS
for finite element representation), and a PDE observable, this code automates the construction of dominant subspaces of the input and output for these mappings.
hIPPYflow
implements both active subspace (AS) and Karhunen Loeve expansion (KLE) for input dimension reduction. hIPPYflow
implements proper orthogonal decomposition (POD) for output dimension reduction.
AS computes the dominant eigenvalue-eigenvector pairs of the following operator:
KLE computes the dominant eigenvalue-eigenvector pairs of the covariance of the parameter distribution
POD computes the dominant eigenvalue-eigenvector pairs of the expectation of the data outer-product matrix:
These constructs also implement the generation of training data to be used in surrogate construction, as well as projection error tests that exemplify how good the different model projectors are at capturing key information, and help to detect the "intrinsic dimensionality" of the mappings from inputs to outputs.
hIPPYlib
, set HIPPYLIB_PATH
, HIPPYFLOW_PATH
environmental variables.import dolfin as dl
import ufl
import numpy as np
import sys, os
sys.path.append(os.environ.get('HIPPYLIB_PATH'))
import hippylib as hp
sys.path.append(os.environ.get('HIPPYFLOW_PATH'))
import hippyflow as hf
# Set up PDE Variational Problem and observable using a function
def build_observable(mesh, **kwargs):
# Set up the PDE problem in hIPPYlib
rank = dl.MPI.rank(mesh.mpi_comm())
Vh2 = dl.FunctionSpace(mesh, 'Lagrange', 2)
Vh1 = dl.FunctionSpace(mesh, 'Lagrange', 1)
Vh = [Vh2, Vh1, Vh2]
# Initialize Expressions
f = dl.Constant(0.0)
def u_boundary(x,on_boundary):
return on_boundary
u_bdr = dl.Expression("x[1]", degree=1)
u_bdr0 = dl.Constant(0.0)
bc = dl.DirichletBC(Vh[hp.STATE], u_bdr, u_boundary)
bc0 = dl.DirichletBC(Vh[hp.STATE], u_bdr0, u_boundary)
def pde_varf(u,m,p):
return ufl.exp(m)*ufl.inner(ufl.grad(u), ufl.grad(p))*ufl.dx - f*p*ufl.dx
pde = hp.PDEVariationalProblem(Vh, pde_varf, bc, bc0, is_fwd_linear=True)
# Instance observable operator (in this case pointwise observation of state)
x_targets = np.linspace(0.1,0.9,10)
y_targets = np.linspace(0.1,0.9,10)
targets = []
for xi in x_targets:
for yi in y_targets:
targets.append((xi,yi))
targets = np.array(targets)
B = hp.assemblePointwiseObservation(Vh[hp.STATE], targets)
return hf.LinearStateObservable(pde,B)
# Set up mesh
ndim = 2
nx = 10
ny = 10
mesh = dl.UnitSquareMesh(nx, ny)
# Instance observable
observable_kwargs = {} # No kwargs given in this example
observable = build_observable(mesh,**observable_kwargs)
# Instance probability distribution for the parameter
prior = hp.BiLaplacian2D(observable.problem.Vh[hp.PARAMETER],gamma = 0.1, delta = 1.0)
# Instance Active Subspace Operator
AS = hf.ActiveSubspaceProjector(observable,prior)
# Compute and save input reduced basis to file:
AS.construct_input_subspace()
# Instance POD Operator to compute POD basis and training data
POD = hf.PODProjector(observable,prior)
POD.construct_subspace()
output_directory = 'location/for/training/data/'
POD.generate_training_data(output_directory)
Given information about dominant subspaces of the input and output spaces for the parametric mappings, hIPPYflow
implements dimension reduced neural network surrogates. These surrogates allow for parsimonious representations of input-output mappings that can achieve good accuracy for very few training data. Few data is a key feature of many high dimensional PDE based inference problems.
Neural network models are implemented in keras
. Training can be handled directly by keras
, or using second order optimizers implemented in hessianlearn
.
Active subspace decomposition preserves low dimensional geometry of the parametric mapping.
Geometry preserving dimension reduction allows for efficient parametrization of neural networks that can generalize well given limited training data.
Useful for outer-loop applications (e.g. uncertainty quantification, Bayesian inverse problems, Bayesian optimal experimental design, optimization under uncertainty, etc.) where repeated evaluation of expensive PDE-based maps is a major computational bottleneck and limitation in practice.
These publications use the hippyflow library
[1] O'Leary-Roseberry, T., Villa, U., Chen P., Ghattas O.,
Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs.
Computer Methods in Applied Mechanics and Engineering. Volume 388, 1 January 2022, 114199.
(Download)BibTeX
@article{OLearyRoseberryVillaChenEtAl2022,
title={Derivative-informed projected neural networks for high-dimensional parametric maps governed by {PDE}s},
author={O’Leary-Roseberry, Thomas and Villa, Umberto and Chen, Peng and Ghattas, Omar},
journal={Computer Methods in Applied Mechanics and Engineering},
volume={388},
pages={114199},
year={2022},
publisher={Elsevier}
}
}
[2] O'Leary-Roseberry, T., Du, X., Chaudhuri, A., Martins, J., Willcox, K., Ghattas, O.,
Learning high-dimensional parametric maps via reduced basis adaptive residual networks.
Computer Methods in Applied Mechanics and Engineering. Volume 402, December 2022, 115730.
(Download)BibTeX
@article{OLearyRoseberryDuChaudhuriEtAl2022,
title={Learning high-dimensional parametric maps via reduced basis adaptive residual networks},
author={O’Leary-Roseberry, Thomas and Du, Xiaosong and Chaudhuri, Anirban and Martins, Joaquim RRA and Willcox, Karen and Ghattas, Omar},
journal={Computer Methods in Applied Mechanics and Engineering},
volume={402},
pages={115730},
year={2022},
publisher={Elsevier}
}
}
[3] Wu, K., O'Leary-Roseberry, T., Chen, P., Ghattas, O.,
Large-Scale Bayesian Optimal Experimental Design with Derivative-Informed Projected Neural Network.
Journal of Scientific Computing 95. Article number: 30 (2023)
(Download)BibTeX
@article{WuOLearyRoseberryChenEtAl2023,
title={Large-Scale {B}ayesian Optimal Experimental Design with Derivative-Informed Projected Neural Network},
author={Wu, Keyi and O’Leary-Roseberry, Thomas and Chen, Peng and Ghattas, Omar},
journal={Journal of Scientific Computing},
volume={95},
number={1},
pages={30},
year={2023},
publisher={Springer}
}
}
[4] Cao, L., O'Leary-Roseberry, T., Jha, P., Oden, J.T., Ghattas, O.,
Residual-Based Error Correction for Neural Operator Accelerated Infinite-Dimensional Bayesian Inverse Problems.
Journal of Computational Physics, 112104
(Download)BibTeX
@article{CaoOLearyRoseberryJhaEtAl2023,
title={Residual-based error correction for neural operator accelerated infinite-dimensional {B}ayesian inverse problems},
author={Cao, Lianghao and O'Leary-Roseberry, Thomas and Jha, Prashant K and Oden, J Tinsley and Ghattas, Omar},
journal={Journal of Computational Physics},
pages={112104},
year={2023},
publisher={Elsevier}
}
}
[5] O'Leary-Roseberry, T., Chen P., Villa, U., Ghattas, O.,
Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning.
Journal of Computational Physics, 112555
(Download)BibTeX
@article{OLearyRoseberryChenVillaEtAl22,
title={Derivative-Informed Neural Operator: an efficient framework for high-dimensional parametric derivative learning},
author={O’Leary-Roseberry, THOMAS and Chen, Peng and Villa, Umberto and Ghattas, Omar},
journal={Journal of Computational Physics},
pages={112555},
year={2024},
publisher={Elsevier}
}
}
[6] Luo, D., O'Leary-Roseberry, T., Chen P., Ghattas, O.,
Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators.
(Download)BibTeX
@article{luo2023efficient,
title={Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators},
author={Luo, Dingcheng and O'Leary-Roseberry, Thomas and Chen, Peng and Ghattas, Omar},
journal={arXiv preprint arXiv:2305.20053},
year={2023}
}
}
[7] Cao, L., O'Leary-Roseberry, T., Ghattas, O.,
Derivative-Informed Neural Operator Acceleration of Geometric MCMC for Infinite-Dimensional Bayesian Inverse Problems.
(Download)BibTeX
@article{cao2024derivative,
title={Derivative-informed neural operator acceleration of geometric MCMC for infinite-dimensional Bayesian inverse problems},
author={Cao, Lianghao and O'Leary-Roseberry, Thomas and Ghattas, Omar},
journal={arXiv preprint arXiv:2403.08220},
year={2024}
}
}