search

JuliaMath / HCubature.jl

pure-Julia multidimensional h-adaptive integration
Other
148 stars 23 forks source link

Mimic `Cubature.INDIVIDUAL` behavior [Feature Req.] #36

Open agerlach opened 3 years ago

agerlach commented 3 years ago

I started to put a PR together to allow for convergence testing based on the individual integrands, like is default in Cubature.jl. However, I quickly realized it may get more involved than I initially thought as the error estimate E is assumed to be scalar in cubrule and the box updates.

Do you have any thoughts/suggestions for adding this feature? Thanks.

stevengj commented 3 years ago

If you use an absolute tolerance, then you can just pass norm=x->norm(x,Inf) (i.e. use the L∞ norm). For relative tolerances, however, that's not equivalent to Cubature.INDIVIDUAL.

Can you explain more about your use-case? Do you have a bunch of integrals with completely unrelated magnitudes? Why not do the integrals separately?

agerlach commented 3 years ago

I want to do optimization w/ reverse mode AD over the quadrature. I have an adjoint written that does ∫f(x,p)dx → ∫(∂/∂p f(x,p))dx. Not having individual convergence criterion for each partial can lead to instabilities in the optimization.

agerlach commented 3 years ago

Why not do the integrals separately?

The integrand is expense to compute. It is of the form g(S(x)) where S is an ODE solve. If interested in more details see our paper https://arxiv.org/pdf/2008.08737.pdf

stevengj commented 3 years ago

I'm not sure I follow. For optimization, isn't it sufficient for the computed gradient (via quadrature) to be sufficiently close to the true gradient in some norm? i.e. isn't a combined error norm precisely what is needed?

agerlach commented 3 years ago

@stevengj You are absolutely correct. Thank you. I miss characterized the behavior I saw. On second thought, the issue may be with respect to using constant tolerances for the quadrature gradient computation in the optimization process. Does it seem reasonable to need to update these as the optimization converges?

stevengj commented 3 years ago

It depends on the optimization algorithm (and how tolerant it is of inexact gradients), but in general I would set the quadrature tolerance to something proportional to the convergence tolerance you want from the optimization.

If you want the optimum to a high accuracy (low tolerance), then it might be reasonable to optimize once using a high tolerance, and then "polish" the solution by running the again with a lower tolerance starting at the high-tolerance solution.