search

SciML / OptimizationBase.jl

The base package for Optimization.jl, containing the structs and basic functions for it.
MIT License
15 stars 7 forks source link

Add stochastic oracles #61

Closed Vaibhavdixit02 closed 2 months ago

Vaibhavdixit02 commented 4 months ago

Right now Optimization.jl handles batching by a crude method. This amounts to asking users to pass in data arg to solve. This is expected to be an iterator, and more specifically a DataLoader https://fluxml.ai/Flux.jl/dev/reference/data/mlutils/#DataLoader. But, stochastic oracles are quite a fundamental part of modern optimization algorithms including recent work on second order and quasi-newton stochastic algs hence it would be more natural to have stochastic oracles which can be used directly by downstream algorithm implementations

ChrisRackauckas commented 4 months ago

Definitely agreed here, we can't really do a stochastic BFGS in the current form and that would be quite a useful method.

Vaibhavdixit02 commented 4 months ago

Yup there's a stochastic newton etc as well 😅