search

baggepinnen / FluxOptTools.jl

Use Optim to train Flux models and visualize loss landscapes
MIT License
59 stars 4 forks source link

LuxOptTools #25

Closed aksuhton closed 8 months ago

aksuhton commented 8 months ago

Thank you for taking the time to write FluxOptTools and make it publicly available. I want to train a Lux model using BFGS, so I've modified your optfun using Optimisers.destructure.

No doubt what I've written is not yet suitable to be merged, but the included /src/LuxOptTools.jl does at least run. I'm hoping you'd like to support Lux in addition to Flux and can guide me towards getting my fork up to par. I suspect that you might know how to improve upon what I've done. In the meantime, I'll match the rest of your functionality/tests.

baggepinnen commented 8 months ago

If I understand correctly, Lux should not require the utilities provided by this package. Lux already has the structure required, i.e., parameters are explicit etc., to use standard optimizers like BFGS.

aksuhton commented 8 months ago

Perfect, thank you for having a look! I'll see if there's a better way

baggepinnen commented 8 months ago

See this prior discussion using standard optimization solvers with Lux https://discourse.julialang.org/t/composing-lux-with-optim-algorithms/101846/2

aksuhton commented 8 months ago

Hey, much appreciated! I missed that link and the gravitational example in Lux itself.

https://lux.csail.mit.edu/dev/tutorials/advanced/1_GravitationalWaveForm

We should probably get rid of this pull request then (: