search

apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.78k stars 6.79k forks source link

[Feature Request] Implementation of L-BFGS optimizer #12132

Open jasonyu1996 opened 6 years ago

jasonyu1996 commented 6 years ago

Hi! Would the developers consider adding an implementation of limited memory BFGS to the library? PyTorch offers one here: https://pytorch.org/docs/stable/optim.html?#torch.optim.LBFGS

adaaaaaa commented 6 years ago

follow with interest...

lanking520 commented 6 years ago

@jasonyu1996 thanks for your question and keep watching this thread. @eric-haibin-lin do we have something similar in MXNet?

@mxnet-label-bot please label this as [feature, memory]

anirudhacharya commented 6 years ago

MXNet does not have this optimizer( see here), and optimizer requests are tracked here - https://github.com/apache/incubator-mxnet/issues/9182

jasonyu1996 commented 6 years ago

Thanks! I have added it to the optimizer request list. Actually I am quite willing to see if I could implement one for MXNet myself.

haojin2 commented 6 years ago

@jasonyu1996 You're more than welcome to add this thing yourself if you want to, most of the optimizers should be here: https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/optimizer.py. You can use any of them (for example, SGD) as reference to implement this. And all unit tests for optimizers are here: https://github.com/apache/incubator-mxnet/blob/master/tests/python/unittest/test_optimizer.py. Please make sure to add a test for your new optimizer once you finish implementing it.