search

openopt / copt

A Python library for mathematical optimization
http://openo.pt/copt
Other
136 stars 35 forks source link

Mini-batches for SAGA/SVRG #71

Closed GeoffNN closed 3 years ago

GeoffNN commented 4 years ago

Both methods use batch size=1.

fabianp commented 4 years ago

I'm not sure how to do this without making the code much slower for sparse matrices. Currently the SAGA/SVRG code is very efficient for sparse matrices using the fact that the partial gradients are sparse. Using minimatches would destroy the sparsity in the gradients, making the code much slower in this regime