Roadmap for Scholar

[msluszniak]

This issue presents the roadmap of the most important functionalities that should be implemented in the first place.

• [ ] Clustering Algorithms - We need to implement more clustering algorithms especially density-based e.g. DBSCAN
• [ ] SVM
• [x] K-Nearest Neighbors
• [ ] More Classification and Regression Metrics
• [ ] Numerical Solvers - They won't be functionalities in Scholar per se, but they will be exceptionally helpful. Using them, we could improve implementations of current functionalities like Logistic Regression, Ridge Regression and so on. We should start with LBFGS
• [ ] KFold

[seanmor5]

@msluszniak On the note of solvers, it may be useful to look at the implementations in jaxopt! JAX also supports LBFGS out of the box I believe, but the XLA version is significantly slower than the SciPy version

[josevalim]

@seanmor5 do you solvers should be in a separate library then? Or here is fine?

Some additional thoughts:

• Splitting data may also be worth moving to Nx, since it will be shared by both Scholar and Axon. So I would postpone tackling this for now

• We most likely cannot implement training of decision trees - this poses the question if this library should implement decision trees in the first place. However, the HummingBird project shows it can be efficient to compile trained decision trees into tensors

[seanmor5]

@josevalim I think having the solvers in here is fine to start, and then if it grows too large it makes sense to pull them out. It may even make sense to combine the optimization stuff in Axon with the solvers implemented in Scholar if they eventually get pulled into a separate library, as the optimizers are more general purpose than just NNs

[tiagodavi]

@msluszniak Is this priority list updated?

[msluszniak]

@msluszniak Is this priority list updated?

@tiagodavi I think that this list is still up to date. But of course, any contribution will be great :)

[cigrainger]

Is anyone working on SVM? What about saving/loading fitted models? I've got both in my near-term roadmap at work and can jump on them if they're not being worked on already.

[josevalim]

@msluszniak is working on SVM. :)

[josevalim]

Closing this for now in favor of smaller issues.