The package is not slow at all due to the following facts:
Written in pure Julia! Julia is really fast.
The linear system solver is an iterative solver (we use the Lanczos version of the conjugate gradient algorithm).
Broadcasting and vectorization is used whenever possible.
On a very rough estimate (real benchmarks pending), a classification problem with the RBF kernel takes less than 1 second to train and predict. Although, we must keep in mind that we have to train and predict a lot. For instance, when doing hyperparameter adjustments, we need to evaluate a lot of models.
The thing is, we can actually do much better, but the main drawbacks are the following
More dependencies. This can make the package load slower than it currently does.
We will have to sacrifice readability and clarity in the code.
The package is not slow at all due to the following facts:
On a very rough estimate (real benchmarks pending), a classification problem with the RBF kernel takes less than 1 second to train and predict. Although, we must keep in mind that we have to train and predict a lot. For instance, when doing hyperparameter adjustments, we need to evaluate a lot of models.
The thing is, we can actually do much better, but the main drawbacks are the following
The following are my proposals
https://github.com/edwinb-ai/Elysivm/blob/fb734747070ddcc7c29109d32f1607a8e4acc188/src/training.jl#L24
we will be using the fantastic
Tullio.jlpackage to exploit CPU vectorization instructions.https://github.com/edwinb-ai/Elysivm/blob/fb734747070ddcc7c29109d32f1607a8e4acc188/src/training.jl#L34
we will be using pure BLAS operations, even if they are level 1 BLAS.
This is not a very difficult issue to do, but it will take some time to get the refactorization done.