Requesting better exposure to MLJFlux in the model browser

[EssamWisam]

Is your feature request related to a problem? Please describe.

The model browser includes sections such as "Bayesian Models", "Iterated Models" and "Class Imbalance". It would be nice to add a section "Deep Learning" because that would make it more obvious to new MLJ users that they can do machine learning, in a high-level fashion, using MLJFlux.

Describe the solution you'd like

Add a "Deep Learning" or "Neural Networks" section and list MLJFlux models.

[ablaom]

This sounds like a reasonable suggestion. If we add this here, then it would be good to have a PR to update this list at LearnAPI.jl.

My vote is for "neural networks". It's true that Flux models don't have to be neural networks - any differentiable function will do, but "gradient descent methods" is probably too unfamiliar to beginners, and catches things like gradient tree boosters (e.g. XGBoost) that likely we don't want to put in that bucket, despite the valid interpretation.

While I acknowledge its naturalisation into the machine learning vernacular, I've never been fond of the term "deep learning". Neural networks have been around since the 1960's (at least), so the term seems redundant. It' like we started calling polynomials of degree 17 "deep functions". How deep is deep anyway? And lots of things are functions that are not polynomials. Of course we all internalise these terms in different (often unconscious) ways, and this is just my subjective view.

At least 95% of the time, MLJFlux models will be garden variety neural networks. Into this box also go, unambiguously, the models listed below:

 (name = AutoEncoder, package_name = BetaML, ... )
 (name = MultitargetNeuralNetworkRegressor, package_name = BetaML, ... )
 (name = NeuralNetworkClassifier, package_name = BetaML, ... )
 (name = NeuralNetworkRegressor, package_name = BetaML, ... )
 (name = KernelPerceptronClassifier, package_name = BetaML, ... )
 (name = PerceptronClassifier, package_name = BetaML, ... )
 (name = PerceptronClassifier, package_name = MLJScikitLearnInterface, ... )

I may have missed some others.

[EssamWisam]

I agree with what you said on "gradient descent methods" and that "neural networks" is more well-defined and generic than "deep learning". That said, I may share my internal definition thereof: it's deep learning if the neural network is sufficiently deep so that the universal approximation theorem holds; i.e., there is as least one hidden layer.

Likewise, I believe most of what could be built with Flux is some sort of neural network (even if not exactly a feedforward neural network). For your list, I think it's sufficiently comprehensive by including the four models exposed in MLJFlux and AutoEncoder. Like the Perceptron, linear and logistic regression could be viewed as special cases of the neural network (e.g., one neuron); however, the Perceptron is more involved in the history of creation of neural networks.