Look into adding non-linear dimensionality reduction preprocessors

[rhiever]

Most of the feature preprocessors that we use are based on linear methods. We should look into adding non-linear dimensionality reduction preprocessors, such as:

• http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.KernelPCA.html
• http://scikit-learn.org/stable/modules/generated/sklearn.manifold.Isomap.html#sklearn.manifold.Isomap

[kadarakos]

This paper http://bit.ly/2gbuKey suggests that non-linear dimensionality reduction techniques fail to improve upon PCA in natural data sets; it actually has KernelPCA in the comparison. Since PCA is super fast compared to KernelPCA and other non-linear techniques I would vote against including non-linear stuff.

[rhiever]

That's very surprising. I bet we could find some examples where those findings don't hold.

[saddy001]

To get a first insight one could include non-linear preprocessors, run TPOT for 2-3 standard datasets and look into the best pipelines, if any of those preprocessors were included.