Add new models

[MKaptein]

The list of models we export is actually larger than in mentioned in the current version. We need to update the docs and "re-push" the package.

[robinvanemden]

	Classification	Regression
Linear	scikit-learn LogisticRegression LogisticRegressionCV PassiveAggressiveClassifier Perceptron RidgeClassifier RidgeClassifierCV SGDClassifier lightning AdaGradClassifier CDClassifier FistaClassifier SAGAClassifier SAGClassifier SDCAClassifier SGDClassifier	scikit-learn ARDRegression BayesianRidge ElasticNet ElasticNetCV HuberRegressor Lars LarsCV Lasso LassoCV LassoLars LassoLarsCV LassoLarsIC LinearRegression OrthogonalMatchingPursuit OrthogonalMatchingPursuitCV PassiveAggressiveRegressor RANSACRegressor(only supported regression estimators can be used as a base estimator) Ridge RidgeCV SGDRegressor TheilSenRegressor StatsModels Generalized Least Squares (GLS) Generalized Least Squares with AR Errors (GLSAR) Ordinary Least Squares (OLS) [Gaussian] Process Regression Using Maximum Likelihood-based Estimation (ProcessMLE) Quantile Regression (QuantReg) Weighted Least Squares (WLS) lightning AdaGradRegressor CDRegressor FistaRegressor SAGARegressor SAGRegressor SDCARegressor
SVM	scikit-learn LinearSVC NuSVC SVC lightning KernelSVC LinearSVC	scikit-learn LinearSVR NuSVR SVR lightning LinearSVR
Tree	DecisionTreeClassifier ExtraTreeClassifier	DecisionTreeRegressor ExtraTreeRegressor
Random Forest	ExtraTreesClassifier LGBMClassifier(rf booster only) RandomForestClassifier XGBRFClassifier(binary only, multiclass is not supported yet)	ExtraTreesRegressor LGBMRegressor(rf booster only) RandomForestRegressor XGBRFRegressor
Boosting	LGBMClassifier(gbdt/dart/goss booster only) XGBClassifier(gbtree/gblinear booster only)	LGBMRegressor(gbdt/dart/goss booster only) XGBRegressor(gbtree/gblinear booster only)

[MKaptein]

In the midst of adding this, but needs an update of the toolchain to support lightgbm (which is also the package name in the bundle).

[MKaptein]

Ran tests for:

package	model	New	Tested (01052020)	Note
lightgbm	LGBMClassifier(rf booster only)	yes	ok
lightgbm	LGBMRegressor(rf booster only)	yes	ok
lightning	AdaGradClassifier	yes	no	Skip for now
lightning	AdaGradRegressor	yes	no	Skip for now
lightning	CDClassifier	yes	no	Skip for now
lightning	CDRegressor	yes	no	Skip for now
lightning	FistaClassifier	yes	no	Skip for now
lightning	FistaRegressor	yes	no	Skip for now
lightning	KernelSVC	yes	no	Skip for now
lightning	SAGAClassifier	yes	no	Skip for now
lightning	SAGARegressor	yes	no	Skip for now
lightning	SAGClassifier	yes	no	Skip for now
lightning	SAGRegressor	yes	no	Skip for now
lightning	SDCAClassifier	yes	no	Skip for now
lightning	SDCARegressor	yes	no	Skip for now
sklearn	ARDRegression	no	ok
sklearn	BayesianRidge	yes	ok
sklearn	DecisionTreeClassifier	no	ok
sklearn	DecisionTreeRegressor	no	ok
sklearn	ElasticNet	no	ok
sklearn	ElasticNetCV	no	ok
sklearn	ExtraTreeClassifier	no	ok
sklearn	ExtraTreeRegresso	no	ok
sklearn	ExtraTreesClassifier	no	ok
sklearn	ExtraTreesRegressor	no	ok
sklearn	HuberRegressor	no	ok
sklearn	Lars	no	ok
sklearn	LarsCV	no	ok
sklearn	Lasso	no	ok
sklearn	LassoCV	no	ok
sklearn	LassoLars	no	ok
sklearn	LassoLarsCV	no	ok
sklearn	LassoLarsIC	no	ok
sklearn	LinearRegression	no	ok
sklearn	LinearSVC	no	ok
sklearn	LinearSVR	no	ok
sklearn	LogisticRegression	yes	ok
sklearn	LogisticRegressionCV	yes	ok
sklearn	NuSVC	no	ok
sklearn	NuSVR	no	ok
sklearn	OrthogonalMatchingPursuit	no	ok
sklearn	OrthogonalMatchingPursuitCV	no	ok
sklearn	PassiveAggressiveClassifier	no	ok
sklearn	PassiveAggressiveRegressor	no	ok
sklearn	Perceptron	yes	ok
sklearn	RandomForestClassifier	no	ok
sklearn	RandomForestRegressor	no	ok
sklearn	RANSACRegressor	no	ok
sklearn	Ridge	no	ok
sklearn	RidgeClassifier	yes	ok
sklearn	RidgeClassifierCV	yes	ok
sklearn	RidgeCV	no	ok
sklearn	SGDClassifier	no	ok
sklearn	SGDRegressor	no	ok
sklearn	SVC	no	ok
sklearn	SVR	no	ok
sklearn	TheilSenRegressor	no	ok
statsmodels	[Gaussian] Process Regression Using Maximum Likelihood-based Estimation (ProcessMLE)	yes	not ok	Skip for now
statsmodels	Generalized Least Squares (GLS)		ok
statsmodels	Generalized Least Squares with AR Errors (GLSAR)	yes	ok
statsmodels	Ordinary Least Squares (OLS)		ok
statsmodels	Quantile Regression (QuantReg)	yes	ok
statsmodels	Weighted Least Squares (WLS)		ok
xgboost	XGBClassifier(gbtree/gblinear booster only)	no	ok
xgboost	XGBRegressor(gbtree/gblinear booster only)	no	ok
xgboost	XGBRFClassifier(binary only, multiclass is not supported yet)	no	ok
xgboost	XGBRFRegressor	no	ok

So, added a few models (including lightgbm) and xgboost. Not added the lightning ones (although in test_all_models.py these are specified already and they should be simple to add by:

1. Checking if _model_fitted() and _precit() work for lightning models
2. Mapping the lightning package to the sklearn toolchain (in main.py)
3. Running the tests.

I am closing this for now.

[MKaptein]

package	model	Tested (05052020)	Note
lightgbm	LGBMClassifier	ok
lightgbm	LGBMRegressor	ok
sklearn	ARDRegression	ok
sklearn	BayesianRidge	ok
sklearn	DecisionTreeClassifier	ok
sklearn	DecisionTreeRegressor	ok
sklearn	ElasticNet	ok
sklearn	ElasticNetCV	ok
sklearn	ExtraTreeClassifier	ok
sklearn	ExtraTreeRegresso	ok
sklearn	ExtraTreesClassifier	ok
sklearn	ExtraTreesRegressor	ok
sklearn	HuberRegressor	ok
sklearn	Lars	ok
sklearn	LarsCV	ok
sklearn	Lasso	ok
sklearn	LassoCV	ok
sklearn	LassoLars	ok
sklearn	LassoLarsCV	ok
sklearn	LassoLarsIC	ok
sklearn	LinearRegression	ok
sklearn	LinearSVC	ok
sklearn	LinearSVR	ok
sklearn	LogisticRegression	ok
sklearn	LogisticRegressionCV	ok
sklearn	NuSVC	ok
sklearn	NuSVR	ok
sklearn	OrthogonalMatchingPursuit	ok
sklearn	OrthogonalMatchingPursuitCV	ok
sklearn	PassiveAggressiveClassifier	ok
sklearn	PassiveAggressiveRegressor	ok
sklearn	Perceptron	ok
sklearn	RandomForestClassifier	ok
sklearn	RandomForestRegressor	ok
sklearn	RANSACRegressor	ok
sklearn	Ridge	ok
sklearn	RidgeClassifier	ok
sklearn	RidgeClassifierCV	ok
sklearn	RidgeCV	ok
sklearn	SGDClassifier	ok
sklearn	SGDRegressor	ok
sklearn	SVC	ok
sklearn	SVR	ok
sklearn	TheilSenRegressor	ok
statsmodels	Generalized Least Squares (GLS)	ok
statsmodels	Generalized Least Squares with AR Errors (GLSAR)	ok
statsmodels	Ordinary Least Squares (OLS)	ok
statsmodels	Quantile Regression (QuantReg)	ok
statsmodels	Weighted Least Squares (WLS)	ok
xgboost	XGBClassifier	ok
xgboost	XGBRegressor	ok
xgboost	XGBRFClassifier	ok	Binary only
xgboost	XGBRFRegressor	ok