Extend predictors to zero-cost case

[jr2021]

In the zerocost branch, Ensemble has been extended to the Zero Cost case and contains a single option for its base predictor, XGBoost. The XGBoost predictor has been adapted to the zero-cost case by implementing a set_pre_computations function, as well as modifying the BaseTree class. Currently, the Ensemble class supports only this single predictor:

trainable_predictors = {
    "xgb": XGBoost(
        ss_type=self.ss_type, zc=self.zc, encoding_type="adjacency_one_hot", zc_only=self.zc_only
    )
}

Should all other predictors be available in the merged Ensemble class and extended to the zero-cost case?

[Neonkraft]

Ideally, yes. But there are about 19 predictors in the original ensemble. Let's focus now on extending the ZC case to the tree-based predictors, which are LGBoost, NGBoost, and RandomForestPredictor.

The other predictors must be available, of course, but without the option of using a ZC predictor. This also means that the get_ensemble method must be modified to return the sets of predictors based on self.zc

[jr2021]

Sounds good.

We found that in the zerocost branch, the XGBoost class contains three functions specific to the zero-cost case, set_pre_computations, _verify_zc_info, and _set_zc_names which are also applicable to the other tree-based predictors.

In order to not duplicate these functions, we placed them in the BaseTree class, which is the parent class of all tree-based predictors.

[jr2021]

One small remaining issue is a discrepancy between the zerocost and Develop implementation of fit in XGBoost.

In the Develop branch, it is possible for the user to load in custom hyper-parameters from a config file

def fit(self, xtrain, ytrain, train_info=None, params=None, **kwargs):
        if self.hparams_from_file and self.hparams_from_file not in ['False', 'None'] \
        and os.path.exists(self.hparams_from_file):
            self.hyperparams = json.load(open(self.hparams_from_file, 'rb'))['xgb']
            print('loaded hyperparams from', self.hparams_from_file)
        elif self.hyperparams is None:
            self.hyperparams = self.default_hyperparams.copy()
        return super(XGBoost, self).fit(xtrain, ytrain, train_info, params, **kwargs)

while in the zerocost branch, this is not an option.

def fit(self, xtrain, ytrain, train_info=None, params=None, **kwargs):
        if self.hyperparams is None:
            self.hyperparams = self.default_hyperparams.copy()
        return super(XGBoost, self).fit(xtrain, ytrain, train_info, params, **kwargs)

Which functionality should be adopted in the Develop_copy branch? Is this a case where the code in the zerocost branch should be taken as the more updated version?

[Neonkraft]

Best to be able to read from config file, too.