Keras classifiers?

[besherh]

Please note that this is not an issue!

I was wondering if I could use see support deep learning classifiers (keras)?

is this possible?

[Menelau]

Hello,

Yes it can support as long as the classifier implement the scikit-learn interface (predict, predict_proba).

[besherh]

Thanks for the reply, I have found an example Here that shows how to implement scikit-learn interface.

Let's assume that I have an ensemble of 30 deep learning models (Keras models in h5 format), how to create my 'pool_classifiers' in order to pass it to one of the DSCs.

Could you advise? Thanks in advance!

[Menelau]

Hello,

In this case you just need to create a list of classifiers in order to pass it to the DS method:

pool_classifiers = [model1, model2, ... model30]

You can also check on this example on how to create a pool of classifiers manually just by adding them to a list: https://deslib.readthedocs.io/en/latest/auto_examples/example_heterogeneous.html#sphx-glr-auto-examples-example-heterogeneous-py

[besherh]

Thanks that was helpful (Y)

[besherh]

Hi again, I was looking for an ensemble pruning library, and I found: https://github.com/viisar/brew, however the project is dead now and it had been advised to check DESlib. Are any ensemble pruning techniques that are currently supported in DESlib? I have a pool of 30 models, and I am looking for a pruning function that reduces my pool's size and suggests the optimal models in the pool! in this context, DS is not going to work for, and that's why I am looking for another approach!

[Menelau]

@besherh

Well DESlib has one static selection method which you can use to prune your ensemble: https://deslib.readthedocs.io/en/latest/modules/static/static_selection.html

It is a very simple ensemble pruning model. As the main focus of the library is on dynamic ensembles, this part was not given much attention until now and there is still a lot of room for improvements in the static selection/pruning part of the DESlib. For example, considering also the diversity measures in the pruning (which we already offer in the utils module: https://deslib.readthedocs.io/en/latest/modules/util/diversity.html).

If you have any ideas of an ensemble pruning method that you would like to see in the library please let me know and I can see what I can do to add it. I don't know exactly which ensemble pruning algorithm was available on the brew library, however I will check that as soon as possible.

[besherh]

Thanks, I will check the static_selection (I could have questions for the future :) ), and you are right about considering the diversity measures as acc measures are not enough. As you know, there are plenty of pruning techniques for ensembles, and it could mainly be divided into three categories: 1- ordering-based pruning: like Kappa pruning & Reduce-error pruning. 2- clustering-based pruning: like k-means. 3- optimisation-based pruning: GA approaches.

If we could see one/two per each category, it will be more than great as we get the chance to compare between diffrent algorithms!

[besherh]

I am trying to fit the static_selection with cifar10 data, and I am doing the following:

def scikit_learn_model(model_name):
    loaded_model = tf.keras.models.load_model(model_name) 
    model = KerasClassifier(build_fn=loaded_model, verbose=0)
    return model
def file_browser(directory_name):
  return [model for model in glob.glob(directory_name + "*.h5")]
def intialize_train_set():
  (x_train,y_train),(x_test,y_test) = tf.keras.datasets.cifar10.load_data()
  x_train = x_train.astype("float32")
  x_train /= 255
  y_train = tf.keras.utils.to_categorical(y_train,10)
  print(x_test.shape[0], ": testing samples")
  print(y_test.shape, " Lables")
  return x_train,y_train
dir_name = "./"
models = file_browser(dir_name)
models_pool = []
for model in models:
    models_pool.append(scikit_learn_model(model))
from sklearn.model_selection import train_test_split
X_train,Y_train = intialize_train_set()
rng = np.random.RandomState(42)

X_train, X_prune, y_train, y_prune = train_test_split(X_train,Y_train, test_size=0.2,
                                                    random_state=rng)
y_prune = np.argmax(y_prune,axis=1)
print(y_prune.shape)
X_prune = X_prune.reshape(10000,32*32*3)
X_prune.shape
static_class.fit(X_prune, y_prune)

and I am ending up with an error: 'KerasClassifier' object has no attribute 'classes_' is this the correct way to do this? Any advice is highly appreciated! Thanks

[Menelau]

@besherh

Sorry for the late reply. I believe this error is due to the fact that the KerasClassifier (which you use to wrap your model) was not trained. In order to have the 'classes_' attribute, you need to actually train the returned model from KerasClassifier to make use of scikit-learn functionalities.

This error happens in which line of code?

[besherh]

Hi, The error is raised when calling static_prune.fit(). I have searched for the error and you are right. I have to look about what I have to do regarding this, as a side note do you think transforming the input images from 3d to 2d (to work with this library) is safe ?

[Menelau]

It should not be a problem as long as you can reshape it back to format your model accepts as input.

However, thinking about that, I believe it is better to remove the restriction imposed in the static selection to accept only a 2d array as input. As it can be seen as a meta-estimator, it should not impose constraints on the input type, rather the estimator itself should deal with the checking and validating whether the input is in the correct standard.

I will check if I can safely remove that constraint this weekend and update the static selection. methods.