lime for multi-output keras models

[scworland]

Thank you for developing and maintaining this package.

Neural networks are often used to generate predictions of multiple outputs for each input observation,

Does lime currently support multi-output models? I would think that it does not because the simpler model used to explore the relationships would also have to be able to generate multiple predictions for each observation. I am using keras to build these models.

[scworland]

I was able to figure out how to do this by tweaking the explainer functions. I added an argument, n, to both the explain and explain.data.frame functions and modified the predict line from,

case_res <- predict_model(explainer$model, case_perm, type = o_type)

to,

case_res <- data.frame(predict(object=explainer$model, x=as.matrix(case_perm))[n]) %>% setNames("Response")

Then in a loop I passed the column I wanted from the keras multi-output model to the explain function,

for (j in 1:ncol(Y)) {
  explanation <- lime::explain(x = Xtest[1:500,], 
                               explainer = explainer, 
                               n_features = 10,
                               kernel_width = 0.2,
                               n_permutations = 2000,
                               feature_select = "highest_weights",
                               n = j) 
}

This conceptually makes sense and the results are believable, so I will close this issue.

[MichaelPeibo]

Hi @scworland Thanks for you code. I am dealing with similar multiple output regression keras model.

I changed explain.data.frame function as your code.

But when I run the the below code, I got this error:

explainer=lime::lime(as.data.frame(df_x_train), model1)

for (j in 1:ncol(df_y_test)) {
  explanation <- explain.data.frame(x = as.data.frame(df_x_test[1:5,]), 
                               explainer = explainer, 
                               n_features = 10,
                               kernel_width = 0.2,
                               n_permutations = 2000,
                               feature_select = "highest_weights",
                               n = j) 
}

Error: The class of model must have a model_type method. See ?model_type to get an overview of models supported out of the box

Any suggestion how to deal with it?

AN UPDATE: I got through with

Error: The class of model must have a model_type method. See ?model_type to get an overview of models supported out of the box

by setting model_type function mannually, while another error occurs:

Error in if (nulldev == 0) stop("y is constant; gaussian glmnet fails at standardization step") : missing value where TRUE/FALSE needed

This time, I got no idea how to fix it...