lime v0.4 doesn't seem to have native support for keras

[scworland]

My previous issue (#104) is growing into two separate questions, so I am starting a new one here. According to this recent post it seems like we shouldn't have to set up a custom predict_model function. Below is a functioning example to illustrate this,

# number of observations
N <- 1000

# synthetic multiple outputs
y <- rnorm(N,20,5) # smallest

# synthetic nonlinear predictors
x1 <- (sqrt(y) + rnorm(N,0,1))^2
x2 <- log10((x1 + y)^3)

X <- data.frame(x1,x2) %>% # combine
  mutate_all(funs(scale)) %>% # scale
  as.matrix() # to matrix

# create train and test set
set.seed(0)
ind <- sample(2, N, replace=T, prob=c(0.50, 0.50))

Xtrain <- X[ind==1,]
Ytrain <- y[ind==1]

Xtest <- X[ind==2,]
Ytest <- y[ind==2] 

model <- keras_model_sequential() %>%
  layer_dense(units = 2, input_shape = dim(X)[2]) %>%
  layer_dense(units = 30,activation="relu") %>%
  layer_dropout(rate = 0.1) %>%
  layer_dense(units = 1) %>%
  compile(optimizer = "rmsprop",
          loss="mse",
          metrics="mae")

model_fit <- model %>% 
  fit(x=Xtrain,
      y=Ytrain, 
      epochs=100, 
      batch_size = 25, 
      validation_split = 0.2,
      verbose=0)

Xtrain <- as.tibble(Xtrain)
Xtest <- as.tibble(Xtest)

# setup lime::predict_model() function for keras
predict_model.keras.models.Sequential <- function(x,newdata,type...) {
  pred <- predict(x, x=as.matrix(newdata)) %>%
    data.frame()
}

# test predict model function
predict_model(x = model, newdata = Xtest, type='raw') %>%
  tibble::as.tibble()

# run lime() on training set
explainer <- lime::lime(x = Xtrain, 
                        model = model, 
                        bin_continuous = FALSE)

# run explain() on the explainer on first 5 rows
explanation <- lime::explain(x = Xtest[1:5,], 
                             explainer = explainer, 
                             n_features = 4,
                             kernel_width = 0.5)

If you remove the following lines,

# setup lime::predict_model() function for keras
predict_model.keras.models.Sequential <- function(x,newdata,type...) {
  pred <- predict(x, x=as.matrix(newdata)) %>%
    data.frame()
}

You will see a value error,

ValueError: No data provided for "dense_24_input". Need data for each key in: ['dense_24_input']

Basically, the keras::predict_model function needs to convert the input to a matrix if is a keras model. Additionally, it will not work with the keras functional API (this one is a real mystery). Replace the sequential model above with,

input_tensor <- layer_input(shape=dim(X)[2])

output_tensor <- input_tensor  %>%
  layer_dense(units = 40,activation="relu") %>%
  layer_dropout(rate = 0.1) %>%
  layer_dense(units = 30,activation="relu") %>%
  layer_dropout(rate = 0.1) %>%
  layer_dense(units = 1)

model <- keras_model(input_tensor,output_tensor) %>%
  compile(optimizer = "rmsprop",
          loss="mse",
          metrics="mae")

and replace the custom predict_model function with,

predict_model.keras.engine.training.Model <- function(x,newdata,type...) {
  pred <- predict(x, x=as.matrix(newdata)) %>%
    data.frame()
}

And if you try to run lime::explain() you will get the value error described above.

[ShirinG]

I have the same problem with lime and keras.

[thomasp85]

Some weird changes in R v3.5 caused the predict_model implementation for keras to fail for unknown reasons - it is fixed in devel now