tidymodels / yardstick

Tidy methods for measuring model performance
https://yardstick.tidymodels.org/
Other
367 stars 54 forks source link

Error in `yardstick_table()`: `estimate` must be a factor. #426

Closed moloscripts closed 8 months ago

moloscripts commented 1 year ago

Tips for a helpful bug report:

While implementing LSTM using this book - https://smltar.com/dllstm.html#lstmfull. I wanted to reproduce a confusion matrix based on the code in section 9.8 The full game LSTM Below is my code produced using reprex


> library(reprex)
> reprex(confusion.matrix <- final_res %>%
+            conf_mat(state, .pred_1) %>%
+            autoplot(type="heatmap")
+        confusion.matrix)

I get an error on my terminal saying

Error: unexpected symbol in:
"           autoplot(type="heatmap")
       confusion.matrix"

Below is the screenshot of the whole error in my terminal. What could be wrong?

> final_res %>%
+   conf_mat(state, .pred_1) %>%
+   autoplot(type="heatmap")
Error in `yardstick_table()`:
! `estimate` must be a factor.
ℹ This is an internal error that was detected in the yardstick package.
  Please report it at <https://github.com/tidymodels/yardstick/issues> with a reprex and the full backtrace.
Run `rlang::last_error()` to see where the error occurred.
> reprex(final_res %>%
+            conf_mat(state, .pred_1) %>%
+            autoplot(type="heatmap")
+        confusion.matrix)
Error: unexpected symbol in:
"           autoplot(type="heatmap")
       confusion.matrix"
> 
Screenshot 2023-04-24 at 12 34 50
EmilHvitfeldt commented 1 year ago

Hello @moloscripts 👋

Are you sure you are running all the code chunks in the chapter before getting to this point?

The errors comes because estimate, .pred_1 in this case, ins't a factor. This should be the case if keras_predict() was used

moloscripts commented 1 year ago

No, I started implementing the code chunks from 9.8 The full game LSTM. Should I start from 9.1?

EmilHvitfeldt commented 1 year ago

You need a couple of code chunks before 9.8,

# Code from 8.1 to load in data and split it
kickstarter <- read_csv("https://github.com/EmilHvitfeldt/smltar/raw/master/data/kickstarter.csv.gz")
set.seed(1234)
kickstarter_split <- kickstarter %>%
  filter(nchar(blurb) >= 15) %>%
  initial_split()

kickstarter_train <- training(kickstarter_split)
kickstarter_test <- testing(kickstarter_split)

# 8.2.4 to get keras_predict()

keras_predict <- function(model, baked_data, response) {
  predictions <- predict(model, baked_data)[, 1]
  tibble(
    .pred_1 = predictions,
    .pred_class = if_else(.pred_1 < 0.5, 0, 1),
    state = response
  ) %>%
    mutate(across(c(state, .pred_class),            ## create factors
                  ~ factor(.x, levels = c(1, 0))))  ## with matching levels
}
moloscripts commented 1 year ago

Let me try it out

github-actions[bot] commented 7 months ago

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.