tidymodels / broom

Convert statistical analysis objects from R into tidy format
https://broom.tidymodels.org
Other
1.43k stars 302 forks source link

augment.fixest not playing nice with nest_by? #1163

Closed RoyalTS closed 1 year ago

RoyalTS commented 1 year ago

I'm trying to generate predictions from a group of fitted fixest models but cannot:

mtcars %>%
  nest_by(cyl) %>%
  mutate(model = list(fixest::feols(mpg ~ wt, data = data))) |> 
  mutate(preds = list(broom::augment(model, newdata=data |> mutate(am=0))))

yields

Error in `mutate()`:
ℹ In argument: `preds = list(broom::augment(model, newdata = mutate(data, am = 0)))`.
ℹ In row 1.
Caused by error:
! in predict.fixest(x, type = type.predict, newdata = ...:
 Error when creating the linear matrix: 
  In model.frame.default(fml, data, na.action = na.pass): 'data' must be a data.frame, environment, or list
Backtrace:
  1. dplyr::mutate(...)
  9. broom:::augment.fixest(model, newdata = mutate(data, am = 0))
 11. fixest:::predict.fixest(x, type = type.predict, newdata = newdata)
 12. fixest:::error_sender(...)
 13. dreamerr::stop_up(msg, "\n  ", call_error, err)
 14. base::stop("in ", my_call, ":\n ", fit_screen(message), call. = FALSE)

This works if I fit the models using lm, which leads me to suspect the issue is with augment.fixest:

mtcars %>%
  nest_by(cyl) %>%
  mutate(model = list(lm(mpg ~ wt, data = data))) |> 
  mutate(preds = list(broom::augment(model, newdata=data |> mutate(am=0))))
simonpcouch commented 1 year ago

This looks like an NSE oddity to me! With slight edits:

library(tidyverse)

mtcars |>
  nest_by(cyl) |>
  mutate(
    model = list(fixest::feols(mpg ~ wt, data = data)),
    preds = list(broom::augment(model, newdata = data |> mutate(am = 0)))
  )
#> # A tibble: 3 × 4
#> # Rowwise:  cyl
#>     cyl                data model    preds             
#>   <dbl> <list<tibble[,10]>> <list>   <list>            
#> 1     4           [11 × 10] <fixest> <tibble [11 × 11]>
#> 2     6            [7 × 10] <fixest> <tibble [7 × 11]> 
#> 3     8           [14 × 10] <fixest> <tibble [14 × 11]>

Created on 2023-06-09 with reprex v2.0.2

RoyalTS commented 1 year ago

What the heck? The only difference between the two is that it all happens within the same mutate()? Is there any way to make it work in separate ones? (hard for me to rejigger my actual code into a single mutate)

simonpcouch commented 1 year ago

The only difference between the two is that it all happens within the same mutate()?

The error from fixest with a different reference for .key may be eliciting:

library(tidyverse)

mtcars %>%
  nest_by(cyl, .key = "data_") %>%
  mutate(model = list(fixest::feols(mpg ~ wt, data = data_))) %>%
  mutate(preds = list(broom::augment(model, newdata = data_ %>% mutate(am=0))))
#> Error in `mutate()`:
#> ℹ In argument: `preds = list(broom::augment(model, newdata = data_ %>%
#>   mutate(am = 0)))`.
#> ℹ In row 1.
#> Caused by error:
#> ! in predict.fixest(x, type = type.predict, newdata = ...:
#>  Error when creating the linear matrix: 
#>   in fixest_model_matrix_extra(object = object, newdat...:
#>   we fetch the data in the enviroment where the estimation was made, but
#> the data does not seem to be there any more (btw it was data_).

This is indeed a head-scratcher, but is outside of broom's control. This stackoverflow discussion may be helpful for you!

Created on 2023-06-11 with reprex v2.0.2

github-actions[bot] commented 1 year ago

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.