search

r-lib / vctrs

Generic programming with typed R vectors
https://vctrs.r-lib.org
Other
290 stars 66 forks source link

Data Join Attempt #1936

Closed bogundele closed 6 months ago

bogundele commented 6 months ago

Error in vctrs::vec_locate_matches(): ! Match procedure results in an allocation larger than 2^31-1 elements. Attempted allocation size was 22846016248. ℹ In file match.c at line 2658. ℹ Install the winch package to get additional debugging info the next time you get this error. ℹ This is an internal error that was detected in the vctrs package. Please report it at https://github.com/r-lib/vctrs/issues with a reprex and the full backtrace. Backtrace: ▆

  1. ├─dplyr::left_join(pbjemployeedetail_2023, generalpbjdata, by = "PROVNUM")
  2. ├─dplyr:::left_join.data.frame(...)
  3. │ └─dplyr:::join_mutate(...)
  4. │ └─dplyr:::join_rows(...)
  5. │ └─dplyr:::dplyr_locate_matches(...)
  6. │ ├─base::withCallingHandlers(...)
  7. │ └─vctrs::vec_locate_matches(...)
  8. └─rlang:::stop_internal_c_lib(...)
  9. └─rlang::abort(message, call = call, .internal = TRUE, .frame = frame)
DavisVaughan commented 6 months ago

If you update dplyr you should get a better error message about this. It looks like you are trying to do a join that results in 22 billion rows, which is way more than dplyr can handle. You are likely missing a join condition in your by argument to dplyr. That has almost always been the case, so I am going to close this one unless you are certain that is not the problem, thanks!