search

tidyverse / multidplyr

A dplyr backend that partitions a data frame over multiple processes
https://multidplyr.tidyverse.org
Other
641 stars 75 forks source link

partition crashes RStudio #94

Closed david-awam-jansen closed 3 years ago

david-awam-jansen commented 4 years ago

I am having issues with Rstudio keeps crashing when I am trying to partition my data. I managed to reproduce the same problem using the nycflights13 data is being used in the vignette of multidplyr.

This works

library(multidplyr)
library(dplyr, warn.conflicts = FALSE)

cluster <- new_cluster(4)

library(nycflights13)

flight_dest <- flights %>% partition(cluster)

But when I do

cluster <- new_cluster(2) ## or other numbers for istance 6

flight_dest <- flights %>% partition(cluster)

It crashes and I get the following message

R Session Aborted R encountered a fatal error The session was terminated

I am using Rstudio (1.2.5033) and R version 3.6.2

When I run in inside R it does work.

willtudorevans commented 4 years ago

Same problem for me

hadley commented 3 years ago

What operating systems are y'all using?

hadley commented 3 years ago

Also, it would be useful to know if this is still happening with latest multidplyr; it now requires latest callr, which is likely to fix the problem.

hadley commented 3 years ago

Given the many updates since you filed this issue, I'm going to close. Please file a new issue if it happens again.