tidyverse / multidplyr

A dplyr backend that partitions a data frame over multiple processes
https://multidplyr.tidyverse.org
Other
641 stars 75 forks source link

"multiplyr" package do not handle integer64 columns correctly. #19

Closed gigamori closed 8 years ago

gigamori commented 8 years ago

Through multidplyr processing (partition() -> collect()), integer64 columns are converted to "numeric" automatically, however interger columns are not.

Example:

dt <- data.table(x=1:5, y=6:10)
dt$x <- as.integer64(dt$x)

class(dt$x)
# [1] "integer64"
class(dt$y)
# [1] "integer"

dt %>% partition %>% collect -> dt2

class(dt2$x)
# [1] "numeric" -> !!!
class(dt2$y)
# [1] "integer"

My environment is:

R version 3.2.2 (2015-08-14)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

multidplyr package version is:

‘0.0.0.9000’

hadley commented 8 years ago

Unfortunately mutlidplyr does not currently support data.table