markfairbanks / tidytable

Tidy interface to 'data.table'
https://markfairbanks.github.io/tidytable/
Other
449 stars 33 forks source link

Allow integration with `dbplyr` #748

Open markfairbanks opened 1 year ago

markfairbanks commented 1 year ago

Basically this should be possible:

library(dbplyr)
library(tidytable)

query <- memdb_frame(x = 1:3, y = c("a", "a", "b")) %>%
  mutate(x = x * 2)

query
#> # Source:   SQL [3 x 2]
#> # Database: sqlite 3.40.1 [:memory:]
#>       x y    
#>   <dbl> <chr>
#> 1     2 a    
#> 2     4 a    
#> 3     6 b

query %>%
  collect() %>%
  summarize(avg_x = mean(x), .by = y)
#> # A tidytable: 2 × 2
#>   y     avg_x
#>   <chr> <dbl>
#> 1 a         3
#> 2 b         6

Notes: We would have to add dplyr, tidyr, and dbplyr to Suggests and use check_installed() to make sure they're available when working on a tbl_lazy. Also it would probably only be doable once we can re-introduce S3 methods to the package when verb.() is removed.

dibbles21 commented 1 year ago

HI Mark. I'm curious whether this issue would potentially allow the issue here to also be solved?

markfairbanks commented 1 year ago

Yes it would - I'm not 100% sure I want to do this, but it's something I'm considering.

The positive is it would allow integration with dbplyr, the negative is I would need to start considering whether to build in integrations with other packages.

Here's a small list of packages that I would have to consider:

...and I'm guessing there are more I'm not thinking of.

dibbles21 commented 1 year ago

Thank you for confirming. I'll keep an eye on this issue as if resolved it's something I'd be keen on using.

markfairbanks commented 1 year ago

I think the most likely scenario is I create a tidytable extension package (I have no idea what the name would be) that would add in these integrations.

FYI - whether this lives in tidytable or is in an extension package, it wouldn't be possible for probably 6+ months as I need to finish deprecating verb.() syntax. I have been unable to build S3 methods for verb() syntax due to CRAN checks being confused by verb.() syntax. And those S3 methods would be what would allow these integrations to exist.

dibbles21 commented 1 year ago

Thanks Mark, it's helpful to know that extra information on your plans and timescales 😊