edwindj / chunked

Chunkwise Text-file Processing for 'dplyr'
https://edwindj.github.io/chunked
164 stars 7 forks source link

dplyr 0.8.0 #13

Closed romainfrancois closed 5 years ago

romainfrancois commented 5 years ago

On checking reverse dependencies of packages with the release candidate of dplyr, chunked fails with below.

The reason is that filter_() gained a .preserve argument. You should be able to do something like this (not tested):

filter_.chunkwise <- function(.data, ..., .dots. preserve = FALSE){
  .dots <- lazyeval::all_dots(.dots, ...)
  cmd <- if(packageVersion("dplyr") < "0.7.99" ) {
    lazyeval::lazy(filter_(.data, .dots=.dots))
  } else {
    lazyeval::lazy(filter_(.data, .dots=.dots, .preserve = .preserve))
  }
  record(.data, cmd)
}

so that chunked works against both can dplyr and the upcoming 0.8.0.

> revdepcheck::revdep_details(revdep = "chunked")
══ Reverse dependency check ════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════ chunked 0.4 ══

Status: BROKEN

── Newly failing

✖ checking examples ... ERROR
✖ checking tests ...

── Before ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
0 errors ✔ | 0 warnings ✔ | 0 notes ✔

── After ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
❯ checking examples ... ERROR
  Running examples in ‘chunked-Ex.R’ failed
  The error most likely occurred in:

  > ### Name: read_csv_chunkwise
  > ### Title: Read chunkwise data from text files
  > ### Aliases: read_csv_chunkwise read_csv2_chunkwise read_table_chunkwise
  > ###   read_laf_chunkwise
  > 
  > ### ** Examples
  > 
  > # create csv file for demo purpose
  > in_file <- file.path(tempdir(), "in.csv")
  > write.csv(women, in_file, row.names = FALSE, quote = FALSE)
  > 
  > #
  > women_chunked <-
  +   read_chunkwise(in_file) %>%  #open chunkwise connection
  +   mutate(ratio = weight/height) %>%
  +   filter(ratio > 2) %>%
  +   select(height, ratio) %>%
  +   inner_join(data.frame(height=63:66)) # you can join with data.frames!
  > 
  > # no processing done until
  > out_file <- file.path(tempdir(), "processed.csv")
  > women_chunked %>%
  +   write_chunkwise(file=out_file)
  Error: `.preserve` (`.preserve = FALSE`) must not be named, do you need `==`?
  Execution halted

❯ checking tests ...
  See below...

── Test failures ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── testthat ────

> library(testthat)
> library(chunked)
Loading required package: dplyr

Attaching package: 'dplyr'

The following object is masked from 'package:testthat':

    matches

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

> 
> test_check("chunked")
── 1. Error: filter(): can filter rows (@test-verbs.R#28)  ─────────────────────
`.preserve` (`.preserve = FALSE`) must not be named, do you need `==`?
1: expect_equal(tbl_women %>% filter(height > 65) %>% as.data.frame, women %>% filter(height > 
       65)) at testthat/test-verbs.R:28
2: quasi_label(enquo(object), label)
3: eval_bare(get_expr(quo), get_env(quo))
4: tbl_women %>% filter(height > 65) %>% as.data.frame
5: withVisible(eval(quote(`_fseq`(`_lhs`)), env, env))
6: eval(quote(`_fseq`(`_lhs`)), env, env)
7: eval(quote(`_fseq`(`_lhs`)), env, env)
8: `_fseq`(`_lhs`)
9: freduce(value, `_function_list`)
10: withVisible(function_list[[k]](value))
...
23: filter_(.data, .dots = .dots)
24: filter_.data.frame(.data, .dots = .dots) at /Users/romain/git/tidyverse/dplyr/R/manip.r:122
25: filter(.data, !!!dots, .preserve = .preserve) at /Users/romain/git/tidyverse/dplyr/R/dataframe.R:66
26: filter.data.frame(.data, !!!dots, .preserve = .preserve) at /Users/romain/git/tidyverse/dplyr/R/manip.r:113
27: as.data.frame(filter(tbl_df(.data), ..., .preserve = .preserve)) at /Users/romain/git/tidyverse/dplyr/R/dataframe.R:61
28: filter(tbl_df(.data), ..., .preserve = .preserve) at /Users/romain/git/tidyverse/dplyr/R/dataframe.R:61
29: filter.tbl_df(tbl_df(.data), ..., .preserve = .preserve) at /Users/romain/git/tidyverse/dplyr/R/manip.r:113
30: bad_eq_ops(bad, "must not be named, do you need `==`?") at /Users/romain/git/tidyverse/dplyr/R/tbl-df.r:50
31: glubort(fmt_wrong_eq_ops(named_calls), ..., .envir = .envir) at /Users/romain/git/tidyverse/dplyr/R/error.R:37
32: .abort(text) at /Users/romain/git/tidyverse/dplyr/R/error.R:51

══ testthat results  ═══════════════════════════════════════════════════════════
OK: 39 SKIPPED: 0 FAILED: 1
1. Error: filter(): can filter rows (@test-verbs.R#28) 

Error: testthat unit tests failed
Execution halted

2 errors ✖ | 0 warnings ✔ | 0 notes ✔
romainfrancois commented 5 years ago

I'll fix this on dplyr's end. Sorry for the noise.