tidyverse / vroom

Fast reading of delimited files
https://vroom.r-lib.org
Other
621 stars 60 forks source link

Silence warning messages when parsing fails #499

Closed matiasandina closed 1 year ago

matiasandina commented 1 year ago

I am reading many files using read_csv as follows

purrr::map(files, ~wrapper_of_read_csv(.x, col_types = col_types) %>% bind_rows()

I get several warning messages regarding the types of the columns

Warning messages:
1: One or more parsing issues, call `problems()` on your data frame for details, e.g.:
  dat <- vroom(...)
  problems(dat)

I can reproduce this warning with

toy <- data.frame(x = c("123", "THIS WILL FAIL", "15"))
readr::write_csv(toy, "toy.csv")
readr::read_csv("toy.csv", col_types = list(x = readr::col_double()))
#> Warning: One or more parsing issues, call `problems()` on your data frame for details,
#> e.g.:
#>   dat <- vroom(...)
#>   problems(dat)
#> # A tibble: 3 × 1
#>       x
#>   <dbl>
#> 1   123
#> 2    NA
#> 3    15

Created on 2023-06-16 with reprex v2.0.2

In my case, I am expecting the input data to be contaminated and I can use the col_double() behavior to coerce things to NA. My wrapper_of_read_csv will handle these NA according to rules and throw proper warnings, but the many warnings dispatched by vroom (one per file) buries these warnings into There were n warnings (use warnings() to see them). Would it be possible to silence these warnings inside vroom when failing to parse (just as we have show_col_types = FALSE) ? I believe it would be cleaner than using suppressWarnings() or use something like options(warn=-1) to momentarily remove warnings from the reading.

hadley commented 1 year ago

Our advice would be to use suppressWarnings() here.