Closed nbbn closed 2 years ago
Can you provide an example that does not involve future and promises?
What versions of readr and vroom do you have?
Not really, the problem is an example of some race condition. You have to read file while other code is saving it. Another example would require two separate scripts running at the same time.
> osVersion
[1] "Ubuntu 20.04.3 LTS"
> packageVersion("vroom")
[1] ‘1.5.7’
> packageVersion("readr")
[1] ‘2.1.1’
> R.version
_
platform x86_64-pc-linux-gnu
arch x86_64
os linux-gnu
system x86_64, linux-gnu
status
major 4
minor 1.2
year 2021
month 11
day 01
svn rev 81115
language R
version.string R version 4.1.2 (2021-11-01)
nickname Bird Hippie
@sbearrows can you tell me why this issue is closed? Was it fixed?
@nbbn We're just concluding a wholesale triage of all readr and vroom issues. While we don't deny what you're seeing, your problem seems to only play out in a rather unusual situation. We'll keep it in mind, in case it ends up being connected to something that comes up in more common usage. But we're simply making some realistic decisions about what our near term priorities are.
I'll also tag @DavisVaughan who may have more previous experience with the promises package, in case he has a quick observation about your situation.
I've managed to make an example with just callr. I'm not sure what readr/vroom can improve on beyond trying not to crash here. I think there is just no data to read at some points due to the race condition.
Select the entire code chunk and try and run it all at once. It should do something like:
library(callr)
path <- tempfile(fileext = ".csv")
vroom::vroom_write(mtcars, path, progress = FALSE)
read_it <- function(path) {
for (i in 1:100) {
vroom::vroom(path)
}
NULL
}
write_it <- function(path) {
for (i in 1:100) {
vroom::vroom_write(mtcars, path, progress = FALSE)
}
NULL
}
bg_read <- r_bg(func = read_it, args = list(path = path))
bg_write <- r_bg(func = write_it, args = list(path = path))
bg_read$wait()
bg_write$wait()
bg_read$get_result()
# > Error in get_result(out, private$options) :
#> callr subprocess failed: could not start R, exited with non-zero status, has crashed or was killed
#> Type .Last.error.trace to see where the error occurred
bg_write$get_result()
Note that it wasn't working in a reprex for whatever reason. The reprex just ran infinitely.
So this implies that the background session that is reading the file is crashing. You can reproduce that in your master session with:
library(callr)
path <- tempfile(fileext = ".csv")
vroom::vroom_write(mtcars, path, progress = FALSE)
read_it <- function(path) {
for (i in 1:100) {
vroom::vroom(path)
}
NULL
}
write_it <- function(path) {
for (i in 1:100) {
vroom::vroom_write(mtcars, path, progress = FALSE)
}
NULL
}
bg_write <- r_bg(func = write_it, args = list(path = path))
read_it(path)
This should crash your R session.
I can make this occur with data.table too, that doesn't crash but it errors because it is trying to read an empty file:
library(callr)
path <- tempfile(fileext = ".csv")
vroom::vroom_write(mtcars, path, progress = FALSE)
read_it <- function(path) {
for (i in 1:100) {
data.table::fread(file = path)
}
NULL
}
write_it <- function(path) {
for (i in 1:100) {
data.table::fwrite(mtcars, path, showProgress = FALSE)
}
NULL
}
bg_read <- r_bg(func = read_it, args = list(path = path))
bg_write <- r_bg(func = write_it, args = list(path = path))
bg_read$wait()
bg_write$wait()
bg_read$get_result()
#> Error: callr subprocess failed: File is empty: /var/folders/41/qx_9ygp112nfysdfgxcssgwc0000gn/T//RtmpH4y2Hl/file16dc230ebf10.csv
bg_write$get_result()
#> NULL
Hi! I've got problem of empty csv file while using
readr
package, that usesvroom
. At some moment I've got absolutely empty file, without even header. File is used in Shiny app and has multiple read and write operations. I guess, save operation should be atomic, please find some example below:"empty file!" message shows that for some time, csv file can be read and is empty. If you will switch in this example to
data.table
variant, you will see no messages.