tudo-r / BatchJobs

BatchJobs: Batch computing with R
Other
85 stars 20 forks source link

Directory '...' does not seem to be empty (on Windows) #65

Closed HenrikBengtsson closed 9 years ago

HenrikBengtsson commented 9 years ago

On Windows, I quite frequently (every 10-20th time), makeRegistry() gives an error like:

Error in checkDir(file.dir, create = TRUE, check.empty = TRUE, check.posix = TRUE,  :
  Directory 'T:/BatchJobs/reg_1125313781-files' does not seem to be empty

It's happening at random.

TROUBLESHOOTING: I've modified `checkDir() to also report on what files/directories it finds, e.g.

Error in checkDir(file.dir, create = TRUE, check.empty = TRUE, check.posix = TRUE,  :
  Directory 'T:/BatchJobs/reg_1125313781-files' does not seem to be empty:
test_write_access_file_e6039406205

After running this several times, it's always about one of these test_write_accessfile* being "left behind".

HenrikBengtsson commented 9 years ago

Just to clarify, this is an error I saw more than a year ago as well, so it's nothing new. ... and don't worry, I have a solution that I will commit soon. I just wanted to report this as an issue, in case someone else out there has the same problem and finds this post.

HenrikBengtsson commented 9 years ago

While trying to setup a package torture test for this, I discovered that the error is more likely to occur when creating registries in the current working directory.

Looking at how package tests are setup and configured, I discovered that whether the test working directory is the current directory or a temporary directory is decided by BatchJobs::isExpensiveExampleOk(), which in turn is controlled by system environment variable R_EXPENSIVE_EXAMPLE_OK. Is this just random, or has some one else before me noticed "problems" using the current directory as the working directory?

HenrikBengtsson commented 9 years ago

It's a bit more complicated than I thought. It seems as if file.exists() can return FALSE while the same file is still seen by list.files(). This causes some of the sanity checks of BatchJobs to fail, which is exactly what I reported above. See commit d5721c9 for more details.