Windows support (ENOTEMPTY issues)

[joliss]

I'd like to re-open the discussion about ENOTEMPTY problems on Windows, as previously raised in #25.

Technical background: On Windows, a file can be "busy" when our process or another process still has an open file handle. This can happen unpredictably; e.g. a webserver might have an open connection serving the file, or a virus scanner might be accessing it. When you remove a busy file (with fs.unlinkSync or fs.unlink), unlinkSync will return successfully, but the file sticks around on the file system until the handle is released. As a result, when rimraf removes a file and then tries to rmdir the containing directory, the rmdir operation can fail with ENOTEMPTY. (This is from memory and hearsay, so some details may be wrong!)

This causes random sporadic failures - see e.g. the stack trace reported in https://github.com/broccolijs/broccoli/issues/232.

There is currently a workaround implemented in rimraf (d819b12733aaa9e26c9d1babe788af408f853bc1, fixing #25). However, it is based on retrying repeatedly with setTimeout, and it only works in the asynchronous version. Of course, this seems pretty hackish and potentially unreliable. We're using rimraf a lot in the Broccoli plugin ecosystem, and it makes me worried that we'll have lots of issues on Windows down the line.

So I'd love it if we could find a proper fix for this issue, rather than working around it.

It surely must be possible to find a fix - and this is why I'm opening this issue again: For example, commands like rmdir /s can't possibly be relying on weird workarounds, can they? And Windows Explorer lets you delete directories, without random (flickering) failures stemming from your virus scanner accessing some file.

More detail for people wishing to dig in: this gist by @stefanpenner; this comment in libuv's fs.c; libuv's unlink implementation on Windows. I wonder if libuv's very roundabout unlink implementation (presumably to mimic POSIX) might be a contributor. Could this be fixable in libuv, or could we come up with a platform-specific C implementation for rimraf on Windows that doesn't use libuv?

[isaacs]

Patch very much welcome.

At this point, there's so much garbage working around weird windows semantics, I'd even consider a "fix" that spawns a child process running rmdir /s /q for directories, though that is also not ideal.

[stefanpenner]

I believe this is an invariant of the underlying file system on windows. :s.

A filename isn't released until all file active handles to it have been released. On Unix, the filename is available immediately after the unlink, regardless of open file handles.

But maybe someone more familiar with the windows platform can tell me I'm wrong and how we can fix...

[stefanpenner]

In all honesty, MS OSS team has been pretty helpful for other issues. Maybe they have some ideas on this one. Reaching out...

[denieler]

Hi! It's really big pain for windows users, but not for all.

[kanongil]

While you can't fully remove an open file on windows, you can move it to another location, which could maybe be used as a workaround? Unfortunately though, if I remember correctly, you need to move it before you unlink it.

[stefanpenner]

Interesting....

[joliss]

I wouldn't even be opposed to blocking until all files become unbusy, if that turns out to be necessary. I'd just like the deletion algorithm to be solid - as in, the proper way to do it on Windows - rather than some ad-hoc workaround.

[jabjab]

I seem to have encountered the same issue on OSX in a Phonegap project.

[17:26:41] Error: ENOTEMPTY, rmdir 'platforms/ios/CordovaLib'
    at Error (native)

The contents of the directory were successfully removed, but the directory itself was not.

[jnslxndr]

Any news on this? I am really having trouble on Windows 7 running a CI server (that needs to build the server stuff on Windows and integrate the ember build afterwards, not my favorite...)

[jnslxndr]

I noticed, Windows 8.1 does not have the issue; just Windows 7. I followed the Windows guides on ember-cli and also tested with the ember-cli-windows helper tool to deactivate defender and the search index. But, no luck.

[jprichardson]

Also having this issue with fs-extra (uses rimraf) after adding appveyor. See more here: https://ci.appveyor.com/project/jprichardson/node-fs-extra/build/1.0.19/job/dlqb9nnn5cw9c5wj (search for ENOTEMPTY). I'll dig around some more to hopefully help with a solution.

[drourke]

I'm on windows 8.1, changing the tmp folder property to hidden fixed it for me.

[ghost]

Experiencing this issue in Windows 7, as well.

[Bouke]

Experiencing this issue on Windows 7 as well. However setting the hidden property didn't resolve the issue.

[avizuber-zz]

Same issue on Windows 7.

[blisst]

I also have the issue on Windows 7

[blisst]

I have found an interesting issue however, which might be related. I am wondering if somehow some npm module does not have symlinking permissions?

I am on Windows 7.

I installed npm can-symlink and it told me that I did not have permissions to create symlinks DESPITE the fact that mklink works fine in an Admin command window.

[isaacs]

@blisst 100% unrelated to rimraf. Did you run can-symlink as an administrator? If so, probably that's a bug to post over there, not here.

[jlogar]

seems like an old old windows issue: http://serverfault.com/a/199994 https://connect.microsoft.com/PowerShell/feedback/details/549039/remove-item-recurse-does-not-work-properly-is-faulty

Doesn't look like MS is jumpin with joy to fix these things.

[silverwind]

Does someone have a test case using rimraf.sync that fails reliably with ENOTEMPTY?

[Fishrock123]

From the above ^ ... if you don't, could you try this node PR and run node's tests with it? https://github.com/nodejs/node/pull/2356

(We use a version of rimraf internally for the tests.)

[stefanpenner]

@silverwind https://gist.github.com/stefanpenner/2cc619b8740fe2463c2a (a gist i made for the original issue, and is linked above – but in-case you missed it)

[aelana]

I am not sure if the workaround in https://github.com/isaacs/rimraf/commit/d819b12733aaa9e26c9d1babe788af408f853bc1 has been overwritten or changed but I have been noticing a lot of these on rimraf async lately. So much so that I have had to revert to spawning a cmd /c rmdir /s /q on error && windows.

Just thought I would mention it in case someone was still tracking and chasing down this issue.

[przemoli]

Hello. Would moving this work: a) Move file to tmp b) Delete file

Is copy behaving differently then unlink?

[shenlong]

This causes issue on Windows 10 too.

I am using package.json to call the scripts file.

"prebuild-master-debug": "rimraf temp  && mkdir temp && git clone --depth=1 web temp/web"

It just hang there as the folder is locked. Can we call an alternative like 'rmdir /s /q' if it timeout?

[isaacs]

Can you please retry with the latest version of rimraf? I believe some recent changes have likely affected this.

[webia1]

Problem still existing,..

[DLiblik]

As a long time Windows developer, I highly doubt this will be resolved by MS as, by default, when you open a handle, it applies a lock, and a lot of existing infrastructure in the OS depends on this - the developer in any situation must explicitly go beyond the default to avoid this. MS (I must imagine) would be reticent to change this 'suddenly' in an OS release or service pack as it would be near impossible to assess (within their own software history) the full impact. The 'rename workaround' - as pointed out above ('move' at the lower layer) only works because originally the technology couldn't track this and it became a well-known workaround for dire support situations where a system couldn't be rebooted and a needed file to update couldn't be released due to a dead process that was still holding a remaining handle (back in the early 90's I'm talking here). And so here we are today with those pipes now buried in the foundation - like 'em or lump 'em, they're highly likely here to stay in the OS. Even Atom fell victim to this (I think this is now fixed), where if Atom was open on a file and you switched branches in git to a branch without that file, the ceiling would come down on you (and ember-cli would crash, leaving you with a mess of tmp sub-folders to hack at with robocopy to mirror to empty folders because rd couldn't handle the file length). It's the downside of MS having been so successful in the enterprise: now too much depends on their original code and they themselves are very frustrated internally as well by not being able to evolve some fundamentals like this - I know this first hand. Sorry the news is unlikely to improve!

[steph643]

This issue prevents Meteor from working correctly under Windows (see https://github.com/meteor/meteor/issues/8485 and https://github.com/meteor/meteor/issues/8663).

[DLiblik]

@joliss I thought of your issue here just now - I shift-deleted (i.e. did not recycle, but actually deleted) a folder on the latest-and-greatest Windows 10 in an Explorer window. Windows chunked away, finished the process without complaint, and yet the folder I deleted is there, and there is a path of folders under it that are all empty. Did it again and it removed the folder - clearly a lingering handle.

More evidence that even MS can't get around it...

[ofarukcaki]

I have the same issue as well

[nuthinking]

Problem still there.

[Lefcott]

Hey! I'm still having the same issue on windows 7. I'm using version 3.0.2.