RFC: Win32 rmdir() is non-blocking, intended or not?

[p5pRT]

Migrated from rt.perl.org#123958 (status was 'open')

Searchable as RT123958$

[p5pRT]

From @wchristian

Described in prosaic form​: If perl tries to rmdir() a director to which a different non-child process has an open handle\, then rmdir() will return success; however the delete will only be scheduled and not actually executed until all other handles on the directory are released.

I have reproduction steps\, and the equivalent of a windows strace (obtained with procmon). Both of them are attached as text files.

Is this something that should be fixed by way of a change in core behavior\, so rmdir in perl is always blocking? Or something that should be documented as a feature on windows in perlport and on the rmdir page? Other options?

[p5pRT]

From @wchristian

non_blocking_log.csv

[p5pRT]

From @wchristian

# this shell hereafter is shell 1 mkdir scratch cd scratch mkdir foo

# the following shell hereafter is shell 2 start cmd

# in shell 2 perl -e "opendir(FH\,q[foo]);sleep(99999)"

# in shell 1 perl -e "undef $!; $ret = rmdir(q[foo]); print qq[ret = -$ret- : err = -$!-\n]" # ret = -1- : err = -- dir # foo will still be present perl -e "undef $!; $ret = rmdir(q[foo]); print qq[ret = -$ret- : err = -$!-\n]" # ret = -0- : err = -Permission denied- dir # foo will still be present

# in shell 2 ctrl+c

# in shell 1 dir # foo will be gone perl -e "undef $!; $ret = rmdir(q[foo]); print qq[ret = -$ret- : err = -$!-\n]" # ret = -0- : err = -No such file or directory-

[p5pRT]

From @tonycoz

On Sun Mar 01 06​:04​:28 2015\, Mithaldu wrote​:

Described in prosaic form​: If perl tries to rmdir() a director to which a different non-child process has an open handle\, then rmdir() will return success; however the delete will only be scheduled and not actually executed until all other handles on the directory are released.

Do you get the same result if you disable your anti-virus?

From looking at the code\, Win32 perl just calls rmdir()\, which at least in VC 6.0 and VC Express 2013 just calls RemoveDirectory().

Tony

[1] the only releases I have CRT source for

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @tonycoz

On Sun Mar 01 15​:15​:54 2015\, tonyc wrote​:

On Sun Mar 01 06​:04​:28 2015\, Mithaldu wrote​:

Described in prosaic form​: If perl tries to rmdir() a director to which a different non-child process has an open handle\, then rmdir() will return success; however the delete will only be scheduled and not actually executed until all other handles on the directory are released.

Do you get the same result if you disable your anti-virus?

From looking at the code\, Win32 perl just calls rmdir()\, which at least in VC 6.0 and VC Express 2013 just calls RemoveDirectory().

Actually\, reading the RemoveDirectory() documentation\, it does just mark the directory for deletion​:

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365488%28v=vs.85%29.aspx

So I guess it could use an entry in perlport.

Tony

[p5pRT]

From @wchristian

I don't have an antivirus that could skew the results in any way.

Additionally the documentation agrees with me​:

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365488(v=vs.85).aspx

Note specifically the remarks section. :)

[p5pRT]

From @wchristian

So I guess it could use an entry in perlport.

Alright\, thanks. A perlport entry also merits a corresponding pointer in the doc for rmdir itself\, yes?

[p5pRT]

From @tonycoz

On Sun\, Mar 01\, 2015 at 03​:32​:32PM -0800\, Christian Walde via RT wrote​:

So I guess it could use an entry in perlport.

Alright\, thanks. A perlport entry also merits a corresponding pointer in the doc for rmdir itself\, yes?

Yes.

[p5pRT]

From @bulk88

On Sun Mar 01 06​:04​:28 2015\, Mithaldu wrote​:

Described in prosaic form​: If perl tries to rmdir() a director to which a different non-child process has an open handle\, then rmdir() will return success; however the delete will only be scheduled and not actually executed until all other handles on the directory are released.

I have reproduction steps\, and the equivalent of a windows strace (obtained with procmon). Both of them are attached as text files.

Is this something that should be fixed by way of a change in core behavior\, so rmdir in perl is always blocking? Or something that should be documented as a feature on windows in perlport and on the rmdir page? Other options?

NtDeleteFile supposedly is instant according to internet rumor https://msdn.microsoft.com/en-us/library/windows/hardware/ff566435%28v=vs.85%29.aspx

DeleteFile/RemoveDirectory set the "delete on close" flag (which is NtSetInformationFile then enum FileDispositionInformation http​://doxygen.reactos.org/de/d06/dll_2win32_2kernel32_2client_2file_2dir_8c_adb85cb9bf296818b34b06aaff3cd91a6.html#adb85cb9bf296818b34b06aaff3cd91a6 ).

Here is a story related to NtDeleteFile.

I often use a small context menu tool called Unlocker to kill handles to a file or a directory so I can delete them in explorer. The TCL/TK git-gui apps go bonkers if a file handle is closed under them\, they throw a TCL exception\, but if you "X" close the window\, it throws an exception the file handle is invalid in a popup iwndow\, then OK on the unhandled TCL exception to close the popup\, then "X" close the window again\, it throws an exception the file handle is invalid with the popup\, ad infinitum until I kill the wish.exe process. cmd.exe\, if I close the handle cmd.exe has to the CWD\, I get "the directory is invalid" on almost every command I try to run. I have to cd out of the now gone dir to get cmd.exe working again.

Now\, Im not sure what errno/GetLastError an app will get if it uses its handle after the file was yanked out from under it with NtDeleteFile (the above examples would've been returning ERROR_INVALID_HANDLE/6). So while yanking the file out is fine with POSIX app\, you will be exposing untested error handling pathways in straight Win32 apps. So I think using NtDeleteFile will cause breakage with other apps unless you are going to argue "deleting an in-use file" is undefined behavior so if the other Win32 app freaks out\, it is your fault for telling perl to delete the file.

-- bulk88 ~ bulk88 at hotmail.com

[p5pRT]

From @bulk88

rug.jpg

[p5pRT]

From @ikegami

On Sun\, Mar 1\, 2015 at 9​:04 AM\, Christian Walde \perlbug\-followup@​perl\.org wrote​:

# New Ticket Created by Christian Walde # Please include the string​: [perl #123958] # in the subject line of all future correspondence about this issue. # \<URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=123958 >

Described in prosaic form​: If perl tries to rmdir() a director to which a different non-child process has an open handle\, then rmdir() will return success; however the delete will only be scheduled and not actually executed until all other handles on the directory are released.

The standard file navigator (Windows Explorer aka "My Computer") behaves the same way.

[p5pRT]

From @wchristian

@​bulk88​:

I've been thinking and i'd like to try and see if rmdir could be implemented such that it detects when a deletion was scheduled\, but not immediately carried out and returns an appropiate error.

I'd like to do that because it would make the non-blocking behavior self-documenting.

However looking around the perl source code it seems that rmdir is implemented by a Win32 api function ALSO called rmdir\, which was apparently deprecated [1]\, so its documentation is not available anymore\, and the function _rmdir it points to [2] does not indicate its behavior in respect to scheduling. I also cannot find any documentation in the reactos doxygen of how it is implemented.

Can you please try and have a look to see if you can find it?

Otherwise\, would it be acceptable to add the equivalent of a -d check to the rmdir implementation in win32.c that generates an error if successful after a successful call to the Win32 api rmdir?

[1] https://msdn.microsoft.com/en-us/library/ms235318.aspx [2] https://msdn.microsoft.com/en-us/library/wt8es881.aspx

[p5pRT]

From @ap

* Christian Walde via RT \perlbug\-followup@&#8203;perl\.org [2015-03-04 12​:30]​:

I've been thinking and i'd like to try and see if rmdir could be implemented such that it detects when a deletion was scheduled\, but not immediately carried out and returns an appropiate error.

I'd like to do that because it would make the non-blocking behavior self-documenting.

That seems liable to cause as many problems as it solves.

The dilemma is this​: the deletion *has* been scheduled.

If you return an error\, and yet the deletion eventually takes place\, that is liable to confuse code written with the usually reasonable assumption that an error means the deletion failed.

Are you going to cancel the deletion to make it consistent? (Assuming that is even possible.) On one hand\, that would fix the problems that would be experienced by code for which an eventual removal in spite of the error is a problem.

OTOH\, it will make directory removal fail far more often on Windows than it does on Unixy platforms. Code that would previously happen to naïvely work fine would start to require special measures to support Windows. That seems unsatisfactory too.

(Maybe claim EINTR? That might make at least *some* programs happen to port over with no Windows-specific effort… possibly.)

Is there any good option here?

If all options are in fact going to suck one way or another anyway\, then the one that involves the least code seems like the preferable one.

Regards\, -- Aristotle Pagaltzis // \<http​://plasmasturm.org/>

[p5pRT]

From @wchristian

Thanks for the thoughts aristotle\, you're right.

Returning an error when it is going to be deleted is incorrect. Removing the deletion flag would also be possible\, but that then might pull out the rug under other programs that might wish to set the delete flag\, have discovered to be there and been content with it.

The only thing i'll need to look at the code for though is the case when a directory has already been marked for deletion. Currently that results in "EPERM"\, which is less than useful.