os.remove OSError: [Errno 13] Permission denied

[209274a3-88f5-4e0c-bc19-409bde0abbc7]

BPO	1425127
Nosy	@tim-one, @terryjreedy, @devdanzin, @tjguk, @MojoVampire
Files	os.remove.py os.remove2.py: use the windows remove function os.remove_win.py os.remove3.py os.remove3_py30.py: works with pyton3.0: changed string to bytes os.remove_win_py30.py: works with pyton3.0: changed string to bytes

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['type-bug', 'OS-windows'] title = 'os.remove OSError: [Errno 13] Permission denied' updated_at = user = 'https://bugs.python.org/atila-cheops' ``` bugs.python.org fields: ```python activity = actor = 'terry.reedy' assignee = 'none' closed = True closed_date = closer = 'terry.reedy' components = ['Windows'] creation = creator = 'atila-cheops' dependencies = [] files = ['1891', '1892', '1893', '1894', '13416', '13417'] hgrepos = [] issue_num = 1425127 keywords = [] message_count = 24.0 messages = ['27444', '27445', '27446', '27447', '27448', '27449', '27450', '27451', '27452', '27453', '27454', '83897', '84147', '84148', '84149', '84150', '89214', '115004', '222096', '222258', '222262', '226119', '232338', '232341'] nosy_count = 10.0 nosy_names = ['tim.peters', 'effbot', 'terry.reedy', 'atila-cheops', 'ajaksu2', 'tim.golden', 'cheops', 'rcronk', 'cgohlke', 'josh.r'] pr_nums = [] priority = 'normal' resolution = 'out of date' stage = 'resolved' status = 'closed' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue1425127' versions = ['Python 2.7', 'Python 3.4', 'Python 3.5'] ```

[209274a3-88f5-4e0c-bc19-409bde0abbc7]

When running the following program I get frequent errors like this one

Exception in thread Thread-4:
Traceback (most recent call last):
  File "C:\Python24\lib\threading.py", line 442, in
__bootstrap
    self.run()
  File "os.remove.py", line 25, in run
    os.remove(filename)
OSError: [Errno 13] Permission denied:
'c:\\docume~1\\joag\\locals~1\\temp\\tmpx91tkx'

When leaving out the touch statement(line 24) in the loop of the class, I do not get any errors. This is on Windows XP SP2 with python-2.4.2 (you should have an exe touch somewhere in you path for this to work) Can somebody shed any light on this please?

Thanks in advance

Joram Agten

[b7a711ff-d634-47b2-ad1b-41e5ae806c8b]

Logged In: YES user_id=38376

If Python gives you a permission error, that's because Windows cannot remove the file. Windows does, in general, not allow you to remove files that are held open by some process.

I suggest taking this issue to comp.lang.python. The bug tracker is not the right place for code review and other support issues.

[tim-one]

Logged In: YES user_id=31435

The problem is that there's no reason to believe anything he did here _does_ leave files open. I can confirm the "permission denied" symptom, and even if I arrange for the call to "touch" to run a touch.bat that doesn't even look at the filename passed to it (let alone open or modify the file).

I also see a large number of errors of this sort:

Exception in thread Thread-8:
Traceback (most recent call last):
  File "C:\python24\lib\threading.py", line 442, in __bootstrap
    self.run()
  File "osremove.py", line 21, in run
    touch(filename)
  File "osremove.py", line 8, in touch
    stdin=None, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
  File "C:\python24\lib\subprocess.py", line 490, in __init__
    _cleanup()
  File "C:\python24\lib\subprocess.py", line 398, in _cleanup
    inst.poll()
  File "C:\python24\lib\subprocess.py", line 739, in poll
    _active.remove(self)
ValueError: list.remove(x): x not in list

Those are clearly due to subprocess.py internals on Windows, where the poll() and wait() methods and the module internal _cleanup() function aren't called in mutually threadsafe ways. _Those errors can be stopped by commenting out the _cleanup() call at the start of Popen.\_init__() (think about what happens when multiple threads call _cleanup() at overlapping times on Windows: all those threads can end up trying to remove the same items from _active, but only one thread per item can succeed).

The "permission denied" errors persist, though.

So there's at least one class of subprocess.py Windows bugs here, and another class of Windows mysteries. I believe subprocess.py is a red herring wrt the latter, though. For example, I see much the same if I use os.system() to run touch instead.

[209274a3-88f5-4e0c-bc19-409bde0abbc7]

Logged In: YES user_id=1276121

I did post on the python mailing list first, but got no responce there, after further looking into it, I seriously think there is at least one bug here.

here is the link to the post: http://mail.python.org/pipermail/python-list/2006- February/323650.html

[b7a711ff-d634-47b2-ad1b-41e5ae806c8b]

Logged In: YES user_id=38376

The problem is that there's no reason to believe anything he did here _does_ leave files open.

Except that he's hitting the file system quite heavily, and asyncronously. My guess is that Windows simply gets behind (a quick filemon test indicates that this is indeed the case; just before a crash, I see the events CREATE/SUCCESS, QUERY/SUCCESS, QUERY/SUCCESS, WRITE/SUCCESS, and OPEN/SHARING VIOLATION for the failing file, with lots of requests for other files interleaved).

Unless someone wants to fix Windows, a simple workaround would be to retry the os.remove a few times before giving up (with a time.sleep(0.1) in between).

[tim-one]

Logged In: YES user_id=31435

[/F]

Except that he's hitting the file system quite heavily,

Except that _without_ the call to touch(), he's hitting it even more heavily, creating and destroying little files just as fast as the OS can do it in each of 10 threads -- but there aren't any errors then.

and asyncronously.

What's asynch here? The OP's touch() function waits for the spawned process to terminate, and the test driver doesn't try to delete the file until after that.

My guess is that Windows simply gets behind (a quick filemon test indicates that this is indeed the case; just before a crash, I see the events CREATE/SUCCESS, QUERY/SUCCESS, QUERY/SUCCESS, WRITE/SUCCESS, and OPEN/SHARING VIOLATION for the failing file, with lots of requests for other files interleaved).

That's consistent with the symptom reported: an exception raised upon trying to remove the file, but not during any other file operation. Does it tell you more than _just_ that? It doesn't for me.

Unless someone wants to fix Windows,

As above, because removing the call to the internal touch function makes all problems go away it's not obvious that this is a Windows problem.

a simple workaround would be to retry the os.remove a few times before giving up (with a time.sleep(0.1) in between).

Because of the internal threading errors in subprocess.py (see my first comment), the threads in the test program still usually die, but with instances of list.remove(x) ValueErrors internal to subprocess.py.

If I hack around that, then this change to the test program's file-removal code appears adequate to eliminate all errors on my box (which is a zippy 3.4 GHz):

        try:
            os.remove(filename)
        except OSError:
            time.sleep(0.1)
            os.remove(filename)

It's possible that some virus-scanning or file-indexing gimmick on my box is opening these little files for its own purposes -- although, if so, I'm at a loss to account for why a single "os.remove(filename)" never raises an exception when the touch() call is commented out.

OTOH, with the touch() call intact, the time.sleep(0.1) above is not adequate to prevent os.remove() errors if I change the file-writing code to:

    f.write("test" * 250000)

Even boosting the sleep() to 0.4 isn't enough then.

That does (mildly) suggest there's another process opening the temp files, and doing something with them that takes time proportional to the file size. However, the os.remove() errors persist when I disable all such gimmicks (that I know about ;-)) on my box.

It seems likely I'll never determine a cause for that. The bad thread behavior in subprocess.py is independent, and should be repaired regardless.

[b7a711ff-d634-47b2-ad1b-41e5ae806c8b]

Logged In: YES user_id=38376

"Does it tell you more than _just_ that? It doesn't for me."

All requests against the file in question were issued by the python process; there's no sign of virus checkers or other external applications.

Also, whenever things failed, there were always multiple requests for cmd.exe (caused by os.system) between the WRITE request and the failing OPEN request.

My feel, after staring at filemon output, is that this is a problem in the Windows file I/O layer. NTFS queues the various operations, and calling an external process with stuff still in the queue messes up the request scheduling.

[209274a3-88f5-4e0c-bc19-409bde0abbc7]

Logged In: YES user_id=1276121

for the subprocess.py I did the following in a few places

try: _active.remove(self) except: pass

see also bug 1199282 https://sourceforge.net/tracker/index.php?func=detail&aid=1199282&group_id=5470&atid=105470

in my current script I circumvent the "Permission denied" error in the following way:

removed = False
while not removed:
try:
    os.remove(file)
    except OSError, error:
        logger.warning("could not remove file %s, %s"
%(file, error))
        time.sleep(1)
    else:
        removed = True

I also have a virus scanner (Mcafee, corporate stuff), and still get the same behaviour when disabling the virus scanner.

My feel, after staring at filemon output, is that this is a problem in the Windows file I/O layer. NTFS queues the various operations, and calling an external process with stuff still in the queue messes up the request scheduling.

this seems strange to me, since every thread works with its own temp files, and all requests are send one after another to the file I/O layer

[209274a3-88f5-4e0c-bc19-409bde0abbc7]

Logged In: YES user_id=1276121

When running the same script, but now with the windows remove function (cmd /c rm filename) still problems occur, so maybe this is a windows problem after all? or does subprocess do things wrong?

[209274a3-88f5-4e0c-bc19-409bde0abbc7]

Logged In: YES user_id=1276121

further looking into the problem: when using the win32api calls to write to the file, this seems to work. see os.remove_win.py

[209274a3-88f5-4e0c-bc19-409bde0abbc7]

Logged In: YES user_id=1276121

os.remove3.py when using the os.write() function and os.close() function to do the file writing, no problems are seen either.

so this seems to be a bug in the 'normal' file input output functions under windows. they don't seem to be thread safe in a way or another. maybe the close() function returns too early?

[90baf024-6604-450d-8341-d796fe6858f3]

Needs confirmation with recent versions, has nice tests.

[244247bc-c726-4c95-832e-102340f9d78c]

Tested with Python 3.0.1 (r301:69561, Feb 13 2009, 20:04:18) [MSC v.1500 32 bit (Intel)] on win32 (windows xp sp2)

os.remove.py still gives the same error

Exception in thread Thread-4:
Traceback (most recent call last):
  File "c:\Python30\lib\threading.py", line 507, in _bootstrap_inner
    self.run()
  File "os.remove.py", line 25, in run
    os.remove(filename)
WindowsError: [Error 32] The process cannot access the file because it
is being used by another process:
'c:\\docume~1\\agtenjo\\locals~1\\temp\\tmpcwbddg'

os.remove2.py still gives the same error

c:\docume\~1\agtenjo\locals\~1\temp\tmpa3plim The process cannot access the file because it is being used by another process.

[244247bc-c726-4c95-832e-102340f9d78c]

Tested with Python 3.0.1 (r301:69561, Feb 13 2009, 20:04:18) [MSC v.1500 32 bit (Intel)] on win32 (windows xp sp2) (pywin 213)

os.remove2_py30.py gives no error

os.remove_winpy30.py gives no error

[244247bc-c726-4c95-832e-102340f9d78c]

os.remove2_py30.py gives no error should be os.remove3_py30.py gives no error

[244247bc-c726-4c95-832e-102340f9d78c]

touch.exe can be found here: http://www.helge.mynetcologne.de/touch/index.htm#download

[ff1b4681-2edf-475e-9088-d700d7fdd39c]

Could this problem be associated with bpo-4749? It was found that something goes wrong when two cmd children processes are spawned from different threads, when the first exits, it is closing file handles shared with the first (or something like that) and it's causing a problem with logging in bpo-4749. That bug has been closed since it's not a problem with logging so I'm searching for other similar bugs to see if we can create a new bug that documents the cause and link to these other bugs that are all showing different symptoms of the bug.
Thoughts?

[83d2e70e-e599-4a04-b820-3814bbdb9bef]

@Brian, Tim, any views on this?

[244247bc-c726-4c95-832e-102340f9d78c]

I think this c win32 issue describes a similar problem http://stackoverflow.com/questions/3764072/c-win32-how-to-wait-for-a-pending-delete-to-complete

[99ffcaa5-b43b-4e8e-a35e-9c890007b9cd]

Similar reference regarding the same basic behavior: http://blogs.msdn.com/b/oldnewthing/archive/2012/09/07/10347136.aspx

Short version: Indexing and anti-virus tools prevent deletion from occurring.

Longer version: DeleteFile (and all the stuff that ultimately devolves to DeleteFile) operate in a funny way on Windows. Internally, it opens a HANDLE to the file, marks it as pending deletion, and closes the HANDLE. If no one snuck in and grabbed another HANDLE to the file during that time, then the file is deleted when DeleteFile's hidden HANDLE is closed. Well designed anti-virus/indexing tools use oplocks ( http://blogs.msdn.com/b/oldnewthing/archive/2013/04/15/10410965.aspx ) so they can open a file, but seamlessly get out of the way if a normal process needs to take exclusive control of a file or delete it. Sadly "well-designed" is not a term usually associated with anti-virus tools, so errors like this are relatively commonplace.

Workarounds like using GetTempFileName() and MoveFile() to move the file out of the way will work, though I believe they introduce their own race conditions (the temp file itself is created but the HANDLE is closed immediately, which could mean a race to open the empty file by the bad anti-virus that would block MoveFile()).

Basically, if you're running on Windows, and you're using unfriendly anti-virus/indexing tools, there is no clean workaround that maintains the same behavior. You can't keep creating and deleting a file of the same name over and over without risking access denied errors.

That said, you could probably get the same results by opening and closing the file only once. Change from the original pseudocode:

while 1:
    with open(myfilename, ...) as myfile:
        myfile.write(...)
    do_stuff_with(myfilename)
    os.remove(myfilename)

to (assuming the default file sharing permissions are amenable):

with open(myfilename, ...) as myfile:
    while 1:
        myfile.write(...)
        myfile.flush()
        myfile.seek(0)
        do_stuff_with(myfilename)
        myfile.truncate()

Same basic pattern, except this time, you're rebuilding the same file over and over without ever leaving it unowned long enough for anti-virus/indexing to swoop in and steal it from you.

[99ffcaa5-b43b-4e8e-a35e-9c890007b9cd]

Please pretend I didn't leave off the actual os.remove call at the end of my second example that bypasses the issue.

[terryjreedy]

Unless someone claims and can show that there is a bug in current cpython on current Windows (not xp), I think this should be closed. Josh, I interpret your post(s) as suggesting that there is not, or am I over-interpreting.

[99ffcaa5-b43b-4e8e-a35e-9c890007b9cd]

I think you're overinterpreting. The bug probably still exists on Windows if you're using a poorly designed anti-virus or indexing tool; nothing fundamental has changed in how files are deleted on Windows since this was opened.

[terryjreedy]

I sometimes have problems deleting files in Windows Explorer. Sometimes I have to wait for reboot, sometimes I have to login as admin. None of this is a bug in Python. The original post, nearly 9 years ago, was not obviously a Python bug report but appears to be question about quirks in Windows. Two Python experts disagreed whether there is a Python bug.

In any case, an open issue needs a repeatable test case that demonstrates a problem with current Python running on a currently supported OS, preferably without 'poorly designed' 3rd party software involved.