Getting OSError(22, 'Invalid argument') instead of deleting files once copied (i.e. move)

[felciano]

I am trying to configure TVNamer to remove files after they have been successfully renamed and copied to a Synology NAS. The copy appears to work fine, but at the end of the copy I see

OSError(22, 'Invalid argument')

printed to the screen and the file isn't removed. My settings include:

"move_files_enable": true, 
"overwrite_destination_on_rename": true,
"overwrite_destination_on_move": true,
"always_move": true,

I've confirmed that the user account has read-write access to the file. Is there some way of getting additional info on the specific operation that resulted in the above OS error?

[dbr]

Hm, strange.. In tvnamer/main.py you could change:

    try:
        cnamer.rename(
            new_fullpath = newFullPath,
            always_move = Config['always_move'],
            always_copy = Config['always_copy'],
            leave_symlink = Config['leave_symlink'],
            force = Config['overwrite_destination'])
    except OSError, e:
        log().warn(e)

to:

except OSError, e:
    import traceback; traceback.print_exc()
    log().warn(e)

..and rerun, and you'll get a full traceback

[felciano]

Well, that might be part of the issue: I don't see any use of

cnamer.rename

in my tvnamer/main.py. I just did a fresh install of version 2.3, and it looks like most of those calls are using cnamer.newPath instead. Some of the other parameters appear to be different as well (e.g. force = Config['overwrite_destination_on_move']) instead of force = Config['overwrite_destination']))

Am I using the wrong build?

[felciano]

I took a chance and added the traceback dump to the code that calls newPath. The error occurs when tvnamer tries to copy over the bits, last access time, etc via shutil.copystat. Here's the trace:

Traceback (most recent call last):
  File "/Users/myuser/virtualenvs/mini/movienamer/lib/python2.7/site-packages/tvnamer/main.py", line 118, in doMoveFile
    force = Config['overwrite_destination_on_move'])
  File "/Users/myuser/virtualenvs/mini/movienamer/lib/python2.7/site-packages/tvnamer/utils.py", line 1107, in newPath
    copy_file(self.filename, new_fullpath)
  File "/Users/myuser/virtualenvs/mini/movienamer/lib/python2.7/site-packages/tvnamer/utils.py", line 1018, in copy_file
    shutil.copystat(old, new)
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 22] Invalid argument: '/Volumes/TV Shows/The Following/Season 01/The Following - S01E01 - Pilot.mkv'
OSError(22, 'Invalid argument')

The destination is a volume mounted from a Synology NAS. Is it possible that this is some OS- or drive format incompatibility?

[lahwaacz]

Just to clarify the confusion, cnamer.rename is used in the ver3 branch, whereas version 2.3 indeed uses cnamer.newPath. Your installation should be just fine...

[felciano]

It looks like the problem lines are:

if hasattr(os, 'chflags') and hasattr(st, 'st_flags'):
    os.chflags(dst, st.st_flags)

According to the os.stat docs on http://docs.python.org/dev/library/os.html, st_flags are "user defined flags for file" that may only be available on some Unix systems. Is it possible that these are available on the source file system (Mac OS Extended Journaled) but no longer valid on the destination file system (Synology, EXT4 format)?

FWIW, the value of the st_flags field returned by os.stat for the source file is 1.

[dbr]

Huh, where is that code? In shutil somewhere? Can you attach a complete traceback?

The error is very likely due to a difference between the two file systems - I've seen chflags throw errno 45 "Operation not supported" before, when copying from an HFS+ filesystem to a NFS mounted storage server at work, in which case the exception should just be ignored (like this bug)

...but EINVAL is strange.

http://lists.freebsd.org/pipermail/freebsd-bugs/2007-April/023308.html seems kind of related

[felciano]

The following reproduces the problem:

import shutil
import sys
import traceback
import stat
import os
import errno

orig_copystat = shutil.copystat
def my_copystat(src, dst):
    try:
        orig_copystat(src, dst)
    except Exception as e:
        sys.stdout.write('copystat failed: %s\n' % e)
        traceback.print_exc()

shutil.copystat = my_copystat

shutil.copystat("/Volumes/BigFile/myexample.mkv", "/Volumes/SynMedia/myexample.mkv")

Running the above copystat-test.py produces:

Mini:renamer felciano$ python copystat-test.py 
copystat failed: [Errno 22] Invalid argument: '/Volumes/SynMedia/myexample.mkv'
Traceback (most recent call last):
  File "copystat-test.py", line 11, in my_copystat
    orig_copystat(src, dst)
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 22] Invalid argument: '/Volumes/SynMedia/myexample.mkv'

[felciano]

Any suggestions on how to chase this down further, or bypass it?

[dbr]

I'd say just ignoring errors form the copystat would be fine, something like:

def copy_file(old, new):
    p("copy %s to %s" % (old, new))
    shutil.copyfile(old, new)
    try:
        shutil.copystat(old, new)
    except OSError, e:
        log().warn("Error copying stats from %s to %s: %s" % (old, new, e))

Not ideal, but I don't think the file stats are vital, and hard to error handle more elegantly.. although I suppose catching the specific Errno 22 might be better?