Getting "[WinError 206] The filename or extension is too long"

[LukasBommes]

Hey jveitchmichaelis,

thanks for this great tool. It helps me a lot in my application. I ran into the following issue on most of the *.SEQ files that I have. Interestingly, the error does not occur on SEQ files which contain a smaller number of frames, e.g. 200 or 400. However, on the larger SEQ files this happens.

C:\Users\Lukas\Desktop\flir test>python split_seqs.py -i "DJI_0052.SEQ"
INFO:__main__:Loading: DJI_0052.SEQ
INFO:flirpy.io.seq:Splitting 1 files
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]INFO:flirpy.io.seq:Splitting DJI_0052.SEQ into C:\Users\Lukas\Desktop\flir test\DJI_0052
2697it [04:31,  9.94it/s]
INFO:flirpy.io.seq:Extracting metadata
  0%|                                                                                            | 0/1 [04:32<?, ?it/s]
Traceback (most recent call last):
  File "split_seqs.py", line 92, in <module>
    folders = splitter.process(files)
  File "C:\Anaconda3\lib\site-packages\flirpy\io\seq.py", line 96, in process
    self.exiftool.write_meta(filemask)
  File "C:\Anaconda3\lib\site-packages\flirpy\util\exiftool.py", line 78, in write_meta
    res = subprocess.call(cmd, cwd=cwd, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
  File "C:\Anaconda3\lib\subprocess.py", line 267, in call
    with Popen(*popenargs, **kwargs) as p:
  File "C:\Anaconda3\lib\subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "C:\Anaconda3\lib\subprocess.py", line 997, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 206] Der Dateiname oder die Erweiterung ist zu lang

The error message in English is "The filename or extension is too long".

Do you have an idea why this happens? I also tried executing the same operation directly in the root directory of my hard drive to ensure short path names, but I get the same error.

Thank you so much.

Lukas

[jveitchmichaelis]

Weird if it works with longer files. It's not that you have a space in your output folder ("flir test")?

Also found this: https://stackoverflow.com/questions/10724642/windowserror-error-206-the-filename-or-extension-is-too-long-after-running worth checking it's not a misleading error.

You could also try running Exiftool directly and see if that fails for some reason.

It seems to be the meta file that's crashing. Can you add a print statement in write_meta to see what cmd or filemake is? e.g. this call:

 File "C:\Anaconda3\lib\site-packages\flirpy\util\exiftool.py", line 78, in write_meta
    res = subprocess.call(cmd, cwd=cwd, stderr=subprocess.PIPE, stdout=subprocess.PIPE)

[LukasBommes]

Thanks for your super fast reply and the hints. I am investigating on this now and will let you know when I found something.

[LukasBommes]

Okay, I printed the cmd in write_meta and found that all 2696 files of my test SEQ-file are passed simultaneously to the Exiftool exe. For the sake of readability I shortened the output a bit by inserting [...]. (Scroll the code below to the right to see the output.)

This might explain why it works for short SEQ files which have only 200 frames, but fails for long SEQ files with many frames. Wouldn't it be better to run the Exiftool sequentially on batches of the frames, say 100 frames at a time?

C:\Users\Lukas\Desktop\flirtest\flirpy\scripts>python split_seqs
INFO:__main__:['C:\\Anaconda3\\lib\\site-packages\\flirpy-0.1.0-py3.6.egg\\flirpy']
INFO:__main__:Loading: DJI_0052.SEQ
INFO:flirpy.io.seq:Splitting 1 files
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]INFO:flirpy.io.seq:Splitting DJI_0052.SEQ into C:\Users\Lukas\Desktop\flirtest\flirpy\scripts\DJI_0052
                  INFO:flirpy.util.exiftool:cmd: ['C:\\Anaconda3\\lib\\site-packages\\flirpy-0.1.0-py3.6.egg\\flirpy\\bin\\exiftool.exe', 'frame_000000.fff', '-w!', '.txt'] ### subprocess: <module 'subprocess' from 'C:\\Anaconda3\\lib\\subprocess.py'>

2it [00:16,  2.13s/it]
2697it [04:10, 10.77it/s]
INFO:flirpy.io.seq:Extracting metadata
INFO:flirpy.io.seq:Before creating metadata paths
INFO:flirpy.io.seq:Filemask C:\Users\Lukas\Desktop\flirtest\flirpy\scripts\DJI_0052\raw\frame_*.fff
INFO:flirpy.util.exiftool:cmd: ['C:\\Anaconda3\\lib\\site-packages\\flirpy-0.1.0-py3.6.egg\\flirpy\\bin\\exiftool.exe', 'frame_000000.fff', 'frame_000001.fff', 'frame_000002.fff', 'frame_000003.fff', 'frame_000004.fff', 'frame_000005.fff', 'frame_000006.fff', 'frame_000007.fff', 'frame_000008.fff', 'frame_000009.fff', 'frame_000010.fff', 'frame_000011.fff', 'frame_000012.fff', 'frame_000013.fff', 'frame_000014.fff', 'frame_000015.fff', 'frame_000016.fff', 'frame_000017.fff', 'frame_000018.fff', 'frame_000019.fff', 'frame_000020.fff', 'frame_000021.fff', 'frame_000022.fff', 'frame_000023.fff', 'frame_000024.fff', 'frame_000025.fff', 'frame_000026.fff', 'frame_000027.fff', 'frame_000028.fff', 'frame_000029.fff', 'frame_000030.fff', [...], 'frame_002688.fff', 'frame_002689.fff', 'frame_002690.fff', 'frame_002691.fff', 'frame_002692.fff', 'frame_002693.fff', 'frame_002694.fff', 'frame_002695.fff', '-w!', '.txt'] ### subprocess: <module 'subprocess' from 'C:\\Anaconda3\\lib\\subprocess.py'>
  0%|                                                                                            | 0/1 [04:10<?, ?it/s]
Traceback (most recent call last):
  File "split_seqs", line 94, in <module>
    folders = splitter.process(files)
  File "C:\Anaconda3\lib\site-packages\flirpy-0.1.0-py3.6.egg\flirpy\io\seq.py", line 99, in process
    self.exiftool.write_meta(filemask)
  File "C:\Anaconda3\lib\site-packages\flirpy-0.1.0-py3.6.egg\flirpy\util\exiftool.py", line 80, in write_meta
    res = subprocess.call(cmd, cwd=cwd, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
  File "C:\Anaconda3\lib\subprocess.py", line 267, in call
    with Popen(*popenargs, **kwargs) as p:
  File "C:\Anaconda3\lib\subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "C:\Anaconda3\lib\subprocess.py", line 997, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 206] Der Dateiname oder die Erweiterung ist zu lang

[jveitchmichaelis]

Interesting - I don't have this problem on any of the SEQs I've tested with, which are 1800 frames.

The program does already run exiftool in the folder to minimise this, but I guess wildcard expansion kills you. I'm sure there's a way to do this on arbitrary numbers of input files with Exiftool e.g. -ext fff .

OK actually that looks good. If i give you the patch here, give it a go and I can merge if it works:

 def write_meta(self, filemask):

        cwd = os.path.dirname(filemask)

        cmd = [self.path]
        cmd.append("-ext")
        cmd.append("fff")
        cmd.append(".")
        cmd.append("-w!")
        cmd.append(".txt")

        logger.debug(" ".join(cmd))

        res = subprocess.call(cmd, cwd=cwd, stderr=subprocess.PIPE, stdout=subprocess.PIPE)

        return res

Just replace in C:\Anaconda3\lib\site-packages\flirpy-0.1.0-py3.6.egg\flirpy\util\exiftool.py

EDIT: Passes unit tests so good to go in as far as I'm concerned. I'll wait to hear back before i push though.

[LukasBommes]

I tried your patch and now the extraction of meta data works fine! Thanks a lot for fixing this.

There is one more problem at a later step, which occurs with all my SEQ-files. As you can see, n_rgb is zero. I am not sure whether this is because my SEQ-files are IR only and not visual. Or am I wrong? In any case, it would be great if the exception was captured and a more informative error message printed.

C:\Users\Lukas\Desktop\flirpy\flirpy\scripts>python split_seqs
INFO:__main__:Loading: DJI_0052.SEQ
INFO:flirpy.io.seq:Splitting 1 files
  0%|                                                    | 0/1 [00:00<?, ?it/s]INFO:flirpy.io.seq:Splitting DJI_0052.SEQ into C:\Users\Lukas\Desktop\flirpy\flir
py\scripts\DJI_0052
                  INFO:flirpy.util.exiftool:Inside patched write_meta
INFO:flirpy.util.exiftool:cmd: ['C:\\Users\\Lukas\\Anaconda3\\lib\\site-packages\\flirpy-0.1.0-py3.6.egg\\flirpy\\bin\\exiftool.exe', '-ext', 'fff', '.', '-w!',
 '.txt'] ### subprocess: <module 'subprocess' from 'C:\\Users\\Lukas\\Anaconda3\\lib\\subprocess.py'>
2697it [02:53, 15.51it/s]
INFO:flirpy.io.seq:Extracting metadata
INFO:flirpy.util.exiftool:Inside patched write_meta
INFO:flirpy.util.exiftool:cmd: ['C:\\Users\\Lukas\\Anaconda3\\lib\\site-packages\\flirpy-0.1.0-py3.6.egg\\flirpy\\bin\\exiftool.exe', '-ext', 'fff', '.', '-w!',
 '.txt'] ### subprocess: <module 'subprocess' from 'C:\\Users\\Lukas\\Anaconda3\\lib\\subprocess.py'>
INFO:flirpy.io.seq:Copying tags to radiometric
INFO:flirpy.io.seq:Copying tags to preview
100%|███████████████████████████████████████████| 1/1 [07:56<00:00, 476.83s/it]
INFO:__main__:Merging folders
  0%|                                                    | 0/1 [00:00<?, ?it/s]INFO:__main__:Copying: C:\Users\Lukas\Desktop\flirpy\flirpy\scripts\DJI_0052
100%|████████████████████████████████████████████| 1/1 [00:36<00:00, 36.39s/it]
INFO:__main__:2696 infrared frames
0it [00:00, ?it/s]
INFO:__main__:n_infrared 2696, n_rgb 0
Traceback (most recent call last):
  File "split_seqs", line 129, in <module>
    assert get_frame_number(n_rgb) == n_infrared
  File "split_seqs", line 127, in <lambda>
    get_frame_number = lambda i: round(n_infrared*float(i)/n_rgb)
ZeroDivisionError: float division by zero

Also, on my platform natsort is not installed by default. Maybe, you can add it to the install_requires in setup.py?

[jveitchmichaelis]

The script shouldn't be checking rgb stats if you've not also provided rgb images - just a bug in having that set as a default.

Yes, good call on natsort too. I'll add some unit tests for the script I think.

[jveitchmichaelis]

https://github.com/LJMUAstroecology/flirpy/commit/10033b170e391afa8f1bc40eb48d1e95d48293a1 batch process

https://github.com/LJMUAstroecology/flirpy/commit/94c36f4f66f62607484131b1be528bfe27af4e9e version bump + natsort

https://github.com/LJMUAstroecology/flirpy/commit/249bdb2c2354b1b6b8d33578409f38ab9188c60f quick fix for the RGB issue

[jveitchmichaelis]

@LukasBommes this should now be up on PyPI (v0.1.1)