nurumaik
commented
1 year ago Updated example that actually leads to bugs
create_tiff_file.py
#!/usr/bin/env python
import numpy as np
import PIL.Image as Image
# Create really big file so it will take time to load. But even bigger file will cause libtiff to complain that we are trying to zip-bomb it
Image.fromarray(np.random.rand(5000, 30000) > 0.9) \
.save("example.tif", format="TIFF", compression="tiff_lzw")reproduce.py
#!/usr/bin/env python
from threading import Thread
import os
from collections import deque
finish = False
def load_files():
descriptors = deque()
while not finish:
try:
fd = os.open("test", os.O_RDONLY)
descriptors.append(fd)
except: # in case of too many descriptors
os.close(descriptors.pop())
for d in descriptors:
os.close(d)
print("success")
t = Thread(target=load_files)
import PIL.Image as Image
print("Right before the with-block", flush=True)
with Image.open("example.tif") as mask:
print("Right in the beginning of the with-block", flush=True)
t.start()
mask.load() # replace it with time.sleep(10) and it works just fine without exceptions
print("Right after load()", flush=True)
print("Right after the with-block", flush=True)
finish = True
t.join()Steps to reproduce:
touch test
./create_tiff_file.py
./reproduce.pyNote: you need to delete example.tif and test before every iteration of running this. otherwise the OS might cache the files and thus the problem won't reproduce because libtiff will load file too fast
Expected behavior:
load_files outputs "success"
Actual behavior:
Right in the beginning of the with-block
Right after load()
Right after the with-block
Exception in thread Thread-1 (load_files):
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "reproduce.py", line 17, in load_files
os.close(d)
OSError: [Errno 9] Bad file descriptorCause of bug
-
tiff loader creates file duplicate of file descriptor for use in external c library code https://github.com/python-pillow/Pillow/blob/5832362215f5fa661fa05ad31e6c97aab87d410a/src/PIL/TiffImagePlugin.py#L1256
-
As comments say "libtiff closes the file descriptor, so pass in a dup.", libtiff probably closes
fpafter decoding here https://github.com/python-pillow/Pillow/blob/5832362215f5fa661fa05ad31e6c97aab87d410a/src/PIL/TiffImagePlugin.py#L1298 -
Pillow closes
fpagain https://github.com/python-pillow/Pillow/blob/5832362215f5fa661fa05ad31e6c97aab87d410a/src/PIL/TiffImagePlugin.py#L1308
If another thread decides to open new file between steps 2 and 3 (like load_files in example), it may get file descriptor that was just closed by libtiff. Then, in step 3, pillow closes this file, making descriptor invalid
radarhere
wiredfool
crazyyao0
scskrause
If I open an LZW-compressed binary TIFF file using pillow and load its data, pillow calls the "close" syscall on one of the two relevant file descriptors twice. The second call results in a "EBADF (Bad file descriptor)" error. The program doesn't crash and everything seems to work ok. But if multithreading is used and a race condition happens, this might crash the program.
In this example, the expected behaviour would be that there is only one "close" syscall on each of the two relevant file descriptors and that there is no "EBADF" error. Below, I show how to reproduce the problem.
Also, I want to note that this caused my training of a neural network to crash once. It happened because Pytorch's
DataLoaderuses multithreading and a race condition happened.