When uploading larger files, cgi.FieldStorage decides to store the files
in an unnamed temporary file in /tmp while parsing the form-data. This
is counter-intuitive and might not work, if the partition hosting /tmp/
is too small. Therefore, we overwrite FieldStorage's make_file() method
to use the targetDir as upload path.
While we're at it, we also use NamedTemporaryFile instead of
TemporaryFile, because that lets us use os.link() to create a "copy" of
the file-data without writing it to disk a second time. This does not
work for small data, because small data is kept in an BytesIO object and
thus never written to file automatically. For this case, we keep the old
code, that's writing down files manually.
We have to inline-define CustomFieldStorage, because FieldStorage will
instantiate a new FieldStorage instance for parsing the parts of a
multipart/form-data body and thus we cannot pass targetDir via
init() argument.
Currently only tested on Python 3.7 locally. Thanks for providing automatic testing on the PR :+1:
When uploading larger files, cgi.FieldStorage decides to store the files in an unnamed temporary file in /tmp while parsing the form-data. This is counter-intuitive and might not work, if the partition hosting /tmp/ is too small. Therefore, we overwrite FieldStorage's make_file() method to use the targetDir as upload path.
While we're at it, we also use NamedTemporaryFile instead of TemporaryFile, because that lets us use os.link() to create a "copy" of the file-data without writing it to disk a second time. This does not work for small data, because small data is kept in an BytesIO object and thus never written to file automatically. For this case, we keep the old code, that's writing down files manually.
We have to inline-define CustomFieldStorage, because FieldStorage will instantiate a new FieldStorage instance for parsing the parts of a multipart/form-data body and thus we cannot pass targetDir via init() argument.
Currently only tested on Python 3.7 locally. Thanks for providing automatic testing on the PR :+1: