search

andersundsehr / aus_driver_amazon_s3

Provides a TYPO3 FAL driver for the Amazon Web Service S3
GNU Lesser General Public License v3.0
21 stars 40 forks source link

temporary files inside the typo3temp/var/transient directory get not deleted #13

Closed mediaessenz closed 7 years ago

mediaessenz commented 7 years ago

Because of this problem, this directory could become huge (in my case over 30 GByte!). It is filled up with hundreds of files like fal-tempfile-204096768596508673.jpg. They are generated, e.g. by clicking on an integrated S3 drive via the file list module.

After diving deeper into this topic, I found out some more things.

Due to a missing API, the LocalImageProzessors.php forces the generation of this temporary-files for all not local storages. The bad thing is: there is no cleanup after doing this.

Because of this, every time a image from the S3 is needed, which is not still in the (remote) proccessed folder, a copy of the original (!) file is generated in this folder.

Imagine if you have an 100 MB image stored on an S3, TYPO3 generates a local copy of this file every time it needs a new proccessed version of it. One for the filelist Module, one for the preview in the page module, if this file is used somewhere as content, one inside the IRRE element, one for each Crop-Variant, One for the Frontend and so on... In this example, this one File on an S3 generates about 800 MB temporary files inside the transient directory!

Lagerregal commented 7 years ago

Thanks for your patch! I think this issue should be solved by the TYPO3 core - but I think this takes longer. So I added your patch as meantime solution.

mediaessenz commented 7 years ago

I'm assuming you've tested my fix under the supported TYPO3 versions, before merged. I did this only for TYPO3 8.7.3. If yes, when do you plan to release it to TER?

Lagerregal commented 7 years ago

Yes I'v edited your patch before merging. A new release is coming soon, but I strongly recommend to install the extension not via TER anymore. Please use composer / packagist https://github.com/andersundsehr/aus_driver_amazon_s3#installation

mediaessenz commented 7 years ago

I already use composer ;-) But I like to sit on a tag and not on master ;-)

I have checked the changes you overtake and missing the @unlink($tempFilePath); inside the Extractor class. Also the @unlink($localFilePath); inside the addFile method of the Driver Class is not included. Why have you removed that?

Lagerregal commented 7 years ago

The Extractor uses the "getForLocalProcessing" function which is covered by the "temporaryPaths" handling. So we don't want to unlink this file twice ;-) The "addFile" does not store any files in typo3temp - the file which is stored in the php temp directory. However, we shouldn't unlink the file without respecting the new parameter of the DriverInterface "$removeOriginal", see #15