andersundsehr / aus_driver_amazon_s3

Provides a TYPO3 FAL driver for the Amazon Web Service S3
GNU Lesser General Public License v3.0
21 stars 40 forks source link

Filelist module (files tree) not working due to recursive folders fetch #155

Open soee opened 1 month ago

soee commented 1 month ago

Hi, I am not 100% sure if this case should be addressed directly in the extension, but probably yes.

We have the S3 storage with some folders and thousands of files moved from time to time between them.

Some time ago the folder tree stopped working - when the reload from the server action was triggered, we had only notification that something went wrong on the server side.

After some investigation on local environment it seems that issue is/was caused by the data returned from the S3 and how the https://github.com/TYPO3/typo3/blob/main/typo3/sysext/backend/Classes/Tree/FileStorageTreeProvider.php#L38 works.

So in our case the S3 driver was used for the dedicated storage (defined as separate storage in the TYPO3). When TYPO3 sends request to ajax/filestorage/tree/fetchData to fetch the folders in the tree it iterates over storages and fetches the folders inside of them. If the storage is browsable and marked as expanded in BE use settings, it will try to fetch subfolders recursively. When the folders from the S3 bucket are returned, there is also a / prefix/folder returned which is later threated as root folder of storage and TYPO3 tries again to fetch all subfolders for it, and again S3 API returns a / prefix/folder and the loop continues.

For a temporary workaround I have added simple check

if ($folderCandidate['Prefix'] === '/') {
    continue;
}

in the https://github.com/andersundsehr/aus_driver_amazon_s3/blob/master/Classes/Driver/AmazonS3Driver.php#L937

I would like to know, what you think about this case, and what would be the best approach to avoid such issues.