[FEATURE] Add "base folder" configuration setting

[cweiske]

The base folder allows us to use the same bucket for multiple projects by forcing every project/TYPO3 instance into an own prefix:

mybucket/
+ project1/
| + file23.jpg
| + subfolder/
| | + subfile42.png
| + _processed_/
|   +  ...
+ project2/
  + file2.jpg
  + _processed_/
    +  ...

The prefix is not visible in the TYPO3 UI (file list, file information) and gets added transparently to S3 requests, and is removed from S3 responses as well.

My initial version used a simpler approach by just overriding getRootLevelFolder() and getParentFolderIdentifierOfIdentifier(), but this led to inconsistencies in the UI, and the processing folder could not be moved into the base folder because of inconsistencies inside the TYPO3 API.

[Lagerregal]

Hi @cweiske, out of curiosity: why don't you just use the TYPO3's file mount feature? This looks exactly like your use case (without implementing this special feature in the fal driver). https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Backend/AccessControl/MoreAboutFileMounts/Index.html

[christophlehmann]

Hey together, it would be really nice to have the base folder setting. With it, buckets can also be used for other purposes (we do it), that are not directly related to TYPO3, but to the project.

Given a have the following folders in the bucket and only wanna see typo3-files/ in TYPO3, then the setting is very handy

• typo3-files/
• private/
• acceptance-screenshots/
• styleguide/v1/

I would like to see it merged 👍

[Lagerregal]

To be honest I still don't see a good use-case for the "base folder". Using a single S3 bucket for multiple use-cases does not look like a best practice architecture to me. For each use-case there should be a separate S3 bucket (e.g. don't mix private and public data in a single bucket). There are serveral reasons for it: Splitting up permissions is much simpler on a bucket level instead off a folder level (or just "prefix" level because AWS S3 does not know "folders") https://stackoverflow.com/questions/32671484/is-it-better-to-have-multiple-s3-buckets-or-one-bucket-with-sub-folders/62232707#62232707 Furthermore: If you store multiple use-cases in a single S3 bucket it's a lot of effort to use different encryption keys for each use-case. So supporting a "mixed" bucket is not aligned with the isolation concept of S3 buckets.

If you're using different folders to setup a permissions layer for a single use-case the TYPO3 built in feature of file mounts is exactly what you're looking for. https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Backend/AccessControl/MoreAboutFileMounts/Index.html

[cweiske]

I do not know why it has once been decided that some of the projects I'm working on are sharing the same S3 bucket. What I know is that permissions on the bucket level have all been correctly setup by the admins.

As for "why base folder instead of file mount": File mounts do only apply to users/groups, but do not administrator users or the TYPO3 code itself to access files outside the project folder. Having the base folder configuration in the driver itself makes accidential errors impossible, because no part of the system is able to read or write files outside the base folder.