Lagerregal
commented
6 years ago Hey, thanks for your report! Which version do you use? What have you done to "sync a bucket from one to another S3"? I don't know the feature to do a "sync" in TYPO3
Closed dni closed 6 years ago
Lagerregal
commented
6 years ago Hey, thanks for your report! Which version do you use? What have you done to "sync a bucket from one to another S3"? I don't know the feature to do a "sync" in TYPO3
dni
commented
6 years ago i had the transfer to a new bucket. so i created a new bucket and synchronized it with the old. this part has nothing to do with typo3. then i changed the settings for the storage in typo3.
use the aws cli to sync old data from the old bucket.
aws s3 sync s3://oldbucket/ s3://newbucket/
after that in FAL this error appears if you select a folder in the storage. the fal driver tries to retrieve the dir from s3, where it doesnt exist. and so this error happens.
i thought this had something todo with this: https://github.com/aws/aws-cli/issues/912
This behavior is known. The reason why the sync command behaves this way is that s3 does not physically use directories. There are only buckets and objects. Objects have prefixes that act like directories, but s3 does not designate a specific physical object to be a directory.
and somehow the driver should not try to retrieve the dir from s3 and just create it for the FAL models
i am using version 1.6.0
Lagerregal
commented
6 years ago Thanks for your feedback! I don't think this is an issue of the FAL driver - this seems to be an issue from aws-cli. So I will close this issue for now - thanks for your workaround!
dni
commented
6 years ago no! thats not true, s3 is doing its thing, the problem is fal expects s3 the have a folder, which it doesnt have, so FAL throws an error file doesnt exists, while trying the get a folder which doesnt exist!
You dont have the Problem if you create a Folder via the FAL Backend, but if you create a folder elsewhere you will run into this error in the FAL driver.
The issue should also be there if you create a folder with a file in the S3 client and then after this try to access the folder via the typo3 backend!
Lagerregal
commented
6 years ago Ah, have you tried the Scheduler task "File Abstraction Layer: Update storage index"?
dni
commented
6 years ago nope! :), im gonna try now... ok seems to work, i keep on looking at it ;), way better then my solution creating dirs by hand... thank you didnt know about that Task.
dni
commented
6 years ago hate to say it... but the Update Storage index Task, doesnt cut it, still getting the same error, transfered a project to new bucket and subdirs arent opening, i think this is a real issue with the driver, creating folder by hand with this much folders isnt fun at all :)
dni
commented
6 years ago maybe i find some time, sooner or later to investigate this myself in the code, and maybe write a PR.
Teaminternet-medi
commented
5 years ago I can confirm that behavior, the workaround works too. And what's really weird, i have the same S3 source for 20 Installation each of it with its own Bucket User but its enough to creat the folder in one Typo3 instalation and from then its fixed for all. So maybe it is an S3 related issue too.
dni
commented
5 years ago hey! yes i'm not alone anymore! pretty sure its with the driver, AWS S3 documentation says, there is no Folders in S3, but FAL requires Folders, i bet this is the problem, in s3 there is no "file" for a folder, but in FAL tries to get the file for the folder and throws file doesnt exist! which is correct because there are no folders in S3. and if you create a Folder FAL everything is fine because the Folder will be set in the database, but if the non existing "folder" is created elsewhere the driver wont create it and thinks its there because of the filelinks.
hi! i've noticed when you sync a bucket from one to another S3 isnt creating a directories, although they are there in S3 the fal driver isnt creating the directories, which leads to the error below.
This error happens when you sync a bucket and then trying to access a subdirectory from the Filelist Module. My workaround is, recreate all subdirectories via the Filelist Module, although the subdirectories are shown, which you think it means that they exist, you can create the directory without a directory exists error. And after that everything works as expected.