v23.06.2 Upload images to s3 doesn't work

[yoghiyb]

Describe the Bug

The error is "File path /updates/images/gallery/{image_name} could nto be uploaded to. like https://github.com/BookStackApp/BookStack/issues/2739

Steps to Reproduce

my poilicy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1420044805001",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:ReplicateObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::my-bookstack",
                "arn:aws:s3:::my-bookstack/*"
            ]
        }
    ]
}

1. Go to any document
2. Edit the document
3. Click on "insert an image" and drag and drop an image to upload it
4. See error
5. Check s3 bucket
6. But the image already uploaded to s3

Expected Behaviour

it shouldn't error

Screenshots or Additional Context

Browser Details

No response

Exact BookStack Version

v23.06.2

PHP Version

8.2

Hosting Environment

Docker

[ssddanbrown]

Hi @yoghiyb, Can you get details of the error event from your BookStack error log?:

https://www.bookstackapp.com/docs/admin/debugging/

[yoghiyb]

Hi @yoghiyb, Can you get details of the error event from your BookStack error log?:

https://www.bookstackapp.com/docs/admin/debugging/

sure, @ssddanbrown

[ssddanbrown]

@yoghiyb Thanks, unfortuantely that cuts off some important lines. Can you show the lines from the start of an error (where the date is shown) followed by production.ERROR, along with the next few lines after it (Up to about #10).

[yoghiyb]

@ssddanbrown sorry for the previous mistake,

this is the full log that i regenerated with a different file

Details

``` [stacktrace] #0 /var/www/bookstack/app/Uploads/ImageService.php(128): BookStack\\Uploads\\ImageService->saveNew('logo2.png', '\\x89PNG\\r\\n\\x1A\\n\\x00\\x00\\x00\\rIHD...', 'cover_book', 1) #1 /var/www/bookstack/app/Uploads/ImageRepo.php(119): BookStack\\Uploads\\ImageService->saveNewFromUpload(Object(Illuminate\\Http\\UploadedFile), 'cover_book', 1, 512, 512, true) #2 /var/www/bookstack/app/Entities/Repos/BaseRepo.php(91): BookStack\\Uploads\\ImageRepo->saveNew(Object(Illuminate\\Http\\UploadedFile), 'cover_book', 1, 512, 512, true) #3 /var/www/bookstack/app/Entities/Repos/BookRepo.php(108): BookStack\\Entities\\Repos\\BaseRepo->updateCoverImage(Object(BookStack\\Entities\\Models\\Book), Object(Illuminate\\Http\\UploadedFile), false) #4 /var/www/bookstack/app/Entities/Controllers/BookController.php(183): BookStack\\Entities\\Repos\\BookRepo->update(Object(BookStack\\Entities\\Models\\Book), Array) #5 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): BookStack\\Entities\\Controllers\\BookController->update(Object(BookStack\\Http\\Request), 'test') #6 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(43): Illuminate\\Routing\\Controller->callAction('update', Array) #7 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Routing/Route.php(259): Illuminate\\Routing\\ControllerDispatcher->dispatch(Object(Illuminate\\Routing\\Route), Object(BookStack\\Entities\\Controllers\\BookController), 'update') #8 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Routing/Route.php(205): Illuminate\\Routing\\Route->runController() #9 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Routing/Router.php(798): Illuminate\\Routing\\Route->run() #10 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(141): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}(Object(BookStack\\Http\\Request)) #11 /var/www/bookstack/app/Http/Middleware/Authenticate.php(23): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #12 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): BookStack\\Http\\Middleware\\Authenticate->handle(Object(BookStack\\Http\\Request), Object(Closure)) #13 /var/www/bookstack/app/Http/Middleware/Localization.php(45): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #14 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): BookStack\\Http\\Middleware\\Localization->handle(Object(BookStack\\Http\\Request), Object(Closure)) #15 /var/www/bookstack/app/Http/Middleware/RunThemeActions.php(26): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #16 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): BookStack\\Http\\Middleware\\RunThemeActions->handle(Object(BookStack\\Http\\Request), Object(Closure)) #17 /var/www/bookstack/app/Http/Middleware/CheckEmailConfirmed.php(47): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #18 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): BookStack\\Http\\Middleware\\CheckEmailConfirmed->handle(Object(BookStack\\Http\\Request), Object(Closure)) #19 /var/www/bookstack/app/Http/Middleware/PreventAuthenticatedResponseCaching.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #20 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): BookStack\\Http\\Middleware\\PreventAuthenticatedResponseCaching->handle(Object(BookStack\\Http\\Request), Object(Closure)) #21 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php(78): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #22 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Foundation\\Http\\Middleware\\VerifyCsrfToken->handle(Object(BookStack\\Http\\Request), Object(Closure)) #23 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php(49): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #24 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\View\\Middleware\\ShareErrorsFromSession->handle(Object(BookStack\\Http\\Request), Object(Closure)) #25 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(121): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #26 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(64): Illuminate\\Session\\Middleware\\StartSession->handleStatefulRequest(Object(BookStack\\Http\\Request), Object(Illuminate\\Session\\Store), Object(Closure)) #27 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Session\\Middleware\\StartSession->handle(Object(BookStack\\Http\\Request), Object(Closure)) #28 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php(37): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #29 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Cookie\\Middleware\\AddQueuedCookiesToResponse->handle(Object(BookStack\\Http\\Request), Object(Closure)) #30 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(67): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #31 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Cookie\\Middleware\\EncryptCookies->handle(Object(BookStack\\Http\\Request), Object(Closure)) #32 /var/www/bookstack/app/Http/Middleware/ApplyCspRules.php(33): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #33 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): BookStack\\Http\\Middleware\\ApplyCspRules->handle(Object(BookStack\\Http\\Request), Object(Closure)) #34 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(116): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #35 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Routing/Router.php(797): Illuminate\\Pipeline\\Pipeline->then(Object(Closure)) #36 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Routing/Router.php(776): Illuminate\\Routing\\Router->runRouteWithinStack(Object(Illuminate\\Routing\\Route), Object(BookStack\\Http\\Request)) #37 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Routing/Router.php(740): Illuminate\\Routing\\Router->runRoute(Object(BookStack\\Http\\Request), Object(Illuminate\\Routing\\Route)) #38 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Routing/Router.php(729): Illuminate\\Routing\\Router->dispatchToRoute(Object(BookStack\\Http\\Request)) #39 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(190): Illuminate\\Routing\\Router->dispatch(Object(BookStack\\Http\\Request)) #40 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(141): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}(Object(BookStack\\Http\\Request)) #41 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Http/Middleware/TrustProxies.php(39): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #42 /var/www/bookstack/app/Http/Middleware/TrustProxies.php(41): Illuminate\\Http\\Middleware\\TrustProxies->handle(Object(BookStack\\Http\\Request), Object(Closure)) #43 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): BookStack\\Http\\Middleware\\TrustProxies->handle(Object(BookStack\\Http\\Request), Object(Closure)) #44 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #45 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(40): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle(Object(BookStack\\Http\\Request), Object(Closure)) #46 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Foundation\\Http\\Middleware\\TrimStrings->handle(Object(BookStack\\Http\\Request), Object(Closure)) #47 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #48 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle(Object(BookStack\\Http\\Request), Object(Closure)) #49 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(86): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #50 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle(Object(BookStack\\Http\\Request), Object(Closure)) #51 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(116): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(BookStack\\Http\\Request)) #52 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(165): Illuminate\\Pipeline\\Pipeline->then(Object(Closure)) #53 /var/www/bookstack/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(134): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter(Object(BookStack\\Http\\Request)) #54 /var/www/bookstack/public/index.php(52): Illuminate\\Foundation\\Http\\Kernel->handle(Object(BookStack\\Http\\Request)) #55 {main} "} [2023-08-03 09:19:49] production.ERROR: Error when attempting image upload:Unable to set visibility for file uploads/images/gallery/2023-08/2VZlogo.png. ```

[yoghiyb]

@ssddanbrown any update for this?

[ssddanbrown]

@yoghiyb No. I'd need to test and attempt to replicate on S3. Not sure when I'll get round to that.

[oxismailxo]

Guys, i have the same issue

@yoghiyb please did you find any solution?

Thanks.

[ssddanbrown]

I've just tested AWS S3 usage on my dev instance (BookStack ~v23.08). All appears to work as expected.

I get that there are a lot of options and variation within S3 that could cause different behaviour though. I'd imagine the "Block public access" setting has to be off since we set public ACLs on images upon upload, so that's maybe something to check.

[oxismailxo]

@ssddanbrown thank you for your intervention & time, here is my S3 config:

Block public access=OFF

1- Permissions -> Bucket policy:

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": "", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::bookstack-wiki-test/uploads/" } ] } ///////////////////////////////////////////////////////////////////////////////// 2- IAM user Permissions policies: { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:ListBucket", "Resource": [ "arn:aws:s3:::bookstack-wiki-test/", "arn:aws:s3:::bookstack-wiki-test" ] }, { "Effect": "Allow", "Action": [ "s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:GetObjectAcl", "s3:DeleteObject" ], "Resource": "arn:aws:s3:::bookstack-wiki-test/" } ] }

NOTE-1: images are uploaded successfully to S3 but the error shows up:

NOTE-2: When I upload attachments the error doesn't show up!

I hope this information helps.

Thank you again @ssddanbrown !

[deanoj]

The upload is failing here: https://github.com/BookStackApp/BookStack/blob/development/app/Uploads/ImageService.php#L220

If you comment out that line - upload works as expected.

[robertjustjones]

Try this hack to avoid that line.

STORAGE_S3_ENDPOINT=https://s3.amazonaws.com

[oxismailxo]

@robertjustjones adding STORAGE_S3_ENDPOINT=https://s3.amazonaws.com not worked for me !

the error when uploading images:

[Joespink]

@oxismailxo Need to add the region aswell - "https://s3.eu-west-2.amazonaws.com" works for me

[ssddanbrown]

Yeah, STORAGE_S3_ENDPOINT will be region dependent, if setting manually for AWS.

From my view, this should work though. Happy to hear from anyone more experienced with AWS what might be required here, and why this appears to work for me but not others (Specifically this is the action of setting ACLs to make images public), especially as from the above it has been shown to be failing for those with PutObjectAcl permissions.

[oxismailxo]

Thank you so much @Joespink ! finally worked for me after adding STORAGE_S3_ENDPOINT=https://s3.eu-west-1.amazonaws.com/,

thank you, everyone!

[nathanclayton]

The upload is failing here: https://github.com/BookStackApp/BookStack/blob/development/app/Uploads/ImageService.php#L220

If you comment out that line - upload works as expected.

That makes sense, since AWS defaults all S3 buckets to private by default now, and if you have a policy in place to disable public sharing (instead sharing the data through a cloudfront endpoint), it could cause it to error out.

[nurradityam]

private by default now, and if you have a policy in place to disable public sharing (instead sharing the data through a cloudfront endpoint),

i can confirm digital ocean spaces (s3 compatible) is also set permission to private as default, any easy workaround for this?

somehow i can connect by using this url structure:

STORAGE_S3_ENDPOINT=https://(region).digitaloceanspaces.com STORAGE_URL=https://(bucket).(region).digitaloceanspaces.com

[jasonpincin]

private by default now, and if you have a policy in place to disable public sharing (instead sharing the data through a cloudfront endpoint),

i can confirm digital ocean spaces (s3 compatible) is also set permission to private as default, any easy workaround for this?

somehow i can connect by using this url structure:

STORAGE_S3_ENDPOINT=https://(region).digitaloceanspaces.com

STORAGE_URL=https://(bucket).(region).digitaloceanspaces.com

Ever figure out how to work around this on digital ocean?

[ssddanbrown]

@nurradityam @jasonpincin Worth having a read of my comment here for DigitalOcean: https://github.com/BookStackApp/BookStack/issues/4556#issuecomment-1723231342