search

Forceu / Gokapi

Lightweight selfhosted Firefox Send alternative without public upload. AWS S3 supported.
GNU Affero General Public License v3.0
1.21k stars 50 forks source link

Timeout when uploading large files to cloud storage #81

Closed reinhart1010 closed 7 months ago

reinhart1010 commented 2 years ago

Hi, I decided to configure Gokapi to use Cloudflare R2, with auto set as the S3 region. When the file is successfully uploaded to the Gokapi server, the server returns an error (504 Gateway Timeout as I used Nginx).

Looking at the logs, I see the following warning, in which I have configured my server with the correct access key and secret access key. Requesting a new one from Cloudflare doesn't fix the issue as well:

caused by: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.

reinhart1010 commented 2 years ago

This issue also persists by using us-east-1 instead of auto.

Forceu commented 2 years ago

I just tried setting up Gokapi with R2 and could not reproduce the problem. Are you using the correct endpoint (https://<ACCOUNT_ID>.r2.cloudflarestorage.com)?

reinhart1010 commented 2 years ago

Apparently I inputted the bucket URL as <ACCOUNT_ID>.r2.cloudflarestorage.com) (without the https://).

reinhart1010 commented 2 years ago

Edit: The issue still persists even when using "https://", and apparently only occur on large files (e.g. a ~80MB APK file). Small files such as PNG images can still be uploaded successfully.

Forceu commented 2 years ago

Thanks for the feedback! I assume the webserver times out after trying to upload a large file to the cloudserver, which is a blocking call. I might have some time to fix it tomorrow, otherwise it might take a couple of days.

Forceu commented 2 years ago

This issue has been fixed in 510ee9d493f08dfc60379040cd67144ad29df14e. If you are using docker, the image latest-dev contains the fix.

I will probably implement websockets at some point, so that a high timeout is not needed anymore.

reinhart1010 commented 1 year ago

I'm currently running the latest stable version for production, but might check out the latest Docker dev version soon

hra42 commented 10 months ago

I run the current latest-dev with Cloudflare R2 storage. When I upload small files <1mb, the upload is fine and everything works. If the uploaded file is >1mb the upload seems to be stuck. After a few minutes the upload is complete and I can share the URL. I tried with a go windows binary as upload and an image. The image seems to be stuck far longer (if it ever completes)

running on arm64

Forceu commented 10 months ago

Thanks for the feedback. Are there any error messages? And I assume you are using the latest version?

hra42 commented 10 months ago

Thanks for the feedback. Are there any error messages? And I assume you are using the latest version?

When I close the session I get EOF, but thats to be expected, otherwise no erros. Yes I'm using latest-dev from dockerhub on Raspberry Pi 5 on the latest docker version. Arm64 image.

hra42 commented 10 months ago

now its even stranger, the issue seems to be gone for files <2mb with the latest-dev version. Just repulled the image and the issue is gone now.

This is might be caused by slow upload speeds from ISP, I would guess.

Forceu commented 7 months ago

This bug appears to be fixed. If you still experience problems with the latest version, please post here and I will reopen the issue.