matrix-org / synapse-s3-storage-provider

Synapse storage provider to fetch and store media in Amazon S3
Apache License 2.0
118 stars 33 forks source link

Uploads result in CANCELED #97

Closed Cyberes closed 1 year ago

Cyberes commented 1 year ago

I've been working to determine why large file uploads fail in Element. I'm not going to re-post everything, but here's the Synapse debugging. https://github.com/matrix-org/synapse/issues/15841

I am attempting to upload two video files (each about 30 MB) to a room at the same time. This always fails with a generic error message popup (The file '[filename]' failed to upload.). There is no error in the console and the network of the debug window shows the fetch request as (canceled). Synapse logs don't show any errors and neither does Openresty. Uploading the files one at a time works, but uploading them together always fails.

media_store_path:                                 "/mnt/matrix-storage/media"

media_storage_providers:
- module:                                         s3_storage_provider.S3StorageProviderBackend
  store_local:                                    True
  store_remote:                                   True
  store_synchronous:                              True
  config:
      bucket:                                     matrix-synapse
      endpoint_url:                               'https://xxx.r2.cloudflarestorage.com/matrix-synapse'
      access_key_id:                              'xxx'
      secret_access_key:                          'xxx'

When I comment out the media_storage_providers block, uploads are able to complete. I tried setting store_synchronous to False. I'm also using a media worker and disabling that didn't change anything.

The Synapse server's network doesn't have the fastest uplink and I'm suspicious that the upload to the S3 bucket is taking too long and the client gives up. Is there a way to have Synapse do the upload to S3 in the background?

Cyberes commented 1 year ago

Actually it looks like it isn't related to the S3 storage provider because uploads are randomly failing.