search

matrix-org / synapse-s3-storage-provider

Synapse storage provider to fetch and store media in Amazon S3
Apache License 2.0
118 stars 33 forks source link

s3_media_upload non-AWS #34

Closed HarHarLinks closed 4 years ago

HarHarLinks commented 4 years ago

The s3_media_upload script suffers from a handful shortcomings that stem from - in contrast to the s3_storage_provider.py module - not inheriting the homeserver config.

Especially the trouble is that the default boto3 client instantiation somehow gets the region (from .aws/config default s3 maybe?) but not the complete URL/domain part - i.e. if you use something other that AWS, synapse will respect that but the upload script won't.

My suggestion is to instead of creating a custom database.yaml config file, to just read the homeserver.yaml instead (or worse, copy all relevant fields into it's own config). Then parse everything relevant, e.g. endpoint_url to this:

https://github.com/matrix-org/synapse-s3-storage-provider/blob/a38b15b2a8588d734ebd43b9471aa9d6278feeb4/scripts/s3_media_upload#L435

richvdh commented 4 years ago

Especially the trouble is that the default boto3 client instantiation somehow gets the region (from .aws/config default s3 maybe?) but not the complete URL/domain part - i.e. if you use something other that AWS, synapse will respect that but the upload script won't.

this sounds like a duplicate of #25.