AWS SDK prepends endpoint URL with `<bucket>.`

[WillNilges]

Hello!

Thanks for making this plugin :) I was having trouble getting it configured with our self-hosted S3 instance. I saw your Readme section about the custom s3 domain, and non-standard configuration, but I cannot seem to get it working with our setup.

when I load a page, I get an error that looks like this

Warning: AmazonS3FileBackend::statUncached: S3Exception: Error executing "HeadObject" on "https://wiki.s3.csh.rit.edu/wiki/thumb/6/60/Willard_baldy2.png/350px-Willard_baldy2.png"; AWS HTTP error: cURL error 6: Could not resolve host: wiki.s3.csh.rit.edu (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://wiki.s3.csh.rit.edu/wiki/thumb/6/60/Willard_baldy2.png/350px-Willard_baldy2.png in /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php on line 1122

I noticed that the URL is wrong. It says

https://wiki.s3.csh.rit.edu/wiki/thumb/6/60/Willard_baldy2.png/350px-Willard_baldy2.png

but it should say

https://s3.csh.rit.edu/wiki/thumb/6/60/Willard_baldy2.png/350px-Willard_baldy2.png

Notice that there's no "wiki" at the beginning of the URL? The first one does not resolve, the second one does. That's intentional.

The bucket name is wiki, but that gets added onto the end right after the TLD instead of before. Not really sure why, I just work here ;) You'll notice that I'm using $wgAWSBucketTopSubdirectory = '/wiki'; to make that happen. Probably not intended use, but that was the closest I could seem to get our url.

Here's the relevant section of our config

# === S3 ===
# speaking of images, have some s3
wfLoadExtension( 'AWS' );

# Configure AWS credentials.
# THIS IS NOT NEEDED if your EC2 instance has an IAM instance profile.
$wgAWSCredentials = [
|   'key' => '-snip-',
|   'secret' => '-snip-',
|   'token' => false
];

$wgAWSRegion = 'us-east-1'; # Northern Virginia

# Replace <something> with the name of your S3 bucket, e.g. wonderfulbali234.
$wgAWSBucketName = "wiki";

#The url used for the API (PutObject, etc.)
$wgFileBackends['s3']['endpoint'] = 'https://s3.csh.rit.edu';
#The url used for showing images. $1 is translated to the bucket name.
$wgAWSBucketDomain = 's3.csh.rit.edu';

# if your images are stored in directory called "some_prefix"
# you can specify an optional prefix
$wgAWSBucketTopSubdirectory = '/wiki';
$wgAWSRepoHashLevels = '2'; # Default 0
# === End S3 ===

Am I doing something wrong? It seems to me like specifying the wgAWSBucketDomain like I did should be correct.

Another thing I noticed, possibly related, while messing with this, is that if I remove the wgAWSBucketName and try it, the error goes away, but none of the images load, the thumbnails say Error creating thumbnail: Unable to save thumbnail to destination, and they all redirect to the homepage. I'm sure that's intentional, but I was hoping that might "trick" the plugin into letting me use the bucket. No such luck though.

[edwardspec]

Please try the following:

$wgFileBackends['s3']['use_path_style_endpoint'] = true; // And don't use $wgAWSBucketTopSubdirectory

If it doesn't help, there is nothing else that can be done on MediaWiki side, because this "wiki." prefix is added by AWS SDK (not by this extension). This extension merely calls $client->headObject( [ 'Bucket' => $bucket, 'Key' => $key ] );

There are, of course, external solutions like making wiki.<domain> a CNAME to <domain> (so that the "wrong" URL would work too).

[WillNilges]

That didn't seem to have any effect, unfortunately. So then, it's an issue with the SDK? I wonder if there's a way to patch it...

I'll try the CNAME idea, I actually hadn't considered that before.

[WillNilges]

The CNAME produces a predictable error:

Warning: AmazonS3FileBackend::statUncached: S3Exception: Error executing "HeadObject" on "https://wiki.s3.csh.rit.edu/wiki/thumb/6/60/Willard_baldy2.png/2000px-Willard_baldy2.png"; AWS HTTP error: cURL error 60: SSL: no alternative certificate subject name matches target host name 'wiki.s3.csh.rit.edu' (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://wiki.s3.csh.rit.edu/wiki/thumb/6/60/Willard_baldy2.png/2000px-Willard_baldy2.png in /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php on line 1122

[WillNilges]

I found a few relevant issues in the AWS SDK: https://github.com/aws/aws-sdk-php/issues/2615 https://github.com/aws/aws-sdk-php/issues/2588

What version of the SDK are you using?

[WillNilges]

Oh, composer.json says 3.67... isn't that like... really old? I have literally no idea how their versioning scheme works, but they're on like 3.277 now...?

[WillNilges]

Forked and update the library version, and now I'm getting 403 forbiddens, which is progress. the URLs seem correct. Maybe my credentials are wrong or it's a permissions error?

[WillNilges]

Oop, figured it out. I had to remove all the specification for temp directory.

[edwardspec]

Since it works for you and this is a non-standard configuration, closing.

Required version of AWS SDK will eventually be increased, but unless there is a serious reason to do so (such as security vulnerabilities), it will be delayed until we drop support for some older versions of MediaWiki. Both MediaWiki and AWS SDK require Guzzle, and they might want different versions of it.