Support environment variable for S3 addressing_style config

[skeggse]

Describe the feature

Allow configuring boto3 to use the virtual host style (MY-BUCKET.s3.amazonaws.com) for talking to a bucket by providing the AWS_S3_ADDRESSING_STYLE environment variable.

Use Case

If you want to force boto3 to include the region in the hostname used for boto3.client("s3", region_name="us-east-1"), you can set the AWS_S3_US_EAST_1_REGIONAL_ENDPOINT=regional in the environment variables.

If you want to force boto3 to use the virtual host style (MY-BUCKET.s3.amazonaws.com) for talking to a bucket, you have to configure it either in application code or in an accessible shared config file. Sometimes, these two options are prohibitively difficult to configure correctly, and configuring this behavior via an environment variable may be significantly easier.

Proposed Solution

Add AWS_S3_ADDRESSING_STYLE to configprovider.py's DEFAULT_S3_CONFIG_VARS' addressing_style entry.

Other Information

No response

Acknowledgements

• [X] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

SDK version used

botocore==1.34.90

Environment details (OS name and version, etc.)

macOS 14.4.1 (23E224)

[tim-finnigan]

Thanks for the feature request, but I think we need more information here. When I run the following:

import boto3
client = boto3.client('s3')
boto3.set_stream_logger('')
client.list_objects(Bucket='us-east-1-bucket')

Then the debug logs show this endpoint, which is the virtual style and contains the region:

Endpoint provider result: https://us-east-1-bucket.s3.us-east-1.amazonaws.com

And if I run this:

import boto3
from botocore.config import Config
boto3.set_stream_logger('')
client_config = Config(s3={"addressing_style": "path"})
client = boto3.client('s3', config=client_config)
client.list_objects(Bucket='us-east-1-bucket')

Then the debug logs show this:

Endpoint provider result: https://s3.us-east-1.amazonaws.com/us-east-1-bucket

So the default behavior is consistent with S3 virtual hosting format: https://bucket-name.s3.region-code.amazonaws.com/key-name

Based on that, I think it's already doing what you're looking for, right? Unless I'm missing something involving your region/version or use case. And there is a (delayed) plan for S3 to deprecate path style requests. So I don't think an environment variable would be considered here.

[skeggse]

Ah, shoot. I looked closer, and it looks like the behavior I observred was the result of using a bucket name that doesn't work as a hostname. I had assumed I was configuring something wrong.

As a side note, will bucket names with underscores stop working at some point? Obviously, we'd rather migrate, but... there are lots of priorities.

[github-actions[bot]]

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.