Ability to specify endpoint-url in profile

[StFS]

Currently I don't seem to be able to specify an endpoint URL in my profile. I always have to specify the --endpoint-url option on the command line.

I would like to be able to do something like the following (in my ~/.aws/config file):

[default]
region = us-east-1
output = json

[profile nextcode]
region = myregion-1a
output = json
endpoint-url = http://c.my.other.aws.compatable.service.com

[kyleknap]

@StFS If you do not mind me asking, what is the specific use case do you need it for (like which service or services would you plan to use it for)? The thing with --endpoint-url is that the value set will most likely not apply across differing services. So if you configure a global endpoint url, it may work for one service, but you will run into errors if you try to make requests to other services.

I could see having a per-service configuration for endpoint-url, but I am having trouble imagining a global endpoint-url across all AWS services.

[StFS]

Thanks for your reply.

Basically, we use AWS for some things but we also have our own EC2/S3 compatable private cloud setup (we're using https://qstack.com/).

Maybe I'm misunderstanding something with this. I know that the endpoints seem to differ between whether you're "talking" S3 or EC2. Basically I think I want to be able to point one of my profiles to a different "set" of endpoints. The aws cli tool works fine for our AWS account, but when I want to use it for our private cloud setup I always have to specify both --profile (to get the credentials right) and --endpoint-url (so that aws contacts our private cloud endpoint instead of the AWS ones).

[StFS]

@kyleknap Am I totally out there with this?

[kyleknap]

@StFS Your use case makes sense to me. I am not sure how many users have a similar use case. I think if we were to add it as a feature we would be more inclined to do it per service, instead of setting a global value for endpoint url. The good news is that if we were to add this feature you would be able to configure the endpoint url for each service per profile (so you can still just use --profile and not have to include --endpoint-url).

Marking as feature request.

[StFS]

Ok thanks for the response again.

I'm a bit confused about these endpoints. It seems that for AWS the endpoint may change depending on what region you're using and also what service you're using.

For us, we would only need to specify ec2_endpoint and s3_endpoint, is that what you mean by "per service"? Are there other services than EC2 and S3? I think I've seen something involving user management as well but I'm not quite sure.

[jerrycattell]

We have a similar use case (S3 alternative in Canada), and we would love to be able to configure the endpoint-url in the config file. Even if that means having a separate profile per service (currently it is only used for object storage), it would allow us to use the same commands in all of our environments.

[kennu]

I think this would also be useful for configuring a "local" profile for accessing DynamoDB Local. Currently you have to write something like

aws --profile local --endpoint-url http://localhost:8000 dynamodb list-tables

[viniciussbs]

Same case here, @kennu. I have to set endpoint-url everytime I need to run some command on dynamodb-local.

I would love to set http://localhost:8000 as my default endpoint-url for dynamodb and set a blank endpoint-url to an production profile. It's more error-proof.

[mathurasri]

How to set dyanmodb local endpoint-url ? I am using ubuntu OS.

[shravanpn7]

+1 for feature-request.

I am currently using aws cli to access minio.io server and specifying --endpoint-url every time is a pain.

[slejworks]

I need this also, connecting to local dnyamodb and I don't want to specify the endpoint for each cmd, simple entry into the config will work for me so I can concentrate on the command. thanks.

[christians3]

+1 for this feature of allowing endpoint-url in the configuration file.

[philloooo]

if accepting endpoint-url on the top level doesn't make sense, what about accepting it on service level? eg:

[profile development]
aws_access_key_id=foo
aws_secret_access_key=bar
s3 =
  endpoint-url = test.org

[jhalterman]

Similar use case, pointing to a Eucalyptus cloud. Would be nice to have a per-service config for endpoint-url.

[lincolnthomas]

How about also for the convenience of not having to type the --endpoint on the cmdline?

[wrsuarez]

+1

[jmarshall]

We have a third-party library that (as a minor aspect of its remote data access facilities) itself parses ~/.aws/credentials in order to construct the HTTP headers to access data in S3 buckets. Similarly to this issue, our users want to access their own S3-compatible data stores (cf samtools/htslib#436) and it would be useful if there were a standard well-known configuration file setting name for this purpose.

[jedi4ever]

👍

[tomviner]

We also have this use case, except our S3-like-service has a different authorisation scheme (a simple authorization: <token> header). How are people using aws-cli to authorise against their S3-like-services?

[wbingli]

Check out this awscli plugin to set endpoint on profile: https://github.com/wbingli/awscli-plugin-endpoint

Once you install(pip install awscli-plugin-endpoint) and config plugin(aws configure set plugins.endpoint awscli_plugin_endpoint), you can set endpoint in your profile per service as following:

[profile local]
dynamodb =
    endpoint_url = http://localhost:8000

Now you can run command with this endpoint for this service with only profile name:

aws dynamodb list-tables --profile local

See more details on project homepage(https://github.com/wbingli/awscli-plugin-endpoint).

Let me know your feedback, :)

[RichardBronosky]

Just to be clear, this is not only a benefit for people who are running AWS-compatible competitive products. It would also benefit those of us who are trying to use Amazon tools for local/offline development. http://stackoverflow.com/a/32260680/117471 is an example with DynamoDB. Using different profiles allows us to run our code with different configs for different environments. Not allowing endpoint_url to be specified in ~/.aws/config means that we have to build logic into our apps to follow a different (although small) path depending on the environment. That is something that should be avoided.

[kennu]

One such toolset is https://github.com/atlassian/localstack which provides local mocks for API Gateway, Kinesis, DynamoDB, DynamoDB Streams, Elasticsearch, S3, Firehose, Lambda, SNS, SQS and Redshift. It would be highly useful to create an AWS profile for deploying e.g. Serverless services to this mock platform and use this profile like any profile in deployment scripts.

[dilzfiesta]

Hello,

I am catering to multiple customers and each customer provided me a different endpoint url to access their accounts. Example below - https://customer-1.signin.aws.amazon.com/console https://customer-2.signin.aws.amazon.com/console

If I try to connect using aws cli - > aws ec2 describe-instances --profile customer1

I am receiving below error - Could not connect to the endpoint URL: "https://ec2.us-east-1a.amazonaws.com/"

Is it possible to add something like this? [profile customer1] region = us-east-1 output = json account = customer1

[profile customer2 region = us-east-1a output = json account = customer2

[elsonrodriguez]

Something like AWS_ENDPOINT_URL would also be fantastic.

[Llammissar]

I was very surprised to find this wasn't already a feature just as a matter of course. Is it any different from being able to specify the region in your config or as an option? I'd expect it to be a fairly trivial change in the same general code path.

[yyolk]

I would also love what @elsonrodriguez mentions

Another use case not mentioned and comes to mind for me is setting up an [elasticmq] profile for local sqs dev with elasticmq "local dev" ... no internet connection ;)

[fatihtekin]

you can do below work around

from boto3 import Session
Session.client.defaults = (None, None, False, None, 'http://localhost:4575', None, None, None, None)

[aldanor]

@fatihtekin The whole question here is how to not do this but rather have it picked up from config.

[fatihtekin]

@aldanor i totally agree that is why it is a workaround if someone needs for testing purposes especially without changing the lambda implementation

[jhart-r7]

+1 for this feature. Given the rise of non-AWS providers providing AWS-like functionality with compatible APIs, having this functionality would be very useful.

[tfili]

+1

[ASayre]

Good Morning!

We're closing this issue here on GitHub, as part of our migration to UserVoice for feature requests involving the AWS CLI.

This will let us get the most important features to you, by making it easier to search for and show support for the features you care the most about, without diluting the conversation with bug reports.

As a quick UserVoice primer (if not already familiar): after an idea is posted, people can vote on the ideas, and the product team will be responding directly to the most popular suggestions.

We’ve imported existing feature requests from GitHub - Search for this issue there!

And don't worry, this issue will still exist on GitHub for posterity's sake. As it’s a text-only import of the original post into UserVoice, we’ll still be keeping in mind the comments and discussion that already exist here on the GitHub issue.

GitHub will remain the channel for reporting bugs.

Once again, this issue can now be found by searching for the title on: https://aws.uservoice.com/forums/598381-aws-command-line-interface

-The AWS SDKs & Tools Team

This entry can specifically be found on UserVoice at : https://aws.uservoice.com/forums/598381-aws-command-line-interface/suggestions/33168307-ability-to-specify-endpoint-url-in-profile

[ankon]

For future googlers: https://aws.uservoice.com/forums/598381-aws-command-line-interface/suggestions/33168307-ability-to-specify-endpoint-url-in-profile

[lpriima]

+1

[jamesls]

Based on community feedback, we have decided to return feature requests to GitHub issues.

[abodnar]

This would be incredibly useful for those that are interacting with GovCloud.

[roccogalluzzo]

+1

[sparrowjack63]

+1

[mojodna]

This would also be helpful when working with Snowball Edges.

[chaijunkun]

Yes, It is awesome that so many people expect this feature. I am using third party s3 provider, and the quality of command line tools which is developed by third party is not so good. we are looking forward amazon official version that satisfies our requirement.

well, the other way to implement this feature indirectly is: set an alias for your general command. such as: modify your ~/.bash_profile add following line: alias awsoss='aws s3 --endpoint-url=http://127.0.0.1:9000' then you can use awsoss to replace aws command for accessing s3.

if you have multiple endpoints, you can also use: alias awsoss='aws s3 --endpoint-url=$OSS_ENDPOINT' alias awsossapi='aws s3api --endpoint-url=$OSS_ENDPOINT'

use: export OSS_ENDPOINT=xxx to switch different endpoints then, you can use awsoss and awsossapi to instead of "aws s3" and "aws s3api"

[dan-bhakta]

+1 for this feature

[cs-andros]

+1 from me as well. We'd really like to leverage per-service endpoint URLs for development and testing.

[onraz]

+1 This is needed for local development/testing and Dockerized environments.

[AlbertoPeon]

+1 for this

[brandonsimpkins]

+1 for this feature.

This is will be a useful feature for interacting with EC2 instances on a Snowball Edge (which act like a local region).

To list the EC2 instances on a snowball I have to specify the endpoint even though I already have a AWS CLI Named Profile that has the rest of the Snowball Edge specific info (secret / access keys):

aws ec2 describe-instances --profile snowballEdge --endpoint http://${snowball_ip}:8008

[eedwards-sk]

+1 to this and environment variable for the same (e.g. AWS_ENDPOINT or AWS_S3_ENDPOINT)

[paul-ang]

+1 for this feature

[dora-korpar]

+1

[YingmingHu]

+1

[husnusensoy]

+1