--aws-profile CLI param is not supported

[david-a-jetter]

Is there an existing issue for this?

• [X] I have searched existing issues, it hasn't been reported yet

Use case description

With a standard serverless deploy command before Serverless Compose, I could pass a named AWS profile to use for the deployment. Currently, I'm using this feature to switch between profiles for different accounts to deploy different stages to different AWS subaccounts.

When running this today, with compose v 1.0.2, the following error is returned:

npx serverless deploy --stage dev --aws-profile default
Environment: win32, node 16.13.2, compose 1.0.2
Docs:        slss.io/docs-compose
Bugs:        github.com/serverless/compose/issues

Error:
Unrecognized CLI options: "--aws-profile"

This could probably be worked around by using the "Per Stages Profile" feature in the serverless.yml file discussed here https://www.serverless.com/framework/docs/providers/aws/guide/credentials. But, in general, it would be nice if there was the ability to "pass through" any non-compose option like --aws-profile.

[mnapoli]

Thanks for opening this.

Just noting 2 workarounds:

1. as you mentioned, setting a profile per stage in serverless.yml (using stage parameters might be even better here)
2. or setting the AWS_PROFILE environment variable (AWS_PROFILE=prod sls deploy)

To be transparent, we're hesitant to add an AWS-specific CLI option to global commands (commands that apply to all services) because there could be services that are not on AWS. Let's keep this discussion open though.

[david-a-jetter]

@mnapoli I appreciate the reply, and certainly understand the desire to keep something like compose agnostic of service implementation details.

I do think that the original serverless CLI offered a wide range of features, some of which were implementation-specific. I'm not quite sure of the architecture of compose, and if it would be conceivable to add some sort of generalized pass-through parameter that would effectively allow compose to ignore those parameters but let original serverless deploy handle them (or fail on them, if need be).

[mnapoli]

To be sure I understand (and maybe so that we can imagine even better solutions), in your case using AWS_PROFILE would be harder/less practical?

[david-a-jetter]

@mnapoli I think so, that using the environment variable is less practical, but I also think that may be because I am operating in a less than best-practice way.

I'm currently developing on a solution with multiple serverless applications sharing a codebase a la "monorepo" style (this isn't the poor practice part) as the sole contributor for the time being. Because I'm the sole contributor, I haven't bothered to set up any sort of a agent-based deployment pipeline, and I handle all of the deployments from my local machine. Setting environment variables typically requires closing/reopening the shell I am in, or occasionally PyCharm if I've running with the integrated terminal. It's a lot faster and more explicit to pass some command line arguments.

I think if it were configured as a real agent-based deployment job, setting the profile by environment variable would be totally reasonable, or may not even need to be done at all.

[mkarsene]

I have 3 AWS profiles setup on my system. Howver, I am having issues with using any of these profiles except the default one. None of the options above work for me.

• explicity stating the profile name under provider.profile is not working
• using the per stage profile setup is also not working
• setting the AWS_PROFILE environment variable Each time, it always uses the default profile, regardless of what I put in...

[mnapoli]

@mkarsene it seems like an actual bug, I have opened a separate issue here: #120 Let's discuss your problem there.

@david-a-jetter I'm wondering if this wouldn't be solvable via stage parameters in serverless.yml:

provider:
  name: aws
  profile: ${stage:profile}

params:
  default:
    profile: default
  prod:
    profile: the-prod-profile

[david-a-jetter]

@mnapoli Apologies for how long it took me to get back to you on this.

Stage params in the yml files could be an option, although somewhat obtuse for my particular case. I'm orchestrating a variety of different operations through a Makefile and that includes direct aws cli calls, which also include the named profile passed in the make call. The use case here is probably solved by serverless-compose, since the goal was to have any shared resources (eg. an EventBridge event bus, a DynamoDB table) be managed directly through a raw CloudFormation template as the "dependency" of the subsequent Serverless Services. Having the yml files reference some other value (eg. maybe an environment variable for the profile name?) allows these two values to diverge, which may cause issues if the wrong CLI command is issued. I suppose everything could reference that environment variable, but then you lose the niceness of being able to change arguments through a CLI.

I tried the --param CLI argument to sls deploy but that failed with: The "--param" option is not supported (yet) in Serverless Framework Compose. Looks like this is covered in #111

Also, I an using @apalchys/serverless-openapi-documentation to assist with generating OpenAPI docs for the two services that expose HTTP endpoints to other consumers. Unfortunately, it seems like the existence of the serverless-compose.yml file interferes with calls to that plugin via serverless openapi generate since I get compose-related errors from those where I am specifying the config and output files: The "--config" option is not supported (yet) in Serverless Framework Compose. This one seems to violate the principle of least surprise - I wouldn't expect serverless compose to intervene on specialized CLI plugin commands like this, but it might be as simple as an order of operations bug where the CLI commands are being validated by compose because of the existence of serverless-compose.yml before being delegated to the plugin.

[david-a-jetter]

Another update here: After doing some work to begin doing packaging and deployments in Github actions, I realized that these CLI arguments make it difficult to be flexible with how the credentials are supplied in various environments. Notably, using https://github.com/aws-actions/configure-aws-credentials doesn't make it easy to set a named profile, so the various commands quickly fail.

I decided to manage setting the appropriate profile as a separate configuration - eg. manually manipulate my environment variables to set my profile - rather than trying to handle this in the CLI args.

I still have a separate blocking issue related to Python requirements packaging that I am going to open up separately, but I think I've solved the issues here by simply rethinking how I was managing my AWS credentials configurations.

Edit: I've also sorted out the problem I had with the openapi plugin by running the doc generation commands as: sls {service_name}:openapi generate 👍

[fernandohonig]

Hi, I am trying to run serverless compose with a named profile but always returns a 'AWS profile "oidc" doesn't seem to be configured'

Running from a Gitlab CI job the script lines are:

- | 
  mkdir -p ~/.aws
  echo "${OIDC_TOKEN}" > /tmp/web_identity_token
  echo -e "[profile oidc]\nrole_arn=arn:aws:iam::${account_id}:role/Example-OIDC\nweb_identity_token_file=/tmp/web_identity_token" > ~/.aws/config
- serverless shared-services:deploy --stage dev

I am running out of ideas. Any help it would be appreciated. Thanks!

[multigl]

@fernandohonig it does not appear that serverless can properly read named profiles, and can only accept default -- I had this same issue this week and the only solution was to change (using your code, my ARN and token files are in different locations):

  echo -e "[profile oidc]\nrole_arn=arn:aws:iam::${account_id}:role/Example-OIDC\nweb_identity_token_file=/tmp/web_identity_token" > ~/.aws/config

to

 echo -e "[default]\nrole_arn=arn:aws:iam::${account_id}:role/Example-OIDC\nweb_identity_token_file=/tmp/web_identity_token" > ~/.aws/config

I only do this in pipelines that have serverless because it's incredibly poor form when everything else supports named profiles ¯_(ツ)_/¯