Open afgallo opened 2 years ago
Hello, @afgallo!
Can you tell me a bit more? I have tried to replicate this error with your param name, but didn't get the same error. Did secret init
work for you? Did you put the secret name into your Load Balanced Web Service manifest? Did you get the error when you ran svc deploy
?
Thanks!
Hi @huanjani yes, I basically ran secret init
and followed the prompts as per below:
Then I added the following to my manifest file which is a load balanced service:
I got the error when I ran svc deploy
I wonder if that was an issue with the AWS api?
I'm still unable to replicate this perplexing error π€ ....
I'll keep looking into it, but as a workaround to get you going, can you manually create JWT_KEY
through the SSM Parameter Store, being sure to tag it with copilot-application
and copilot-environment
? (https://aws.github.io/copilot-cli/docs/developing/secrets/)
Okay, I'm finally getting that error! Stay tuned...
Thanks for not giving up on this, @huanjani! I was getting concerned you would not be able to reproduce it as it somehow worked for me today π€
Oh, really! That's good to know! At least you're unblocked. I was going to suggest deleting and trying again, but that wasn't a solution that got to the bottom of this. π€£
The way I got it was atypical, though, and now it's working! π€
π I'm also experiencing this issue when setting an IP address as a secret with copilot secret init
. Important to note, that this was working previously, but I needed to add a domain and HTTPS endpoint for our app and so after destroying and recreating the app and attempting to deploy a service I receive:
Resource handler returned message: "Invalid request provided: Create T
askDefinition: The Systems Manager parameter name specified for secret
MY_SECRET_HERE is invalid. The parameter name can be up to 2048 character
s and include the following letters and symbols: a-zA-Z0-9_.-, (Servic
e: AmazonECS; Status Code: 400; Error Code: ClientException; Request I
D: 967b0aac-3f81-4910-8e05-52601c8897eb; Proxy: null)" (RequestToken:
5d80ebd9-b11a-02b2-27da-43040b7fc2a9, HandlerErrorCode: InvalidRequest
)
I have also tried deleting and manually recreating the secret in SSM parameter store, but no luck.
Sidenote: I just updated to 1.19 today from 1.18 and am running a "Load Balanced Web Service"
UPDATE:
In a bizarre twist of events, I renamed my secrets, which all began with MONGO_
(for MongoDB) to begin with just DB_
instead, "hand typed" the secrets instead of pasting them from a .env
in vsCode and it started working π€ Now, I'm wondering if something with copy/paste was inserting some incorrect characters into the cli command OR if renaming had any bearing.
Hi @aflansburg ! That's so strange π€ I wonder if the hypothesis can be validated by running copilot svc package
and seeing if the secret names in the TaskDefinition contain any weird characters π
But I'm glad you got it working!
Hi @efekarakus , @huanjani I got the same problem! When Codepipeline running trigger by code merged, at the last phase
checked on Cloudformation:
Try to delete a parameter store, and recreate by copilot secret init
but still error.
I tried by create a change set from Cloudformation by the template I uploaded, It's updated successful without any errors. But it's just a workaround solution...I need the pipeline working well...
Sometime is another param
Hi @mvn-bachhuynh-dn ! thank you for reaching out to us,
would you mind copy pasting the SSM parameter names in secrets
if there is nothing confidential? The whole secrets
block would be super helpful for us to try to reproduce the issue
I ended up having this issue again, but determined it was a copy and paste error. When receiving the guidance from the CLI to update the manifest file with the secrets I was copying output like this:
MY_SECRET: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/MY_SECRET`
and leaving the character ` in the manifest file........ removing that fixed the issue π€¦
omg!
ok it sounds like if we print with code blocks ```
instead of just the `
character this bug scenario would go away.
We should replace https://github.com/aws/copilot-cli/blob/a28a4682b4bf53e616c7840077c30e6224170498/internal/pkg/cli/secret_init.go#L419
with color.HighlightCodeBlock
Hey all! I did some digging and was able to reproduce the issue (thanks @aflansburg!) and find something that I think makes this a hard bug to track down:
To reproduce:
TEST_SECRET_ONE
(i copy-pasted the same value in for each of my envβs)copilot svc deploy
- it works finecopilot svc deploy
, doesnβt work (relevant cfn in copilot svc package
:
Secrets:
- Name: TEST_SECRET_ONE
ValueFrom: /copilot/playground/beta/secrets/TEST_SECRET_ONE`
Error message:
Resource handler returned message: "Invalid request provided: Create T
askDefinition: The Systems Manager parameter name specified for secret
TEST_SECRET_ONE is invalid. The parameter name can be up to 2048 char
acters and include the following letters and symbols: a-zA-Z0-9_.-, (S
ervice: AmazonECS; Status Code: 400; Error Code: ClientException; Requ
est ID: 44b16b74-452a-4e55-ae73-10c7ae5f611e; Proxy: null)" (RequestTo
ken: ba1b9a59-caf9-de68-b843-c28b531ccc8e, HandlerErrorCode: InvalidRe
quest)
β deploy service iss-3446 to environment beta: deploy service: stack playground-beta-iss-3446 did not complete successfully and exited with status UPDATE_ROLLBACK_COMPLETE
copilot svc deploy
, still fails (relevant cfn from copilot svc package
:
Secrets:
- Name: TEST_SECRET_ONE
ValueFrom: /copilot/playground/beta/secrets/TEST_SECRET_ONE
(so the CFN template generated by Copilot appears to be getting updated - the ` is gone)
Error message:
β deploy service iss-3446 to environment beta: deploy service: change set with name copilot-14edaf98-e209-4ce3-913c-45b8c4fbcd2d for stack playground-beta-iss-3446 has no changes: Resource handler returned message: "Invalid request provided: Create TaskDefinition: The Systems Manager parameter name specified for secret TEST_SECRET_ONE is invalid. The parameter name can be up to 2048 characters and include the following letters and symbols: a-zA-Z0-9_.-, (Service: AmazonECS; Status Code: 400; Error Code: ClientException; Request ID: 44b16b74-452a-4e55-ae73-10c7ae5f611e; Proxy: null)" (RequestToken: ba1b9a59-caf9-de68-b843-c28b531ccc8e, HandlerErrorCode: InvalidRequest)
Based on this last error message, it appears that CloudFormation doesn't generate a change set when ValueFrom
changes in a Task Definition. If you make a different change in the manifest (like change count
), CloudFormation will pick up the secret change as well and deploy a new Task Definition that works.
Next steps:
ValueFrom
outside of Copilot and follow up with relevant the internal teams to make sure changes to ValueFrom
get detectedHi all and @efekarakus , here my secrets block on my manifest.yml. No any "`" at the end on any line.
secrets:
APP_NAME: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/APP_NAME
APP_ENV: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/APP_ENV
APP_KEY: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/APP_KEY
APP_DEBUG: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/APP_DEBUG
APP_URL: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/APP_URL
ASSET_URL: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/ASSET_URL
LOG_CHANNEL: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/LOG_CHANNEL
LOG_LEVEL: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/LOG_LEVEL
DB_CONNECTION: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/DB_CONNECTION
DB_HOST: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/DB_HOST
DB_PORT: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/DB_PORT
DB_DATABASE: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/DB_DATABASE
DB_USERNAME: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/DB_USERNAME
DB_PASSWORD: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/DB_PASSWORD
BROADCAST_DRIVER: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/BROADCAST_DRIVER
CACHE_DRIVER: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/CACHE_DRIVER
QUEUE_CONNECTION: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/QUEUE_CONNECTION
SESSION_DRIVER: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/SESSION_DRIVER
SESSION_LIFETIME: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/SESSION_LIFETIME
REDIS_HOST: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/REDIS_HOST
REDIS_PASSWORD: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/REDIS_PASSWORD
REDIS_PORT: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/REDIS_PORT
REDIS_CLIENT: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/REDIS_CLIENT
MAIL_MAILER: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/MAIL_MAILER
MAIL_HOST: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/MAIL_HOST
MAIL_PORT: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/MAIL_PORT
MAIL_USERNAME: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/MAIL_USERNAME
MAIL_PASSWORD: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/MAIL_PASSWORD
MAIL_ENCRYPTION: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/MAIL_ENCRYPTION
MAIL_FROM_ADDRESS: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/MAIL_FROM_ADDRESS
MAIL_FROM_NAME: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/MAIL_FROM_NAME
PUSHER_APP_ID: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/PUSHER_APP_ID
PUSHER_APP_KEY: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/PUSHER_APP_KEY
PUSHER_APP_SECRET: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/PUSHER_APP_SECRET
PUSHER_APP_CLUSTER: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/PUSHER_APP_CLUSTER
JWT_SECRET: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/JWT_SECRET
AWS_BUCKET: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/AWS_BUCKET
AWS_URL: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/AWS_URL
AWS_ENDPOINT: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/AWS_ENDPOINT
AWS_DEFAULT_REGION: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/AWS_DEFAULT_REGION
FE_URL: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/FE_URL
LOG_SLACK_WEBHOOK_URL: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/LOG_SLACK_WEBHOOK_URL
GATSBY_API_URL: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/GATSBY_API_URL
REACT_APP_API_URI: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/REACT_APP_API_URI
FE_ADMIN_URL: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/FE_ADMIN_URL
HAI2MAIL_API_PATH: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/HAI2MAIL_API_PATH
HAI2MAIL_API_TOKEN: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/HAI2MAIL_API_TOKEN
I wonder the Codebuild step have a problem, in the step, it will generate Cloudformation stack template and the Deploy step will use the generated CloudFormation stack template to create change set and excute. So please take a look again in the Codebuild phase.
When I use current CloudFormation stack template, try to add new secret manual, use it to create change set and excute...It works normally.
Hi All, I resolved it!
Rootcause:
buildspec.yml was too old for copilot old version. and It does not support ${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/
in the SSM name
Resolve:
copilot pipeline delete
copilot pipeline init
copilot pipeline deploy
Hey @mvn-bachhuynh-dn, glad you were able to resolve it! Do you know what version of Copilot you were using?
@dannyrandall Hi Danny, The old version I used: v1.8 Now, I updated it to v1.17
No matter how hard I try, I can't really get passed this error:
JWT_KEY
seems to be perfectly fine as a param name.I've reviewed this page a dozen times and still can't find what I am missing https://aws.github.io/copilot-cli/docs/commands/secret-init/
Any help is really appreciated
UPDATE: I'm running copilot v1.16.0 This is a Load Balanced Web Service