search

openflighthpc / flight-cloud

Cloud orchestration tool
Eclipse Public License 2.0
3 stars 2 forks source link

Can not destroy a deployment if the credentials are invalid #286

Open WilliamMcCumstie opened 5 years ago

WilliamMcCumstie commented 5 years ago

I was having a credentialing issue that was causing the deployments to fail (as expect). The deployments fail in a "Running" state, which prevents them being redeployed without calling destroy first. This is to handle other error conditions where the stack exists in the failed state.

It is probably worth failing in a offline state if bad credentials are used. This will mitigate credentialing errors but can not remove them entirely. It is still possible for credentials to expire or be revoked and thus lead to a similar bug as follows below.

[root@forgiemodo flight-cloud]# bin/cloud deploy domain
[┐] Deploying domain... Done
cloud: An error has occurred deploying the domain.
`cloud list --verbose`

[root@forgiemodo flight-cloud]# bin/cloud list --verbose
Deployment: ‘domain’
ERROR: An error occured whilst deploying this template

Creation Date: 2019-09-05 13:01:10 +0100
Status: Running
Provider Tag: forgiemodo-domain-c2b8e00da5

  Results
  No deployment results

  Replacements
  ● deployment_name: domain

  Error
  NOTE: This is aws's raw error message
  Refer to their documentation for further details

  The security token included in the request is invalid.

The following bug occurs when destroy is called. The command appears to be running correctly, but is actually failing silently:

[root@forgiemodo flight-cloud]# bin/cloud destroy node01
[┌] Destroying resources for node01... Done

[root@forgiemodo flight-cloud]# bin/cloud list --verbose
Deployment: ‘domain’
ERROR: An error occured whilst deploying this template

Creation Date: 2019-09-05 13:01:10 +0100
Status: Running
Provider Tag: forgiemodo-domain-c2b8e00da5

  Results
  No deployment results

  Replacements
  ● deployment_name: domain

  Error
  NOTE: This is aws's raw error message
  Refer to their documentation for further details

  The security token included in the request is invalid.

The destroy is failing because it can not contact AWS due to the bad credentials. In this case, the stack does not exist, but this can not always be assumed. It is possible the credentials have been revoked and the stack does indeed exist. Therefore an error should be raised saying the destroy failed and leave deployment in the Running state

WilliamMcCumstie commented 5 years ago

This maybe related to this issue: #217