servicecatalog-puppet- pipeline is not updating the products in the spoke accounts

[vijayrajarajana]

Hi Everyone,

I am having issue with the below error and found none of the products are updating using the servicecatalog-puppet-pipeline but I can see the latest version of the product when it triggering through servicecatalog-factory-pipeline and abc-blueprint-v3-pipeline which were successful. My SC-factory and SC-puppet pipelines are in the same account and region.

When I am checking the same in Dev environment everything looks normal and I am getting issue in Prod environment. Hoping for the earliest resolution. Thanks in advance.

From the CodeBuild Logs INFO MainThread Sending failure to opscenter: ProvisionProductTask failed: abc-blueprint - 1234567890xx - us-east-1 ProvisionProductTask failed parameters: account_id: '1234567890xx' account_parameters: '{''Path'': {''default'': ''''}, ''RoleName'': {''default'': ''''}}' execution: hub launch_name: abc-blueprint launch_parameters: '{}' manifest_file_path: manifest-expanded.yaml manifest_parameters: '{}' portfolio: ***-global-mandatory-blueprints product: abc-blueprint puppet_account_id: 0987654321xx region: us-east-1 requested_priority: 0 retry_count: 1 ssm_param_inputs: [] ssm_param_outputs: [] version: v1 worker_timeout: 0

Traceback (most recent call last):

File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/luigi/worker.py", line 191, in run new_deps = self._run_get_new_deps()

File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/luigi/worker.py", line 133, in _run_get_new_deps task_gen = self.task.run()

File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/servicecatalog_puppet/workflow/launch/provision_product_task.py", line 225, in run self.should_use_sns,

File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/servicecatalog_puppet/***.py", line 240, in provision_product_with_plan service_catalog.execute_provisioned_product_plan(PlanId=plan_id)

File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/botocore/client.py", line 391, in _api_call return self._make_api_call(operation_name, kwargs)

File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/botocore/client.py", line 719, in _make_api_call raise error_class(parsed_response, operation_name)

botocore.errorfactory.InvalidParametersException: An error occurred (InvalidParametersException) when calling the ExecuteProvisionedProductPlan operation: Cannot execute ProvisionProduct Plan. Plan is in Invalid State CREATE_FAILED

[eamonnfaherty]

Sorry to hear you are having an issue. Could you please share which version of puppet you are using.

[vijayrajarajana]

I am using aws-service-catalog-puppet==0.135.1.

[vijayrajarajana]

Hello @eamonnfaherty,

Any update on this.

[eamonnfaherty]

Hi

Could you please retest this with the latest version. This was an issue in the past and was fixed.

[eamonnfaherty]

Any update on this? I will close this issue soon if I do not hear back.

[eamonnfaherty]

If you cannot update you may have to remediate the issue yourself. You can try deleting the AWS Service Catalog Provisioning Plan and retrying the pipeline.

[sowmyapallapothu]

Hi @eamonnfaherty During the account retirement process, we are facing with the below error when we tried to delete the products like vpc,& account details

ERROR: [pid 742] Worker Worker(salt=891478314, workers=, host=7a34aaad4309, username=root, pid=447) failed ProvisionProductTask(manifest_file_path=manifest-expanded.yaml, launch_name=guardduty-org-bootstrap, puppet_account_id=, region=us-east-1, account_id=, portfolio=its-test-ct-test-portfolio, product=guardduty-org-bootstrap, version=v1, execution=hub)

192 | Traceback (most recent call last): 193 | File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/luigi/worker.py", line 191, in run 194 | new_deps = self._run_get_new_deps() 195 | File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/luigi/worker.py", line 133, in _run_get_new_deps 196 | task_gen = self.task.run() 197 | File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/servicecatalog_puppet/workflow/launch/provision_product_task.py", line 225, in run 198 | self.should_use_sns, 199 | File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/servicecatalog_puppet/*.py", line 240, in provision_product_with_plan 200 | service_catalog.execute_provisioned_product_plan(PlanId=plan_id) 201 | File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call 202 | return self._make_api_call(operation_name, kwargs) 203 | File "/root/.pyenv/versions/3.7.16/lib/python3.7/site-packages/botocore/client.py", line 676, in _make_api_call 204 | raise error_class(parsed_response, operation_name) 205 | botocore.errorfactory.InvalidParametersException: An error occurred (InvalidParametersException) when calling the ExecuteProvisionedProductPlan operation: Cannot execute ProvisionProduct Plan. Plan is in Invalid State CREATE_FAILED 206 | ERROR MainThread [pid 742] Worker Worker(salt=891478314, workers=**, host=7a34aaad4309, username=root, pid=447) failed ProvisionProductTask(manifest_file_path=manifest-expanded.yaml, launch_name=guardduty-org-bootstrap, puppet_account_id=***, region=us-east-1, account_id=****, portfolio=its-test-ct-test-portfolio, product=guardduty-org-bootstrap, version=v1, execution=hub)

The puppet version that we are currently using is 0.129.0. Do you think this is an issue with the version? Because in a different environment where it has the same version the pipeline successfully. But facing issues in test environment. Please suggest.

[eamonnfaherty]

I think it may be worth having a call to discuss this. If you have my email could you please email me 3 different times that work for you. If you do not have my email let me know here and I will contact you.

[sowmyapallapothu]

Hi Eamonn, Thank you for your quick response like always. Naveen from AWS has informed me that he would be scheduling a call over chime. Provided him my availability, looking forward to talk to you.