TAO dashboard is unable to be redeployed after upstream changes

[nchicong]

3 months ago, I pulled aws-cudos-framework-deployment repo, did some changes (mostly add vpc config as my org requires lamda to be placed in a vpc) and have TAO dashboard deployed successfully. At the time, I also had to put cid-0.2.22.zip into our own bucket as my stack could not reach s3://aws-managed-cost-intelligence-dashboards/cid-resource-lambda-layer/...

Today when I use the same CloudFormation code to remove and redeploy TAO dashboard, the stack throws error.

Received response status [FAILED] from custom resource. Message returned: Deploy failed: [... {'Type': 'COLUMN_REPLACEMENT_MISSING', 'Message': "Required column 'Estimated Monthly Savings Trend' is missing in data set 'arn:aws:quicksight:ap-southeast-1:xxx:dataset/ta-organizational-view'"}, {'Type': 'COLUMN_REPLACEMENT_MISSING', 'Message': "Required column 'Estimated Monthly Savings' is missing in data set 'arn:aws:quicksight:ap-southeast-1:xxxx:dataset/ta-organizational-view'"},...}]

I found that there were changes 2 months ago in this upstream git repo that breaks https://github.com/aws-samples/aws-cudos-framework-deployment/blame/68829fffb4e5b00c578117f79337a7bc45de0cfe/cid/builtin/core/data/datasets/tao/dataset.json#L1378

The question now is that how can I re-deploy the same TAO dashboard I had previously as I do not want to upgrade to latest dashboard and deal with merge conflicts as well as risk of changes.

---

In CidCustomResourceDashboard lambda log I saw a line

{'AwsAccountId': 'xxx', 'DashboardId': 'ta-organizational-view', 'Name': 'Trusted Advisor Organizational View', 'SourceEntity': {'SourceTemplate': {'Arn': 'arn:aws:quicksight:us-east-1:223485597511:template/ta-organizational-view/version/18', 'DataSetReferences': [{'DataSetPlaceholder': 'ta-organizational-view', 'DataSetArn': 'arn:aws:quicksight:ap-southeast-1:xxx:dataset/ta-organizational-view'}]}}, ....}

I guess it is automatically getting the template with latest version 18. Please advise how would I specify the right version for cid-0.2.22

[nchicong]

@esc1144 @iakov-aws Can you help to look into how to lock version of dashboard template?

[nchicong]

By checking log from August, I put hardcoded version number into this line https://github.com/aws-samples/aws-cudos-framework-deployment/blob/1b817a9d7931039d979c818e7f9bbff21ed3e0c7/cid/helpers/quicksight/__init__.py#L1270

This workaround helped to restore the dashboard.

if definition.get('dashboardId') == "ta-organizational-view":
            definition.get('sourceTemplate').version = 14

Would like to find the right way to fix this.

[yprikhodko]

Hi @nchicong ,

We only support update to the latest version of the dashboard template. In your case seems like you haven't done any customizations to the dashboard hence you could use the same stack to re-deploy dashboard, but you'd need to specify the latest version 0.2.39 of cid-cmd in the parameter. This version includes required schema for the dashboard template, so you'd be able to use latest version of the TAO dashboard with existing customized CloudFormation template.

The workaround which you've implemented also works, but the one I suggested above would help you to deploy the latest version of the dashboard though

[nchicong]

Hi @yprikhodko

I would not want to upgrade any part of our current dashboard stack as it might change some of the athena tables, views, quicksight datasets.

I'm using fixed version of cid-0.2.22.zip as I upload to our own S3, but it still fetch the latest dashboard template.

For now I switched to this code modifications as in this screenshot:

[yprikhodko]

I thought the goal was not to make changes to the modified stack. QuickSight datasets are managed by custom resource hence their update wouldn't require any change of the stack, just version of cid-cmd in parameter. But if concern is to keep dashboard and dataset version the same as well then I can confirm that the workaround which you've made is the only possible way to achieve that.