Error while deploying dashboard with cid-cmd

[andreyaurelien]

I try to deploy the CUDOS dashboard with the commande line (cid-cmd).

• I followed the instruction and launch successfully :

python3 -m ensurepip --upgrade
pip3 install --upgrade cid-cmd

Checking the version cid-cmd :

Then I launch the deployment with cid-cmd deploy and followed the command line steps.

At the step where we choose the crawler role, I let the defaut value and I got the following error :

 [crawler-role] Provide a crawler role name: CidCmdCurCrawlerRole <CREATE NEW>
CRITICAL - Started crawler CidCmdCurCrawler                                                                                                                                                                                                                                              
Please check crawler in a few minutes https://console.aws.amazon.com/glue/home?#/v2/data-catalog/crawlers
Checking if CUR is enabled and available...
Traceback (most recent call last):
  File "/home/cloudshell-user/.local/bin/cid-cmd", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/cli.py", line 35, in wrapper
    res = func(ctx, **kwargs)
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/cli.py", line 140, in deploy
    ctx.obj.deploy(**kwargs)
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/common.py", line 169, in wrap
    return func(self, *args, **kwargs)
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/common.py", line 376, in deploy
    self._deploy(dashboard_id, recursive, update, **kwargs)
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/common.py", line 503, in _deploy
    self.create_datasets(required_datasets_names, dashboard_datasets, recursive=recursive, update=update)
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/common.py", line 1222, in create_datasets
    if self.create_or_update_dataset(dataset_definition, dataset_id, recursive=recursive, update=update):
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/common.py", line 1475, in create_or_update_dataset
    self.create_or_update_view(view_name, recursive=recursive, update=update)
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/common.py", line 1579, in create_or_update_view
    self.accountMap.create(view_name) #FIXME: add or_update
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/common.py", line 143, in accountMap
    _account_map.cur = self.cur
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/common.py", line 121, in cur
    if not _cur.metadata:
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/helpers/cur.py", line 177, in metadata
    self._metadata = self.athena.get_table_metadata(table_name)
  File "/home/cloudshell-user/.local/lib/python3.9/site-packages/cid/helpers/athena.py", line 280, in get_table_metadata
    table_metadata = self.client.get_table_metadata(**params).get('TableMetadata')
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 565, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 1021, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.MetadataException: An error occurred (MetadataException) when calling the GetTableMetadata operation: Entity Not Found (Service: AmazonDataCatalog; Status Code: 400; Error Code: EntityNotFoundException; Request ID: f5d9bbfa-0090-4918-9dce-27b9d0574dc4; Proxy: null)

It seems that we don't wait enough time to go to the next step and to use the role which is going to be create : i've opened a second windows with IAM console and the role seems to be successfully created few moments later.

When I retry to deploy just after, I don't get any error and the dashboard and all associated resources are well createad.

I reproduced this behavior each time I start from fresh.

Thanks for your support.

[iakov-aws]

Thank you for reporting this. Testing the fix in https://github.com/aws-samples/aws-cudos-framework-deployment/pull/827

[iakov-aws]

released in the new version.