AWS::QuickSight::Template - AWS::QuickSight::*

[pgarbe]

1. Title

AWS::QuickSight::*

2. Scope of request

There'se no QuickSight resources available, although the scope of the API has recently been extended.

3. Expected behavior

Create DataSets, Analysis, Dashboards and Reportings via CloudFormation.

5. Helpful Links to speed up research and evaluation

https://docs.aws.amazon.com/quicksight/latest/APIReference/API_Operations.html

6. Category (required)

7. Analytics (Athena, EMR, Glue,...)

[moltar]

Probably not quite the right place to ask for this for it, but API support for configuring VPC Connections to QuickSight would be nice. https://docs.aws.amazon.com/quicksight/latest/user/vpc-creating-a-connection-in-quicksight.html

Then adding CloudFormation support for it too would be wonderful!

This is really a huge blocker.

We can codify everything: application, RDS, security groups, VPC.

And we can codify a data source, a data set, and a dashboard.

But the bridge between these two is missing, and thereby it breaks everything, especially when using things like AWS CDK (infra-as-code).

[vjancich]

Has anybody created AWS::QuickSight::DataSet for ATHENA DataSource ?

I'm stuck with "Model validation failed (#: extraneous key [DataSourceArn] is not permitted)" (RequestToken: 61292db1-xxx, HandlerErrorCode: InvalidRequest)

I have

  QSDataSource:
    Type: AWS::QuickSight::DataSource
    Properties:
      AwsAccountId: !Ref 'AWS::AccountId'
      DataSourceId: 'QSDataSource-XXX'
      Permissions:
        - Actions:
            - 'quicksight:DescribeDataSource'
            - 'quicksight:DescribeDataSourcePermissions'
            - 'quicksight:PassDataSource'
            - 'quicksight:UpdateDataSource'
            - 'quicksight:DeleteDataSource'
            - 'quicksight:UpdateDataSourcePermissions'
          Principal: !Sub
            - 'arn:aws:quicksight:eu-central-1:${AWS::AccountId}:user/default/myname'
            - Account: !Ref 'AWS::AccountId'
      Name: !Sub 'ApiGw-${Environment}'
      SslProperties:
        DisableSsl: false
      Type: ATHENA

  QSDataSet:
    DependsOn: QSDataSource
    Type: AWS::QuickSight::DataSet
    Properties:
      AwsAccountId: !Ref 'AWS::AccountId'
      DataSetId: 'QSDataSet-XXX'
      ImportMode: SPICE
      Name: ApiGwShort
      Permissions:
        - Actions:
            - 'quicksight:CreateDataSet'
            - 'quicksight:DeleteDataSet'
            - 'quicksight:DescribeDataSet'
            - 'quicksight:DescribeDataSetPermissions'
            - 'quicksight:PassDataSet'
            - 'quicksight:UpdateDataSet'
            - 'quicksight:UpdateDataSetPermissions'
          Principal: !Sub
            - 'arn:aws:quicksight:eu-central-1:${AWS::AccountId}:user/default/myname'
            - Account: !Ref 'AWS::AccountId'
      PhysicalTableMap:
        RelationalTable:
          DataSourceArn: !Sub 'arn:aws:quicksight:eu-central-1:${AWS::AccountId}:datasource/QSDataSource-XXX'
          InputColumns:
            - servicename
            - dt
            - responsesize
          Name: apigw_short
          Schema: !Sub 'logs-db-${Environment}'

I has the same issue @alerof, had to contact AWS support. PhysicalTableMap in DataSet in CF should look like this. It works for me know

      PhysicalTableMap:
        table:
          RelationalTable:
            Name: yourName
            DataSourceArn: !GetAtt DataSource.Arn
            InputColumns:
              - Name: column
                Type: STRING
            Schema: yourSchema

[denys-tyshetskyy]

Hi, I am trying to join 2 logical tables with Aurora being a datasource. Getting error: Unsupported join on clause "ptr-process-type = process-type-id" Code looks like this:

QSProcessDataSet:
    Type: AWS::QuickSight::DataSet
    Properties:
      DataSetId: !Join
        - ''
        - - QS-Process-DataSet
          - !Ref Suffix
      Name: !Join
        - ''
        - - QS-Process-DataSet
          - !Ref Suffix
      AwsAccountId: !Ref AWS::AccountId
      ImportMode: SPICE
      PhysicalTableMap:
          PhysicalTable1:
            RelationalTable:
              DataSourceArn: !GetAtt QSDataSource.Arn
              InputColumns:
              - Name: id
                Type: INTEGER
              - Name: ptr_process_type
                Type: INTEGER
              Name: process
          PhysicalTable2:
            RelationalTable:
              DataSourceArn: !GetAtt QSDataSource.Arn
              InputColumns:
              - Name: id
                Type: INTEGER
              Name: process_type
      LogicalTableMap:
          LogicalTable1:
            Alias: QS-Process-Table
            DataTransforms:
            - RenameColumnOperation:
                ColumnName: ptr_process_type
                NewColumnName: ptr-process-type
            Source:
              PhysicalTableId: PhysicalTable1
          LogicalTable2:
            Alias: QS-ProcessType-Table
            DataTransforms:
            - RenameColumnOperation:
                ColumnName: id
                NewColumnName: process-type-id
            Source:
              PhysicalTableId: PhysicalTable2
          LogicalTable3:
            Alias: QS-ProcessProcessTypeJoin-Table
            Source:
              JoinInstruction: 
                LeftOperand: LogicalTable1
                RightOperand: LogicalTable2
                RightJoinKeyProperties:
                   UniqueKey: true
                OnClause: ptr-process-type = process-type-id
                Type: LEFT

Am I missing something?

Solved: I had to change the order of the tables in the join

[denys-tyshetskyy]

Is there a way to join two datasets in cloudformation? When I am attempting to join LogicalTable from second dataset with LogicalTable from the first dataset, the cloudformation is unable to recognize the name of the LogicalTable from another dataset. Invalid request provided: Operands in JoinInstruction must refer to logical table id

[gunarkroeger]

• There seems to be a circular dependency between the AWS::QuickSight::Analysis and AWS::QuickSight::Template resources. Seemingly, you can't create an analysis without referencing a template. But you can't create a template without referencing an analysis or another template. How are you supposed to create the first template?

I am also running into this issue and agree that this should be re-opened.

It's not possible creating this with API/CLI nor CFN and CDK. Seems to be not only CFN who has a circular dependency.

I created this ticket in aws-cdk for this specific problem: https://github.com/aws/aws-cdk/issues/19212

[michuer]

In my brief experience trying to use QuickSight with CloudFormation, I've already seen a number of problems:

• As mentioned above by @nathanagez, the documentation is out of sync with reality. The Required field frequently does not reflect its true value (e.g. here and here)
• There seems to be a circular dependency between the AWS::QuickSight::Analysis and AWS::QuickSight::Template resources. Seemingly, you can't create an analysis without referencing a template. But you can't create a template without referencing an analysis or another template. How are you supposed to create the first template?
• I was expecting a way to describe the details of an analysis rather than just defining its related data sets (i.e. adding sheets, adding visuals to those sheets, etc.). Not being able to describe these things will require me to edit them through the UI in production, which somewhat defeats the purpose of using IAC. Like @mjdean1994, my organization really frowns on having to click through a UI to configure things

@rohits-spec I'd also suggest that this ticket is reopened, because the CloudFormation support does not seem usable in its current state. CC @jthomerson

Agreed this issue should be re-opened.

Hey, i can help you out with the circular dependency part.

1. create a Template using an "Manually created Analysis" as a Source entity.
2. Then Create one more template and analysis using Step1 template as Source entity.
3. i am creating template in Step2 to create Dashboards using that template.

References : https://devops.learnquicksight.online/quicksight-via-cloudformation.html you can download the cloudformation sample from here and they are using a publicly created template's arn, so you can describe to get the code of that.

Thankyou

Thank you for this workaround, it help with circular dependency problem

[sidpremkumar]

This is a huge problem for us, is there any update on when / if ever this will be fixed?

[gunarkroeger]

@zjinmei please move this out of shipped. As many before have pointed out, there is currently no way to automate the template to dashboard creation with CDK.

Problem 1: Circular dependency it is not possible to create a template without an analysis or an analysis without a template. The only way to workaround it is to manually create an analysis, which is not a proper solution for CD/CI

Problem 2: Changes are not diff-able if there are some changes done, we cannot do proper versioning with the code because the analysis is not really defined by cdk. Also, although we can define QuickSight dashboards on alpha, beta and prod stages, any changes will not be propagated because the pipeline does not detect that there were any changes.

Proposed solution: As mentioned before, we need to be able to create the template fully with CDK, with JSON fields to define the visuals, titles, etc.

[AkifRafique]

As there still unresolved issues changing status back to researching.

[Disciple153]

I wrote a solution to this problem with Python, and I am working on a PR to add functionality to CDK. Until QuickSight resources can be created using pure CDK, I think the best solution is to provide import functionality, and enable functionality to create copies of those imported resources. Once QuickSight resources can be created using pure CDK, that functionality can be added as well.

[drewmoore]

Also, cloudformation doesn't support tag-based row level security on datasets. This feature is essential for anonymous embedding, which is essential for my company's use case.

[seanlogan-wh]

It does not seem possible to reference a template version when creating a dashboard. As part of the pipeline I update the template from an existing template to create a new version. The dashboard references the template but it does not seem possible to pull the latest version from the template without a custom resource

  Dashboard:
    Type: AWS::QuickSight::Dashboard
    Properties:
      AwsAccountId: !Ref AWS::AccountId
      DashboardId: dashboard-id
      Name: dashboard-name
      SourceEntity:
        SourceTemplate:
        # Can't reference the template from CloudFormation above, it defaults to version 1 and can't use !GetAtt Version on Template
          Arn: !GetAtt Template.Version. VersionNumber or !GetAtt Template.Version (not currently supported)
          DataSetReferences:
            - DataSetPlaceholder: Placeholder
              DataSetArn: !GetAtt DataSet.Arn

I receive an error

Requested attribute Version.VersionNumber must be a readonly property or Attribute 'Version' does not exist Looking at the cloudformation documentation for template it lists version as a return value

[gunarkroeger]

I see it is now possible to create templates using json and the CLI commands: https://docs.aws.amazon.com/cli/latest/reference/quicksight/create-template.html

@AkifRafique, when can we expect this to be available for CDK too?

[TheDanBlanco]

hey folks, being entirely honest, this is a hard thread to follow as a non-quicksights user/expert. i found some comments on the cdk thread that leads me to believe this is solved. can anyone confirm?

[ericzbeard]

Since the original request on this issue has been completed, I'm going to close this issue and recommend that any outstanding bugs or coverage gaps in the QuickSight resources be reported on new issues. It is very difficult to isolate individual problems that need to be addressed from the long chain of comments here. While many of the gaps have been covered, we recognize that there are still some missing pieces to fully automating a CI/CD solution for this service. Please report each of them separately, and we'll get them assigned and prioritized.