DontShaveTheYak / cf2tf

Convert Cloudformation templates to Terraform.
GNU General Public License v3.0
501 stars 82 forks source link

Not Converting AWS::Glue::Table to aws_glue_catalog_table #320

Open rknechtel opened 4 months ago

rknechtel commented 4 months ago

cf2tf is not Not Converting a CloudFormation AWS::Glue::Table to aws_glue_catalog_table. It creates it as an aws_route_table.

Example CF Definition:

  VpcFlowLogsAthenafl025c2085d26ae2213daily2024070920240709:
    Type: AWS::Glue::Table
    DependsOn:
      - VpcFlowLogsAthenaDatabase
    Properties:
      CatalogId: !Ref AWS::AccountId
      DatabaseName: !Ref VpcFlowLogsAthenaDatabase
      TableInput:
        Description: This table has the schema for vpc flow logs information.
        Name: fl025c2085d26ae2213daily2024070920240709
        PartitionKeys:
          - Name: year
            Type: string
          - Name: month
            Type: string
          - Name: day
            Type: string
        TableType: EXTERNAL_TABLE
        StorageDescriptor:
          Location:  s3://vpcflowlogs/AWSLogs/123456789012/vpcflowlogs/us-east-1/
          InputFormat: org.apache.hadoop.mapred.TextInputFormat
          OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
          SerdeInfo:
            Parameters:
              skip.header.line.count: "1"
              EXTERNAL: "true"
              field.delim: ' '
              serialization.format: ' '
            SerializationLibrary: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
          Columns:
            - Name: 'version'
              Type: int
            - Name: 'account_id'
              Type: string
            - Name: 'interface_id'
              Type: string
            - Name: 'srcaddr'
              Type: string
            - Name: 'dstaddr'
              Type: string
            - Name: 'srcport'
              Type: int
            - Name: 'dstport'
              Type: int
            - Name: 'protocol'
              Type: bigint
            - Name: 'packets'
              Type: bigint
            - Name: 'bytes'
              Type: bigint
            - Name: 'start'
              Type: bigint
            - Name: 'end'
              Type: bigint
            - Name: 'action'
              Type: string
            - Name: 'log_status'
              Type: string

Output:

resource "aws_route_table" "vpc_flow_logs_athena_table" {
  vpc_id = data.aws_caller_identity.current.account_id
  // CF Property(DatabaseName) = aws_glue_catalog_database.awsdefault_vpc_flow_logs_athena_database.arn
  // CF Property(TableInput) = {
  //   Description = "This table has the schema for vpc flow logs information."
  //   Name = "fl025c2085d26ae2213daily2024070920240709"
  //   PartitionKeys = [
  //     {
  //       Name = "year"
  //       Type = "string"
  //     },
  //     {
  //       Name = "month"
  //       Type = "string"
  //     },
  //     {
  //       Name = "day"
  //       Type = "string"
  //     }
  //   ]
  //   TableType = "EXTERNAL_TABLE"
  //   StorageDescriptor = {
  //     Location = "s3://vpcflowlogs/AWSLogs/123456789012/vpcflowlogs/us-east-1/"
  //     InputFormat = "org.apache.hadoop.mapred.TextInputFormat"
  //     OutputFormat = "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
  //     SerdeInfo = {
  //       Parameters = {
  //         skip.header.line.count = "1"
  //         EXTERNAL = "true"
  //         field.delim = " "
  //         serialization.format = " "
  //       }
  //       SerializationLibrary = "org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"
  //     }
  //     Columns = [
  //       {
  //         Name = "version"
  //         Type = "int"
  //       },
  //       {
  //         Name = "account_id"
  //         Type = "string"
  //       },
  //       {
  //         Name = "interface_id"
  //         Type = "string"
  //       },
  //       {
  //         Name = "srcaddr"
  //         Type = "string"
  //       },
  //       {
  //         Name = "dstaddr"
  //         Type = "string"
  //       },
  //       {
  //         Name = "srcport"
  //         Type = "int"
  //       },
  //       {
  //         Name = "dstport"
  //         Type = "int"
  //       },
  //       {
  //         Name = "protocol"
  //         Type = "bigint"
  //       },
  //       {
  //         Name = "packets"
  //         Type = "bigint"
  //       },
  //       {
  //         Name = "bytes"
  //         Type = "bigint"
  //       },
  //       {
  //         Name = "start"
  //         Type = "bigint"
  //       },
  //       {
  //         Name = "end"
  //         Type = "bigint"
  //       },
  //       {
  //         Name = "action"
  //         Type = "string"
  //       },
  //       {
  //         Name = "log_status"
  //         Type = "string"
  //       }
  //     ]
  //   }
  // }
}

It should create a resource like: resource "aws_glue_catalog_table" "vpc_flow_logs_athena_table" {

shadycuz commented 3 weeks ago

@rknechtel Thanks for reporting this =)