datacontract / datacontract-cli

CLI to manage your datacontract.yaml files
https://cli.datacontract.com
Other
448 stars 86 forks source link

Object Arrays in SQL Not Being Created #467

Open mr-griffles opened 11 hours ago

mr-griffles commented 11 hours ago

Arrays of objects/structs are not being outputted in the SQL schema automated generation.

Example Contract

dataContractSpecification: 0.9.3
id: worms
info:
  title: Worms
  version: 0.0.1
  description: List of Worm Teams
  links:
    blog: hungovercoders.com
servers:
  production:
    type: databricks # this will default to context running in - see servers in data contract spec if want to be more remote 
    catalog: main
    schema: worms
models:
  worm_team:
    description: Worm teams and their members
    type: table
    fields:
      team_name:
        type: string
        enum: 
          - Aliens
          - South Park
          - Ghostbusters
          - The A Team
        required: true
        primary: true
        unique: true
        description: Name of the worms team
        example: "Aliens"
        pii: false
        domain: Worms
      team_members:
        type: array
        items:
          type: object
          description: Worm team members
          fields:
            name:
              type: string
              required: true
              description: Name of the worm
              example: "Hudson"
              pii: false
              domain: Worms
            age:
              type: integer
              required: true
              description: Age of the worm
              example: "10"
              pii: false
              domain: Worms

Current Output

-- Data Contract: worms
-- SQL Dialect: databricks
CREATE OR REPLACE TABLE main.worms.worm_team (
  team_name STRING not null primary key COMMENT "Name of the worms team",
  team_members ARRAY
) COMMENT "Worm teams and their members";

Desired Output

-- Data Contract: worms
-- SQL Dialect: databricks
CREATE OR REPLACE TABLE main.worms.worm_team (
  team_name STRING not null primary key COMMENT "Name of the worms team",
  team_members ARRAY <STRUCT<name STRING, age INT>> COMMENT "Worm Team Members"
) COMMENT "Worm teams and their members";
simonharrer commented 11 hours ago

Thanks for reporting this nicely written issue! I second this. This is also a good starting point for anyone who wants to contribute to the Data Contract CLI tool. @mr-griffles would you, by any chance, would like to have a go to contribute a pull request for this issue?