lookml_module updated to handle TIME and RECORD fields

[LydiaB123]

Problem Focus

Handling time fields

Droughty is not able to handle time fields, example below:

Big Query Field type time:

Looker LookML dimension type time, is not a valid type for a dimension.

Handling nested fields

From a previous PR the code included being able to handle big query records however there is a bug in the code. As the LookML explore output has the incorrect syntax shown below. Issues include:

• No braces around the referenced view_name.field_name
• Explore join calling the wrong field

LookML Sets ordering

Every time we run droughty, the order of the fields within the sets changes even when the warehouse tables have not changed this can cause merge conflict issues and make it difficult to see the changes that matter.

Solution

Handling time fields

When creating a view file from table in Looker the LookML creates fields with type time as a string Example:

  dimension: time_entry_start_time {
    type: string
    description: "A timestamp which defines the start time of the time entry."
    sql: ${TABLE}.time_entry_start_time ;;
  }
  dimension: time_entry_task_note {
    type: string
    description: "The time entry task notes in TimeCamp."
    sql: ${TABLE}.time_entry_task_note ;;
  }

Handling nested fields

Code changes include updates to the lookml_module.py:

"sql": f"left join unnest(${{{key}.{nested_view_name.split('__')[2]}}}) AS {nested_view_name}",

Output:

    join: wh_utilisation__jira_issues_fact__issue_comments {
      view_label: "Wh Utilisation Jira Issues Fact: Issue Comments"
      sql: LEFT JOIN UNNEST(${wh_utilisation__jira_issues_fact.issue_comments}) as wh_utilisation__jira_issues_fact__issue_comments ;;
      relationship: one_to_many
    }

In the screenshot below shows the existing droughty output and the new droughty output from the new code changes:

LookML Sets ordering

Code changes include updates to the lookml_module.py:

            # Convert the set back to a list for LookML set block and sort it alphabetically
            non_nested_fields = sorted(list(non_nested_fields_set))
In the screenshot below shows the existing droughty output and the new droughty output from the new code changes: