malloydata / malloy

Malloy is an experimental language for describing data relationships and transformations.
http://www.malloydata.dev
MIT License
1.92k stars 75 forks source link

Snowflake: Nested Pipelines #1655

Open lloydtabb opened 3 months ago

lloydtabb commented 3 months ago

In Snowflake you can't use a view that contains a pipeline as a nest. Other dialects handle this with correlated subqueries over arrays. Unfortunately Snowflake doesn't support correlated subqueries.

Here is an example of a test that fails

         source: airports is snowflake.table('malloytest.airports') extend {
          measure: airport_count is count()

          view: pipe is {
            group_by: city, state
            aggregate: a is airport_count
          }
          -> {
            group_by: state
            aggregate: b is a.sum()
          }
        }

        run: airports -> {
          group_by: faa_region
          aggregate: airport_count
          nest: pipe
        }