malloydata / malloy

Malloy is an experimental language for describing data relationships and transformations.
http://www.malloydata.dev
MIT License
1.99k stars 76 forks source link

Project is not working correctly in pipelines... #148

Closed lloydtabb closed 2 years ago

lloydtabb commented 2 years ago

This fails to compile. Changing project to group_by works.

      query: table('malloytest.aircraft')->{
        aggregate: f is count(*)
      }->{
        project: f2 is f+1
      }

This also fails. Might be the same thing. Changing 'project' to 'group_by' allows these to compile.

      query: table('malloytest.airports')->{
        aggregate: airport_count is count()
        nest: pipe_turtle is {
          group_by: [
            state
            county
          ]
          aggregate: a is count()
        } -> {
          project: [
            state is upper(state)
            a
          ]
        } -> {
          group_by: state
          aggregate: total_airports is a.sum()
        }
      }
mtoy-googly-moogly commented 2 years ago

OK, what things are legal in project ...

1) project: [ exisitingFieldName, newName is Expression ]

Anything else which is legal. or which should be illegal?

mtoy-googly-moogly commented 2 years ago

Removed "do not allow any measures in project" and now

query: a -> { aggregate: f is count() } -> { project: f2 is f + 1 }

compiles without error

exploreName->{
  project: [
    measureName, // pretty sure this is legal
    newM is measureName * 2, // lloyd seems to indicate this is also legal
    avgM is ANY_AGGREGATE_FUNCT(), // this is non-sensical and illegal, right?
  ]

Not entirely sure how I make "avgM" error, that might be some work.

lloydtabb commented 2 years ago

Your example says 'measureName', but I think you mean 'dimensionName'. Right?

lloydtabb commented 2 years ago

In your example

query: a -> { aggregate: f is count() } -> { project: f2 is f + 1 }

f is the output name of a measure calculation in the first segment f is a dimension in the second segment (as is f2)

lloydtabb commented 2 years ago

I'm curious how come you think f is a measure in the second segment. If you are asking malloy_query for the output structure, it will tell you it is a dimension...

mtoy-googly-moogly commented 2 years ago

ok, project: [] lists need to be like group_by -- scalars and scalar defs only

mtoy-googly-moogly commented 2 years ago

I believe this is now (ea8d294c5b7759693293a93d63674e78adcc4528) as fixed as it is going to be. project: is a little bit picky about what goes in there, but it could be more picky and when that comes up, we'll file a new bug