malloydata / malloy

Malloy is an experimental language for describing data relationships and transformations.
http://www.malloydata.dev
MIT License
1.96k stars 76 forks source link

`rename` should not require the field to be listed in `accept` #1288

Open vitiral opened 1 year ago

vitiral commented 1 year ago

What happens?

You seem to need to specify the field twice, both in accept: and in rename:

This is redundant and also surprising behavior coming from SQL. Malloy should auto-accept all renamed fields.

To Reproduce

source: groundhogs is table('duckdb:data/groundhogs.csv') extend {
    primary_key: the_day
    accept: the_day, lug, shortname, city, region,
            isGroundhog,
           -- shadow --   <---- this line is required
    rename: saw_shadow is shadow
}

line 5: no viable alternative at input 'table('duckdb:data/groundhogs.csv')extend{primary_key:the_dayaccept:the_day,lug,shortname,city,region,isGroundhog,-- shadow\nrename:'
  |     rename: saw_shadow is shadow
  |     ^

OS:

Linux

Malloy Client:

local VSCode

Malloy Client Version:

latest VS Code

Database Connection:

DuckDB

lloydtabb commented 1 year ago

Some SQL languages make accept and rename part of the same statement. Perhaps this is better as some kind of blocking syntax? @mtoy-googly-moogly putting a bug in your ear.

vitiral commented 1 year ago

Another pain point related to this: you need to accept a field to use it in dimension and where, even when you don't want it in the output schema. This requires me to make an extra query to get the data in the shape where it's ergonomic to view.

This is a regression from SQL's ergonomics and is surprising behavior for that reason.

lloydtabb commented 1 year ago

Maybe private:. ...

On Sat, Aug 12, 2023, 11:17 AM Rett Berg @.***> wrote:

Another pain point related to this: you need to accept a field to use it in dimension and where, even when you don't want it in the output schema. This requires me to make an extra query to get the data in the shape where it's ergonomic to view.

— Reply to this email directly, view it on GitHub https://github.com/malloydata/malloy/issues/1288#issuecomment-1676014895, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIK6UTKPKUA4567GN7XX5LXU63DFANCNFSM6AAAAAA3NXNULM . You are receiving this because you were assigned.Message ID: @.***>

vitiral commented 1 year ago

private would be nice for removing a sub-calculation, but I'm referring to something like this

source: mySource is ducdb.table('whatever.csv') extend {
  accept: a, b, c
  dimension:
    -- fails because `d` is not in accept
    d_minus_a is (d - a)
}

Private would let you do something like

source: mySource is ducdb.table('whatever.csv') extend {
  accept: a, b, c
  dimension:
    -- fails because `d` is not in accept
    d_minus_a is (d - a)
    d_minus_a_times_b is (d_minus_a * b)
  private:
    d_minus_a
}
lloydtabb commented 1 year ago

Yes, at some point do something like this. public, protected, private is the way it is handled in OOP, often. We will get to something like this when it makes it to the top of the list.