malloydata / malloy

Malloy is an experimental language for describing data relationships and transformations.
http://www.malloydata.dev
MIT License
2.01k stars 76 forks source link

Improve source namespace editing actions (`accept`, `except`, `rename`) #1808

Open christopherswenson opened 3 months ago

christopherswenson commented 3 months ago

FROM TODO: https://github.com/malloydata/malloy/blob/main/test/src/databases/all/parameters.spec.ts#L90-L94

Excepting a field outright removes it from the source, without consideration to other fields that use that removed field in their definition. Since those other fields have already been translated, they are still usable, but then fail to compile.

I think we should consider changing the behavior of accept, except, and rename to modify the "output space" of the new source, while leaving the "input space" the same.

source: state_facts is duckdb.table('malloytest.state_facts') extend {
  dimension: state_copy is state
}

source: state_facts_ext is state_facts extend {
  except: state
}

run: state_facts_ext -> {
  group_by: state_copy // Compile-time error
  order_by: state_copy desc
  limit: 1
}

Today, however, rename combined with dimension can be used to deeply override the definition of a field, making other field definitions that use the field (defined in the base source) use the new definition. (This is akin to a subclass overriding the definition of a method originally defined in a base class in OO languages.) We could introduce a specific mechanism for this, e.g.:

source: state_facts is duckdb.table('malloytest.state_facts')

source: state_facts_ext is state_facts extend {
  // The old way
  // rename: old_state is state
  // dimension: state is concat("STATE: ", old_state)

  // The new way
  override dimension: state is concat("STATE: ", super.state)
}