Implement "WITH" relvar macro

[agentm]

The TutorialD WITH macro allows the user to create a handy relvar macro useable for the duration of a single expression.

WITH (ct:=s{city,status}) : <relational expression>

The implementation of this feature may hint that we need to finally separate the AST from the execution ADT.

[YuMingLiao]

hi @agentm

Does that mean we need something like this code?

[agentm]

I haven't thought about it too much, but I imagined a sort of "Macro" named value, which, when evaluated, would temporarily add a relvar to the relvar dictionary. I don't think we would rely on reification, or, at least, I don't see the advantage yet.

The WITH macro syntax allows a relation to be constructed and used multiple times within a relational expression, similarly as the same keyword in SQL.

[YuMingLiao]

Oh, sorry I wasn't clear about my question. I was trying to understand what you mean about separating AST and ADT. Did you mean right now the relational expression is ADT and you want an AST?

[agentm]

Oh, I see. I was thinking that we could have two ADTs, one to better represent the TutorialD structures (AST), and a simpler one specific to the relational algebra. Right now, they are the same ADT. The TutorialD representation would compile to the simpler ADT, just like GHC compiles Haskell to Core.

The downside to this approach is a lot of duplicate/similarly-named ADTs.

For this specific issue, I don't think it's necessary to implement that separation- we can consider that for the future.

[agentm]

Oh, I just realized that it might not be possible to implement this without changes to the ADT for RelationalExpr. The reason is that evaluation of RelationalExpr (evalRelationalExpr) does not allow for modification of the relation variable map. I think that's OK, though- I see two possible implementation strategies:

• create a new tutd-specific ADT to better represent the AST and compile to RelationalExpr
• pass a temporary read-write relation variable mapping in evaluation of RelationalExpr

I have no strong preference, but I suspect that option 1 would take more time to implement but probably be a better long-term solution. What do you think?

[YuMingLiao]

Option1 sounds better for long term to me too.

From what I can understand, relational algebra ADT is for just relational expression. And a new tutd-specific ADT can express more, like DatabaseContextExpr, DatabaseContextIOExpr, ... maybe even a DataFrameExpr in the future.

The downside to this approach is a lot of duplicate/similarly-named ADTs.

What if we data TutdExpr = ... | With Relation RelationalExpr | ... using existing ADT? Can it save some duplicates?

[YuMingLiao]

Oh, I just realized that data TutdExpr = ... | With RelationalExpr RelationalExpr | ... alone can't solve the problem. evalRelationalExpr still needs to pass relation variable mapping around.

[YuMingLiao]

With macro feels like something similar with #52. What if we add relation variable mapping in RelationalExprStateElems (something I haven't got it through)? What's long term gain of option1 over option2?

[agentm]

I don't have a strong feeling that option 1 is better, but it would allow for future complexity in the tutd grammar without affecting the RelationalExpr structure. Otherwise, I think it's fine to use the relation expression state to pass around temporary relvars as a first pass and I would merge such a patch.

[YuMingLiao]

I've made a first pass, using the relation expression state. I use AS instead of := at a first pass just try to help me implement it from SQL WITH. However, It can be changed for any good reason. It supports multiple views. However, it can only reference previous ones.

TutorialD (master/main): :importexample date
TutorialD (master/main): :showexpr WITH (a AS s{s#,city}, b AS a{city}) b
┌──────────┐
│city::Text│
├──────────┤
│"Paris"   │
│"London"  │
│"Athens"  │
└──────────┘