Instruction set of the `noharm-ir`

[SchrodingerZhu]

What are the basic operators for modern analytical queries? How to model IO operations to make them portable on various storage engine? What can be expected from a model much stronger than naive relational algebra?

[SchrodingerZhu]

cc @anqurvanillapy @fuzhe1989 @chuigda

[fuzhe1989]

What do you mean by "operators" here? Logical operators or physical operators? @SchrodingerZhu

[chuigda]

I'll summarize several fucking real-world relational algebra/relational database problems and maybe we'll be able to find some thing in common. Will update this floor once I have time.

Also, @imlyzh did some work with relational algebra so let her in.

---

1. A fucking multi-tenant problem

Multi-tenant is a common requirement in many to-B systems. Generally speaking, result of queries should get filtered automatically according to certain fields (tenants, departments, ...). Here are two problems:

• a) Writing filter conditions everywhere sucks, while automatically inject these conditions everywhere also sucks.
• b) For RDBMS, it is uneasy to change organization structure. For example when you have tenant -> department -> room, when user asks for a "sub-department" it would be a hazard.

To be updated then

[leiysky]

Is noharm-ir something like arrow and velox? Or datafusion?

[SchrodingerZhu]

@leiysky more like something that can be executed on an executor written w/ velox and then read data from arrow.

[leiysky]

@SchrodingerZhu I see. Maybe you can take a look at datafusion, which is a query engine based on arrow.

By the way, I was going to develop a query engine project https://github.com/db-bridge/db-bridge for educational purpose, but had no spare time for it then. If you are interested in it, we can take some discussion.