Big TODO list

[xitology]

Big TODO List

This is a list of features that need to be implemented before the project could become truly usable.

Compiler & Semantics

Selection

• [x] Prototype of :select().
• [x] Current implementation of select() cannot be followed by filter() and other combinators: employee:select(name):filter(salary>100000) fails. This is because select() works as a tuple constructor. Instead, select() should record the selection in the current scope, but leave the combinator expression intact. The tuple combinator should be attached in postprocessing step.
• [x] Default selection: employee should generate employee:select(name,surname,position,salary).
• [x] Column naming: how columns in select() are named by default, how as() (or :=, =>?) changes it.
• [ ] Semantics of *.
• [x] :dataframe() operator, similar to :select(), produces a DataFrame instead of a tuple.
• [x] Query parameters
• [x] :define() operator.
• [x] Selecting at the root scope: this:select(count(employee),max(employee.salary))?

  Locators

• [x] UniqueSeq monad (:locate() produces Opt for UniqueSeq input, Seq for Seq input).
• [ ] Configuring class identity.
• [x] id() and :locate() operators. The type of id?
• [ ] Composite identities.
• [ ] Can identities vary depending on the path (user[rbt-lang].repo[rbt-proto] vs repo[rbt-lang.rbt-proto])?
• [ ] When should the default selector produce the identity value (individual.mother is individual.mother.id)?

  Sort and Limit

• [x] :sort(), :asc and :desc. How do :asc and :desc affect the current scope?
• [ ] Default sorting for entities; :reverse.
• [x] :limit(), :first(), :last() and :one operators.

  Projections, etc

• [x] NonEmptySeq, NonEmptyUniqueSeq.
• [x] Quotient operator :by()
• [x] Variants :rollup(), :cube(), etc.
• [x] Transitive closure operator.
• [ ] Detaching operator; linking on arbitrary predicates.
• [x] Comonadic operations on NonEmptySeq: fork, prev, next, running aggregates.
• [x] Support for UNION ALL.

  Standard Library

• [x] Core logic and arithmetic operators; count() and max() aggregates.
• [x] :filter().
• [x] Support for more data types and standard functions, operators and aggregates. Automatic promotion of Julia functions?
• [x] Automatic lifting of null-oblivious scalar functions to Opt and Seq. Semantics of null-aware functions.

  Parser

• [x] Core syntax: literals, function calls (including postfix notation) and composition.
• [ ] Standalone parser.
• [ ] Selector: {} and table{} syntax.
• [ ] Filter: ? syntax. Resolving ambiguities.
• [ ] Locator: [] syntax, quoting?
• [ ] Syntax for sort, limit?
• [ ] Quotient: ^ syntax?

  Backends

• [x] Toy in-memory backend.
• [ ] Error reporting with context.
• [ ] "system" backend: user and process.
• [ ] Trivial SQL backend (rex.deploy only?).
• [ ] Some REST backends (github?).
• [ ] CSV to star schema backend.
• [ ] Pushing processing to SQL.
• [ ] Improved in-memory processing.
• [ ] Data caching and processing layer (based on a key-value storage?).

  Documentation

  Design Documentation

• [x] Introduction.
• [x] JSON combinators.
• [ ] Converting arbitrary schema to hierarchical model.

  User Guide and Language Reference

• [ ] Table of contents.

  API Documentation

• [ ] RBT.jl.
• [ ] PyRBT.

  Sample Queries

• [x] City of Chicago employees.
• [ ] More CSV datasets.
• [x] TPC-H queries.
• [ ] RexStudy?
• [ ] GitHub? Jira?

  API

• [ ] Python port.
• [ ] Cython speedups for Python port.
• [ ] Attaching and configuring data sources.
• [ ] Constructing and executing queries.
• [ ] Formatting output.
• [ ] Adding custom functions and data types.
• [ ] Adding custom data backends.

  Related Projects

• [ ] CRUD and ETL operations.
• [ ] Schema construction with idempotent operators.
• [ ] CRUD UI as "querying".

[zindel]

How about the JavaScript implementation?