daphne-eu / daphne

DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines
Apache License 2.0
67 stars 62 forks source link

Codegen-only compilation pipeline for LA operations #693

Open philipportner opened 7 months ago

philipportner commented 7 months ago

Motivation: By default, DAPHNE lowers domain-specific operations from linear and relational algebra to calls to pre- compiled C++ kernels (physical operators). However, being based on MLIR/LLVM, on-the-fly code generation for these operations is an alternative option.

Task: Implement a proof-of-concept for a compilation pipeline in DAPHNE, which only involves code-generated operations. For simplicity, you can focus on linear algebra operations, whereby dense and sparse matrices should be supported. The resulting code may only target CPUs or, optionally, also lower to GPUs.

To start, take DaphneDSL of an example (e.g., from scripts/) and save the generated DaphneIR, or write an example DaphneIR file by hand. Use the daphne-opt tool to quickly iterate on your lowering passes and the whole lowering pipeline. Once this works for a set of scripts, bring the alternative compilation pipeline to the DAPHNE compiler through an optional command-line argument.

Note: Working on the GPU part of this task is optional and requires a GPU.