search

StarRocks / starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
https://starrocks.io
Apache License 2.0
9k stars 1.81k forks source link

[Feature] Support expression JIT #28331

Closed Moonm3n closed 3 months ago

Moonm3n commented 1 year ago

Feature request

Description

Overview

For OLAP database, optimizing CPU performance is paramount. JIT (Just-In-Time compilation) can dynamically adapt to the workload and data type, generating customized machine code that minimizes instruction counts, ultimately resulting in faster data processing. Using JIT together with vectorized execution can further enhance runtime efficiency.

Expression evaluation, characterized by intricate procedures and substantial materializations of intermediate results, stands out as a pivotal candidate for JIT optimization. The use of JIT compilation can streamline these complexities by merging multiple operations, thus reducing the temporary storage and enhancing CPU cache utilization. This, in turn, minimizes the pressure on memory bandwidth.

Primary Benefits

  1. Reduction in Materialization of Intermediary Results. Before implementing JIT, each expression operator creates a temporary column to store the results. The number of materialization operations equals the number of expression nodes. By optimizing with JIT, multiple operators can be combined into a single operator, performing only one materialization operation and making better use of registers and CPU cache performance.
  2. Decrease in Instruction Count. Adaptable optimization is performed based on the actual execution circumstances of the program during runtime, reducing the total number of instructions.
  3. Elimination of Function Calls. Inline optimization occurs according to the runtime data types of the program, reducing the overhead associated with function calls.

Solution

Overall, I plan to implement StarRocks expression JIT in conjunction with LLVM JIT, which involves the following components:

  1. Adding the LLVM library to StarRocks project.
  2. Implementing a JIT execution engine. This involves creating a simple wrapper around the LLVM ORCv2 execution engine and developing a custom memory manager.
  3. Generating LLVM IR. Utilizing the LLVM-provided IRBuilder interface, we can manually implement JIT Scalar Functions, expression IRs, etc.
  4. Identifying compilable expr subtrees.
  5. Implementing compilation cache.
  6. Unit testing, integration testing, and performance testing.

Additional

I plan to contribute by submitting PRs to tackle this issue in multiple steps:

  1. PR1: Incorporate the LLVM third-party library into the project. #28383
  2. PR2: Implement a prototype demonstration on #28477 #34365
  3. PR3: Implement type cast expression JIT. #33590 #36742
  4. PR4: Implement DIV and MOD expression JIT. #36742
github-actions[bot] commented 4 months ago

We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!