facebookincubator / velox

A composable and fully extensible C++ execution engine library for data management systems.
https://velox-lib.io/
Apache License 2.0
3.53k stars 1.16k forks source link

Custom Join Bridge Not Created for Any Join Node After First in Query Plan #11516

Open yjshen opened 1 week ago

yjshen commented 1 week ago

Bug description

When executing a query plan with multiple join nodes that require custom join bridges, only the first join node successfully creates a join bridge. All subsequent join nodes fail with the "Join bridge not found" error.

The issue occurs in Task.cpp:

void Task::addCustomJoinBridgesLocked(
    uint32_t splitGroupId,
    const std::vector<core::PlanNodePtr>& planNodes) {
  auto& splitGroupState = splitGroupStates_[splitGroupId];
  for (const auto& planNode : planNodes) {
    if (auto joinBridge = Operator::joinBridgeFromPlanNode(planNode)) {
      auto const inserted = splitGroupState.custom_bridges
                                .emplace(planNode->id(), std::move(joinBridge))
                                .second;
      VELOX_CHECK(
          inserted,
          "Join bridge for node {} is already present",
          planNode->id());
      return;  // <-- Bug: Early return after first join bridge
    }
  }
}

The early return statement causes the function to exit after processing the first join bridge, preventing the creation of bridges for subsequent joins.

Expected Behavior:

Join bridges should be created for all join nodes in the plan that require them, regardless of their position in the plan.

System information

I'm using an old version of Velox, but I checked the code for Task::addCustomJoinBridgesLocked, and it is unchanged.

Velox System Info v0.0.2 Commit: https://github.com/facebookincubator/velox/commit/5d315fbf05d56370ace659cd6c000a1ca15d98f6 CMake Version: 3.28.3 System: Linux-6.8.0-1017-gcp Arch: x86_64 C++ Compiler: /usr/bin/c++ C++ Compiler Version: 11.4.0 C Compiler: /usr/bin/cc C Compiler Version: 11.4.0 CMake Prefix Path: /usr/local;/usr;/;/usr/local/lib/python3.10/dist-packages/cmake/data;/usr/local;/usr/X11R6;/usr/pkg;/opt

Relevant logs

No response

Yuhta commented 1 week ago

Would you add a fix?