dubinc / dub

Open-source link management infrastructure. Loved by modern marketing teams like Vercel, Raycast, and Perplexity.
https://dub.co
GNU Affero General Public License v3.0
18.99k stars 2.06k forks source link

[#1483 duplicate] Error: `Cannot materialize node: The pipe has columns ['metadata'] not found in the destination Data Source` when running Dub.co's pipe #1622

Closed unrenamed closed 1 month ago

unrenamed commented 1 month ago

Issue Summary

While following the guide to run Dub.co’s codebase locally, I encountered an error when executing the ./pipes/dub_lead_events_pipe.pipe. The error message indicates that the pipe is attempting to materialize columns that are not found in the destination Data Source, specifically mentioning the column ['metadata'].

Steps to Reproduce

  1. Follow the setup instructions provided in the guide up to Step 2: Set up the Tinybird Clickhouse database.
  2. After you run tb push, wait for the command to fail and observe the error message like on the screenshot below.

Expected behavior

The pipe should execute successfully, materializing the required data into the destination Data Source without errors.

How to fix

The metadata column must be either added to dub_lead_events_mv.datasource if it's still required or removed from dub_lead_events_pipe.pipe in case it's no more relevant.

Screenshots

image image

Version info

linear[bot] commented 1 month ago

ENG-869 Error: `Cannot materialize node: The pipe has columns ['metadata'] not found in the destination Data Source` when running Dub.co's pipe

unrenamed commented 1 month ago

My bad, forgot to check if anyone's facing the same issue. This is a duplicate of #1483. I'm keeping it for now as it has some screenshots with more insights. @steven-tey feel free to close any of these two.