tb push error during tinybird setup

[NirbhaySirsikar]

When the tb push command is ran, I get the following error in the terminal:

** Running 'dub_lead_events_pipe'
Error:
** Failed running ./pipes\dub_lead_events_pipe.pipe:
** Invalid results, read description below to fix the error:
** Failed pushing pipe dub_lead_events_pipe__checker: [Error] Missing columns: 'ip' 'qr' while processing query: 'SELECT timestamp, event_id, click_id, link_id, customer_id, event_name, metadata, url, country, city, device, browser, os, referer, qr, ip FROM nirbhay_new_dub_workspace.dub_lead_events AS dub_lead_events', required columns: 'link_id' 'timestamp' 'event_name' 'customer_id' 'click_id' 'country' 'city' 'qr' 'device' 'event_id' 'os' 'ip' 'referer' 'metadata' 'url' 'browser', maybe you meant: 'link_id', 'timestamp', 'event_name', 'customer_id', 'click_id', 'country', 'city', 'device', 'event_id', 'os', 'referer', 'metadata', 'url' or 'browser'. (UNKNOWN_IDENTIFIER)

[linear[bot]]

ENG-362 tb push error during tinybird setup

[Pratham271]

@NirbhaySirsikar can you share dub_click_events_pipe.pipe file that is in packages folder

[NirbhaySirsikar]

dub_click_events_pipe.pipe

NODE mv
SQL >

    SELECT
        timestamp,
        click_id,
        link_id,
        url,
        country,
        city,
        device,
        browser,
        os,
        referer,
        qr,
        ip
    FROM dub_click_events

TYPE materialized
DATASOURCE dub_click_events_mv

[Pratham271]

@NirbhaySirsikar, In the datasources folder, there are two files: dub_click_events_mv.datasource and dub_click_events.datasource. Please check both of these files to see if they contain the qr and ip fields.

[NirbhaySirsikar]

Yes, they do have ip and pr fields, and I haven’t modified any files in the /packages directory. I did a git pull a few days ago, and ever since then, I’ve had this error. Before that, everything was working fine.

[Pratham271]

sorry my bad I asked you to check in a wrong file, can you please check these in dub_lead_events_mv.datasource & dub_lead_events.datasource or can you share these file contents

[NirbhaySirsikar]

dub_lead_events_pipe.pipe

NODE mv
SQL >

    SELECT
        timestamp,
        event_id,
        click_id,
        link_id,
        customer_id,
        event_name,
        metadata,
        url,
        country,
        city,
        device,
        browser,
        os,
        referer,
        qr,
        ip
    FROM dub_lead_events

TYPE materialized
DATASOURCE dub_lead_events_mv

dub_lead_events.datasource

SCHEMA >
  `timestamp` DateTime64(3) `json:$.timestamp` DEFAULT now(),
  `event_id` String `json:$.event_id`,
  `event_name` String `json:$.event_name`,
  `metadata` String `json:$.metadata`,
  `customer_id` String `json:$.customer_id`,   
  `click_id` String `json:$.click_id`,
  `link_id` String `json:$.link_id`,    
  `url` String `json:$.url`,     
  `country` LowCardinality(String) `json:$.country`,
  `city` String `json:$.city`,
  `region` String `json:$.region`,
  `latitude` String `json:$.latitude`,
  `longitude` String `json:$.longitude`,
  `device` LowCardinality(String) `json:$.device`,
  `device_model` LowCardinality(String) `json:$.device_model`,
  `device_vendor` LowCardinality(String) `json:$.device_vendor`,
  `browser` LowCardinality(String) `json:$.browser`,
  `browser_version` String `json:$.browser_version`,
  `os` LowCardinality(String) `json:$.os`,
  `os_version` String `json:$.os_version`,
  `engine` LowCardinality(String) `json:$.engine`,
  `engine_version` String `json:$.engine_version`,
  `cpu_architecture` LowCardinality(String) `json:$.cpu_architecture`,
  `ua` String `json:$.ua`,
  `bot` UInt8 `json:$.bot`,
  `referer` String `json:$.referer`,
  `referer_url` String `json:$.referer_url`,
  `ip` String `json:$.ip`,
  `qr` UInt8 `json:$.qr`

ENGINE "MergeTree"
ENGINE_PARTITION_KEY "toYYYYMM(timestamp)"
ENGINE_SORTING_KEY "timestamp, link_id, customer_id"

dub_lead_events_mv.datasource

SCHEMA >
    `timestamp` DateTime64(3),
    `event_id` String,
    `click_id` String,
    `link_id` String,
    `customer_id` String,
    `event_name` LowCardinality(String),
    `metadata` String,
    `url` String,
    `country` LowCardinality(String),
    `city` String,
    `device` LowCardinality(String),
    `browser` LowCardinality(String),
    `os` LowCardinality(String),
    `referer` String,
    `qr` UInt8,
    `ip` String

ENGINE "MergeTree"
ENGINE_PARTITION_KEY "toYYYYMM(timestamp)"
ENGINE_SORTING_KEY "link_id, timestamp"

[steven-tey]

Hey! This should be fixed now – we had an outdated Tinybird schema/pipes before, but it was updated recently – sorry about that!

Feel free to open a new issue if you're still running into issues!