MaterializeInc / materialize

The data warehouse for operational workloads.
https://materialize.com
Other
5.66k stars 457 forks source link

parallel-workload: dependency u34 was removed #24367

Open def- opened 5 months ago

def- commented 5 months ago

What version of Materialize are you using?

6512ac0fb4

What is the issue?

Seen in https://buildkite.com/materialize/nightlies/builds/5943#018cf8e8-7ea9-4454-9d9d-f03a911fe97d This uses my open PR adding mz_sleep in queries: https://github.com/MaterializeInc/materialize/pull/24365

[worker_3] Query failed: SELECT "db-pw-1704983091-0"."s-1"."v-13"."value1-kafka_table1", COUNT("db-pw-1704983091-0"."s-1"."v-13"."c-0-bytea-t-3") OVER (PARTITION BY "db-pw-1704983091-0"."s-1"."v-13"."c-0-bytea-t-3" ORDER BY "db-pw-1704983091-0"."s-1"."v-13"."c-0-bytea-t-3"), mz_unsafe.mz_sleep(3) FROM "db-pw-1704983091-0"."s-1"."v-13" LIMIT 1; {'S': 'ERROR', 'C': '0A000', 'M': 'dependency u34 was removed'}

I haven't seen this kind of error before, so wondering if this is expected or not.

Thanks to @ggevay for suggesting using mz_sleep in queries.

ci-regexp: dependency .* was removed

nrainer-materialize commented 5 months ago

This also occurred in https://buildkite.com/materialize/nightlies/builds/5946#018cf9ca-e5b6-43fa-a8d7-dca5816edd9c and retry https://buildkite.com/materialize/nightlies/builds/5946#018cfa55-2e2a-4dbf-b0e0-c013f3d62891 in nightly on main.

def- commented 5 months ago

@nrainer-materialize informed me that this also happened in main: https://buildkite.com/materialize/nightlies/builds/5946#018cf9ca-e5b6-43fa-a8d7-dca5816edd9c

So it's not (solely) caused by my PR, but seems to be a potential regression. As @aalexandrov noted this could be a byproduct of moving optimization off-thread. For now I will add an ignore in parallel-workload.