Drop View Propagation Error

[aykut-bozkurt]

We propagate CREATE VIEW commands by filtering NON_COORDINATOR_METADATA_NODES (https://github.com/citusdata/citus/blob/v12.1.1/src/backend/distributed/commands/utility_hook.c#L1155), which chooses all worker nodes with hasmetadata = true. (looks like expected behaviour)

But we propagate "DROP VIEW" command by filtering NON_COORDINATOR_NODES (https://github.com/citusdata/citus/blob/v12.1.1/src/backend/distributed/commands/view.c#L289, which chooses all worker nodes including those with hasmetadata = false. (looks like wrong)

Above two causes a bug when worker nodes hasmetadata = false. Run below steps to reproduce it: (cluster is created by citus_dev make testCluster --destroy)

set citus.log_remote_commands TO 1;

create schema s1;
create table t(id int);
select create_distributed_table('t', 'id');

update pg_dist_node SET (metadatasynced, hasmetadata) = (false, false) where nodeid = 2;
update pg_dist_node SET (metadatasynced, hasmetadata) = (false, false) where nodeid = 3;

-- view won't be propagated to workers since they have `hasmetadata = false`
create view s1.testView as select * from t; 

-- view is tried to be removed from workers even if it is not created there
drop view s1.testView;

Workaround: One can run select start_metadata_sync_to_all_nodes(); sync metadata to workers before dropping the view.

Note We use NON_COORDINATOR_NODES instead of NON_COORDINATOR_METADATA_NODES while propagating many DDL. We should also consider those.

[onurctirtir]

Thanks for the detailed report, great investigation and workaround suggestion!