pg_shard keeps a cache of shard interval metadata within each session which is never refreshed. Within pg_shard this is fine, since all shards are created up front: caching forever isn't wrong. But now that Citus integration is more full-fledged, we're noticing that things like appending new shards or rebalancing existing ones results in pg_shard operating using stale metadata.
We need to come up with some sort of contract to let other pieces of software tell pg_shard about changes in the shard intervals of a distributed table. We may be able to reuse some existing pieces of invalidation logic within PostgreSQL, but that requires some reading to determine.
pg_shardkeeps a cache of shard interval metadata within each session which is never refreshed. Withinpg_shardthis is fine, since all shards are created up front: caching forever isn't wrong. But now that Citus integration is more full-fledged, we're noticing that things like appending new shards or rebalancing existing ones results inpg_shardoperating using stale metadata.We need to come up with some sort of contract to let other pieces of software tell
pg_shardabout changes in the shard intervals of a distributed table. We may be able to reuse some existing pieces of invalidation logic within PostgreSQL, but that requires some reading to determine.