citusdata / citus

Distributed PostgreSQL as an extension
https://www.citusdata.com
GNU Affero General Public License v3.0
10.5k stars 666 forks source link

Columnar: options ignored during ALTER TABLE rewrite #5927

Closed jeff-davis closed 2 years ago

jeff-davis commented 2 years ago
CREATE TABLE test(i int) USING columnar;
SELECT alter_columnar_table_set('test'::regclass, compression=>'lz4');
INSERT INTO test VALUES(1);
VACUUM VERBOSE test;
INFO:  statistics for "test":
storage id: 10000000006
total file size: 24576, total data size: 6
compression rate: 0.83x
total row count: 1, stripe count: 1, average rows per stripe: 1
chunk count: 1, containing data for dropped columns: 0, lz4 compressed: 1

Compressed with `lz4'.

ALTER TABLE test ALTER COLUMN i TYPE int8;
VACUUM VERBOSE test;
INFO:  statistics for "test":
storage id: 10000000007
total file size: 24576, total data size: 18
compression rate: 0.50x
total row count: 1, stripe count: 1, average rows per stripe: 1
chunk count: 1, containing data for dropped columns: 0, zstd compressed: 1

Now it's compressed with zstd.

INSERT INTO test VALUES(2);
VACUUM VERBOSE test;
INFO:  statistics for "test":
storage id: 10000000007
total file size: 32768, total data size: 28
compression rate: 0.64x
total row count: 2, stripe count: 2, average rows per stripe: 1
chunk count: 2, containing data for dropped columns: 0, lz4 compressed: 1, zstd compressed: 1

New stripes are compressed with lz4 again.

jeff-davis commented 2 years ago

This is caused by using a regclass in columnar.options instead of the storage ID.

yxu2162 commented 2 years ago

As discussed with Jeff, the problem occurs because during the ALTER TABLE process, PG creates a new copy of the original table and the copy does not have the same table OID yet so it compresses with its default zstd format instead.