search

citusdata / citus

Distributed PostgreSQL as an extension
https://www.citusdata.com
GNU Affero General Public License v3.0
10.18k stars 652 forks source link

Support more compression methods for columnar (e.g. lz4hc) #6626

Open fingoldo opened 1 year ago

fingoldo commented 1 year ago

Columnar storage page lists a lot of format limitations, including

No support for foreign keys, unique constraints, or exclusion constraints

But Citus updates post claims:

With the index support on columnar tables, Citus 10.2 supports the following constraints on columnar tables: primary keys, unique constraints, and exclude constraints.

This was a deciding factor for me and I almost declined Citus option when making a decision. If it's indeed supported, can someone fix the readme inaccuracy, please.

fingoldo commented 1 year ago

Besides, I get "unknown compression type for columnar table: lz4hc" on Ubuntu 20.04 LTS. Is the -hc version deprecated at all? https://github.com/lz4/lz4/issues/660

onurctirtir commented 1 year ago

Columnar storage page lists a lot of format limitations, including

No support for foreign keys, unique constraints, or exclusion constraints

But Citus updates post claims:

With the index support on columnar tables, Citus 10.2 supports the following constraints on columnar tables: primary keys, unique constraints, and exclude constraints.

This was a deciding factor for me and I almost declined Citus option when making a decision. If it's indeed supported, can someone fix the readme inaccuracy, please.

Thank you for reporting this and contributions are always welcome! :)

Besides, I get "unknown compression type for columnar table: lz4hc" on Ubuntu 20.04 LTS. Is the -hc version deprecated at all? lz4/lz4#660

This is because Citus columnar only supports pglz, lz4 and zstd atm; does any of those fit in your use-case ?

fingoldo commented 1 year ago

I'm too unexperienced with Citus yet to contribute to the docs :-) Sure, pglz, lz4 and zstd will work, but I was just testing all options on my particular dataset to compare by size & access speed, and lz4hc was advertised everywhere in the docs, what's why I was surprised by the error.

onurctirtir commented 1 year ago

Then keeping this issue open as a feature request.

mortgagemetrix commented 5 months ago

If you don't support lz4hc can you at least remove the option from the docs?