LemmyNet / lemmy

🐀 A link aggregator and forum for the fediverse
https://join-lemmy.org
GNU Affero General Public License v3.0
13.18k stars 871 forks source link

Store aggregates without separate tables (e.g. combine `post` and `post_aggregates`) #4869

Open dullbananas opened 3 months ago

dullbananas commented 3 months ago

This would get rid of a lot of complexity. It would also reduce the chance of things that have been big problems before: trigger bugs, and queries that can't efficiently use indexes.

Diesel's 32 column limit makes this hard. The first thing that should be done is to create a crate with just the schema.rs file, enable the 64 columns feature, apply the changes in schema.rs, and measure the incremental compile time difference using cargo timings. One of these things about the compile time increase caused by the 64 columns feature might be revealed:

dessalines commented 2 months ago

I wouldn't be opposed, but yeah the increase in compile time would be an issue.

You could test the difference by just adding that diesel feature. I believe it'll increase the compile time even if no table has past 32 columns.

Nutomic commented 2 months ago

Yes the compile time increases directly after enabling the 64 column feature, no matter if you use them or not. Its because diesel has to generate a lot more code (macros?) to handle these extra columns. So to test the compile time difference, simply add 64 column feature to Lemmy's cargo.toml.

Nutomic commented 1 week ago

Tested this, the increase is rather minor (2m 41s to 3m 03s with 64 column feature enabled). So we can do this if it gives the advantages you mention.

dessalines commented 5 days ago

Nice, in that case I wouldn't be opposed to get rid of all the one-to-one aggregates tables.