search

omgnetwork / omg-childchain-v2

pronounced /Ch-ch/
Apache License 2.0
5 stars 2 forks source link

postgres table sharding POC #150

Open InoMurko opened 3 years ago

InoMurko commented 3 years ago

https://pgdash.io/blog/postgres-11-sharding.html

https://www.highgo.ca/2020/03/12/horizontal-scalability-sharding-in-postgresql-core-missing-pieces-of-the-puzzle/

https://wiki.postgresql.org/wiki/Built-in_Sharding

InoMurko commented 3 years ago

So the goal of this exercise is to see if PG internal sharding (partitioning) is helpful in splitting up the transaction table. We would like to se:

  1. local docker compose setup
  2. demonstrable test with increased performance over the current
  3. estimation of any necessary code changes and it's implications
InoMurko commented 3 years ago

The table that gets hit most and we would like to see it being more performant is Engine.DB.Output and perhaps also Engine.DB.Transaction. But at least for the Engine.DB.Transaction we have plans to prune it if it's going to be doable.

So these two tables should be your focus!

cc @pgebal @mederic-p if you have any more inputs since you've been doing some performance measurements, please leave them here for @DanielaIvanova !