search

sipcapture / homer-app

HOMER 7.x Front-End and API Server
http://sipcapture.io
GNU Affero General Public License v3.0
204 stars 79 forks source link

Increase databases capacity #407

Closed gledim closed 3 years ago

gledim commented 3 years ago

Hi team,

This is not actually an issue, but more of a kind request of thoughts and suggestions from your side.

Currently we are successfully using 3 PostgreS databases in parallel, but we further need to increase the capacity, due to the huge amount of data.

Instead of scaling up the current hosts, which would be a short term solution, as they are quite powerful, we are thinking of expanding the number of databases.

Would that be viable with the homer-app? Is there any better practice that you suggest for storing large amounts of data?

Thank you

adubovikov commented 3 years ago

Hi @gledim,

so, you can do it vertical or horizontal: create multiple DB and use it across one postfgress or create multiple postgress instances and (cluster/shard it). Homer-app can make connection to multipe sources and agregate data by multiple select.

My recommendation: make something like sharding (user id) and split data between multiple tables/hosts.

Regards, Alexandr

gledim commented 3 years ago

Thank you for the suggestions @adubovikov For the moment we did both a vertical and horizontal scale. We increased the number of nodes from 3 to 6, and upgraded the storage capacity. I think the next step is sharding :)