Horizontal Scaling Scripts

Samagra-Development / WarpSQL

WarpSQL

https://samagra-development.github.io/WarpSQL/

Apache License 2.0

15 stars 30 forks source link

Horizontal Scaling Scripts #49

Open singhalkarun opened 1 year ago

singhalkarun commented 1 year ago

As a part of deployment config, user should be able to specify number of read replicas they want and requests should be automatically load balanced over read repicas.

User, at any point, should also be able to update number of read replicas and accordingy system should adjust.

Future Scopes:

Dynamic Scaling Configuration, where in user can specify minimum and maximum replicas and system should auto scale.

Reference: https://severalnines.com/blog/comparing-load-balancers-postgresql/

PiyushRaj927 commented 1 year ago

Hello @singhalkarun, can you provide further information regarding this issue. It would be helpful to know the specific orchestration we are targeting and any crucial project requirements that need to be addressed. Thank you!

singhalkarun commented 1 year ago

Hello @singhalkarun, can you provide further information regarding this issue. It would be helpful to know the specific orchestration we are targeting and any crucial project requirements that need to be addressed. Thank you!

Hi @PiyushRaj927 we are specifically targeting to scale read replicas horizontally as a first step where in user should be able to specify number of read replicas in the config itself and the requests should be automatically load balanced over those read replicas.

srishtiv424 commented 1 year ago

Hello @singhalkarun which orchestration tool are we using and can you provide some more details regarding this .

thanks. looking forward

singhalkarun commented 1 year ago

Hello @singhalkarun which orchestration tool are we using and can you provide some more details regarding this .

thanks. looking forward

Hey @srishtiv424 we are open in terms of Orchestration tool, maybe we can start with Docker Swarm and move to Kubernetes in future.

jayanth-kumar-morem commented 1 year ago

Hey @singhalkarun , I can take this up. Shall we go ahead using Pgpool-II for configuring load balancing ?

singhalkarun commented 1 year ago

Hey @singhalkarun , I can take this up. Shall we go ahead using Pgpool-II for configuring load balancing ?

Hey @jayanth-kumar-morem let's do a bit of research on other available solutions available as well and create a comparasion with pros and cons of each and then let's decide on which one to use.

Reference: https://severalnines.com/blog/comparing-load-balancers-postgresql/

jayanth-kumar-morem commented 1 year ago

@singhalkarun Check out the Comparison matrix here https://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling

c4gt-community-support[bot] commented 10 months ago

Hi! Mandatory Details - The following details essential to submit tickets to C4GT Community Program are missing. Please add them!

Product Name - Please add a heading called Product Name and mention the name of the product below it.
Tech Skills Needed - Please add relevant tech skills
Mentors(s) Please tag the relevant mentors on the ticket
Complexity - Please mention the complexity only as High/Medium/Low
Category - Please add one or more of these options [CI/CD], [Integrations], [Performance Improvement], [Security], [UI/UX/Design], [Bug], [Feature], [Documentation], [Deployment], [Test], [PoC]

Without these details, the ticket cannot be listed on the C4GT Community Listing.

Important Details - These following details are helpful for contributors to effectively identify and contribute to tickets.

Organization Name
Domain - Area of governance
Sub-Category - Please mention the sub-category if any for the ticket

Please update the ticket