goharbor / harbor

An open source trusted cloud native registry project that stores, signs, and scans content.
https://goharbor.io
Apache License 2.0
24.05k stars 4.75k forks source link

Connecting harbor to postgres via pgpool. #19942

Closed snoop2048 closed 4 months ago

snoop2048 commented 9 months ago

We have built harbor in a k8s cluster with external postgres, s3.

However there appears to be no way to connect harbor to pgpool to use the HA of the postgres cluster. Can harbor only take a single IP for postgres connection? psql uses 2, so pgpool ensures the active DB is used. If we can only use a single IP must we manually repoint harbor if the DB switches its active node?

Ta

MinerYang commented 9 months ago

Hi @snoop2048 ,

Thanks for connecting with us, and yes, we are currently only support single hostname for Postgres server. If we can only use a single IP must we manually repoint harbor if the DB switches its active node?

I am not recommend to do so even you have tools to handle this 'ip' changes. Since there's might be some concerned with the data inconsistent during the transit.

But if you have handy experience on this, welcome to sharing with the community!

snoop2048 commented 9 months ago

So harbor does not support any sort of HA Postgres backend?

MinerYang commented 9 months ago

Not exactly, I will take a look later

MinerYang commented 9 months ago

I am not sure how your pgpool is configured. But theoretically, if you set up a load balancer in front of your pg service, I think one single hostname for pg is enough. And you don't need to worry about fail over handle and synchronization of the db.

snoop2048 commented 9 months ago

Hey, thanks for coming back to me. pgpool negates the need to an external LB for the postgres cluster as it handles loadbalancing itself (amongst other things). Adding another LB (HAproxy or whatever) in front of is probably going to just cause issues. Admittedly getting slightly out of my depth here given I am not a DB engineer but I will try and get one to comment.

MinerYang commented 9 months ago

Thanks @snoop2048 ,

I am not familiar with the pgpool usage as well. All this LB things is just based on my assumption. So If you have any progress or experience here, please share with users who have the same doubt. I will keep this issue open for further discussion and references.

Best, Miner

Kajot-dev commented 8 months ago

In terms of HA, I'm running Postgresql in HA with Patroni (using k8s operator) with no issues for the past year

prima101112 commented 7 months ago

@snoop2048

Regarding High Availability (HA), pgpool appears to function similarly to other PostgreSQL balancers such as pgbouncer. It's recommended to deploy a standalone pgpool for managing multiple databases. Clients can then connect to a single pgpool instance, which distributes the load across all databases it serves.

You might find valuable insights in this discussion: Postgres with pgpool Architecture.

Thanks.

github-actions[bot] commented 5 months ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

github-actions[bot] commented 4 months ago

This issue was closed because it has been stalled for 30 days with no activity. If this issue is still relevant, please re-open a new issue.