search

awspring / spring-cloud-aws

The New Home for Spring Cloud AWS
http://awspring.io
Apache License 2.0
892 stars 303 forks source link

RDS Cluster Support #50

Closed maciejwalkowiak closed 2 years ago

maciejwalkowiak commented 3 years ago

From spring-cloud-aws created by davidsloan-productlab: spring-cloud/spring-cloud-aws#356

Instead of an RDS instance I'm trying to use an RDS Aurora cluster.

I can use an instance within the cluster using @EnableRdsInstance

However when trying to use the cluster itself I see an exception with this as the cause: No database instance with id:'my-cluster-name' found. Please specify a valid db instance

Looking into the code, I can see that in AmazonRdsDataSourceFactoryBean, only instances are queried using AmazonRDS.describeDBInstances

It would be good to either amend this to support a cluster, or introduce a new annotation @EnableRdsCluster

What do you think the chances are for RDS cluster support in Spring Cloud AWS?

maciejwalkowiak commented 3 years ago

Much needed one..

People are now going for serverless RDS, so cluster connection establishment is really required.

maciejwalkowiak commented 3 years ago

Vlad's post is interesting and certainly helps for day-to-day running. But it's of little help to Spring projects that are looking to reap the benefits of Aurora clusters for failover. The current provision of a retry interceptor isn't really good enough. Services should be able to connect to a cluster, the SDK should be aware of which instances are write and read-only, and should direct requests accordingly.

Roll on 3.0.0.M1. Hopefully that addresses the failover issue.

maciejwalkowiak commented 3 years ago

@aravindparappil46 currently you have to do it manually - look at Vlad's post for inspiration: https://vladmihalcea.com/read-write-read-only-transaction-routing-spring/

maciejwalkowiak commented 3 years ago

This has also been reported here: This issue has also been reported here: https://github.com/spring-cloud/spring-cloud-aws/issues/302

maciejwalkowiak commented 3 years ago

I've done it for now using reader/writer datasources + AOP . basic idea is here: https://fable.sh/blog/splitting-read-and-write-operations-in-spring-boot/

Looking forward to 3.0.0.M1 milestone!

maciejwalkowiak commented 3 years ago

Do we know if this is on the road map at all?

maciejwalkowiak commented 3 years ago

Looking forward to the 3.0.0.M1 milestone! Definitely need this feature to be able to connect to an Aurora RDS Cluster (through annotation and through the application.properties file) for Spring & Spring Boot projects.

Meanwhile, if anyone knows a workaround for connecting to an Aurora cluster in Spring Boot and enabling read replica support, please post here!

maciejwalkowiak commented 3 years ago

unable to connect to an Aurora RDS Cluster with read replica (through annotation and through the application.properties file) for spring boot project using spring cloud Any news Please !

dlamblin commented 3 years ago

This is unexpected and leads an RDS Aurora MySQL user to consider just preferring a jdbc url to the cluster, which works normally, finds the writeable instance and handles the read/write failover afaiks; unlike the specific RDS support provided here.

BerndFarkaDyna commented 3 years ago

@maciejwalkowiak:

is there any update on this issue:

for me currently it looks like this is severe blocker using RDS in combination with spring boot:

the functionality to utilize the read replicas is simple broken in combination with spring-cloud-aws-jdbc?

or am I missing something?

Lucas3oo commented 3 years ago

Also most people want to select the connection pool themselves. Spring cloud AWS kind of hard code it to Tomcat's connection pool.

maciejwalkowiak commented 3 years ago

@dlamblin for the failover scenario, I would also recommend just using plain JDBC url. Read replica support is the interesting feature of Spring Cloud AWS.

@BerndFarkaDyna read replicas do work, just not with Aurora.

@Lucas3oo very much agreed.

fletchgqc commented 3 years ago

I'm trying to switch from RDS Postgresql to Aurora Serverless and I found that it doesn't work with Spring Cloud AWS.

Am I correct in surmising that the reason it doesn't work is that Aurora Serverless only provides a cluster (instead of instance) URL? And therefore, in order to use Aurora Serverless, this issue needs to be fixed?

If so, I think that as a very first step, this should be addressed in the documentation. Perhaps where it says "The minimum configuration parameters are a unique id for the data source, a valid db-instance-identifier attribute that points to a valid Amazon RDS database instance." we should add "Aurora Serverless is currently not supported, because it provides a cluster URL and Spring Cloud AWS does not yet support cluster endpoints."

maciejwalkowiak commented 3 years ago

@fletchgqc both traditional Aurora and Aurora Serverless aren't supported.

fletchgqc commented 3 years ago

@maciejwalkowiak Ahh OK, thanks for the info. Maybe that should be mentioned in the docs? Because Aurora is a part of RDS, and RDS is partly supported.

maciejwalkowiak commented 3 years ago

@fletchgqc you're right https://github.com/awspring/spring-cloud-aws/issues/172

Lucas3oo commented 3 years ago

To add some info in general: You can setup Aurora as a cluster with only one instance in it. Then the "endpoint" will be the "writer" endpoint, But the specific "reader" endpoint will in fact be the same hostname in case of a one instance cluster. So it is perfectly OK to use an Aurora cluster like that from Spring. But Since the failover is done using DNS names you should set the DNS cache TTL in JRE to absolute minimum. Default is cache forever.

sr-nu commented 3 years ago

Yes having seamless Reader Writer support for Aurora similar to RDS is really important. Please prioritize or provide a workaround.

nickcaballero commented 3 years ago

This kind of works out of the box if you use the MariaDB driver in Aurora mode. When using a readOnly transaction, the driver will choose one of the reader connections instead of using the writer.

https://mariadb.com/kb/en/failover-and-high-availability-with-mariadb-connector-j/#masterslave-connection-selection

sr-nu commented 3 years ago

Not working for Postgres as expected. All queries directed to Writer.

On Fri, 1 Oct 2021, 01:45 Nick Caballero, @.***> wrote:

This kind of works out of the box if you use the MariaDB driver in Aurora mode. When using a readOnly transaction, the driver will choose one of the reader connections instead of using the writer.

https://mariadb.com/kb/en/failover-and-high-availability-with-mariadb-connector-j/#masterslave-connection-selection

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/awspring/spring-cloud-aws/issues/50#issuecomment-931532800, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA75PXZIVPCZLFNAG65P2LUESO2DANCNFSM4XLLFDNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

hemju commented 2 years ago

+1 to the feature request

maciejwalkowiak commented 2 years ago

Replaced by #322