search

scylladb / alternator-load-balancing

Various tricks, scripts, and libraries, for load balancing multiple Alternator nodes
Apache License 2.0
18 stars 11 forks source link

support for dynamodbv2 classes #5

Closed fruch closed 3 years ago

fruch commented 4 years ago

I trying to work on the alternator streams example, and use this client side load balancing.

but seems like it's doesn't fit the AmazonDynamoDBClientBuilder which is part of dynamodbv2, that our example is using: https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-dynamodb/src/main/java/com/amazonaws/services/dynamodbv2/AmazonDynamoDBClientBuilder.java

the current AlternatorClient example doesn't exactly fit

nyh commented 4 years ago

@psarna you did the Java version, do you know what he is asking about? I have a vague recollection that we discussed there being some v1 and v2 variants of the Java API and that you supported v2? But I really don't remember anything, I hope you do... @fruch thanks for testing this code!

psarna commented 4 years ago

There was a java and javav2 versions of the driver I had to pick from and I picked javav2, but I have no clue what it is, and I also don't know what dynamodbv2 is...

edit: Ok, after a little investigation it looks like there exists:

... and from these two, I based my wrapper on aws-sdk-java-v2. I assume it's newer, since github history shows more activity on this repository. Could you try to create an example on this second sdk?

nyh commented 4 years ago

Wow, what a mess Amazon made :-) I also remember now looking at examples from Amazon's website and running into a confusing mess of examples that didn't work because I tried v1 examples on v2 jar, or vice versa...

I wonder if there is some text we could add to java/README.md to make this clearer that we only support aws-sdk-java-v2 currently. And @fruch if you think this is a mistake, please let us know - I think we chose to support v2 and not v1 mostly because it seemed newer - not because we had any definite knowledge that most users would be using it.

nyh commented 3 years ago

To complicate things even more, there is now an "enhanced" v2, which is an enhancement of v2 with features from v1: https://aws.amazon.com/blogs/developer/introducing-enhanced-dynamodb-client-in-the-aws-sdk-for-java-v2/ We'll need to check all three variants :-( I'm now starting to look into it.

nyh commented 3 years ago

I'm trying to clean up our Java library, and find all sorts of problems, even with the v2 API. For example, if I try to use it from a test not in the same package, I get

com.scylladb.alternator.AlternatorClient is not public in com.scylladb.alternator; cannot be accessed from outside

I'm trying now to fix this and other problems, and trying to create demos for both v2 and v1 APIs.

psarna commented 3 years ago

Sure, if you have any questions along the way then I'll try to help, but I have limited knowledge of Java anyway, so you'll probably be better off without that kind of "help" :)

nyh commented 3 years ago

@psarna this is a fun exercise, because on one hand you made some small mistakes that makes me wonder how your demo ever worked (e.g., one place had "8000" hard-coded, so how did HTTPS work?) but on the other hand, you solved some difficult problems which saves me a lot of effort to redo what you already did it - so thanks. If I get v1 to work too, I'll try to clean things and send a patch. Eventually I'll have to figure out how to post our Jar on Maven. Unfortunately, when I worked heavily on Java (until 10 years ago) Maven wasn't "the thing" so I have zero experience with it :-(

psarna commented 3 years ago

@avelanarius it's a shot in the dark, but perhaps you have some maven expertise to lend, if need be?

nyh commented 3 years ago

The good news is that I have load balancing with AWS Java SDK v1 working. The even better news is that it is significantly simpler than the existing implementation for v2 - instead of creating a new HTTP client implementation, I used an existing mechanism that can modify requests - https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/handlers/RequestHandler2.html - and it seems I can modify the request's destination at a good time - before it is signed. Both HTTPS and request signature checking seems to be working well.

However, there is one difference - and potentially a big problem - in my new approach. When the load balancer decides to send a request is sent to https://127.0.0.3:8043 it uses this URL, not the original url (like https://dog.scylla.com:8043) in the request. This not great for SSL - it means each of the nodes will need a different SSL certificate. I'm still thinking how to fix that :-(

nyh commented 3 years ago

According to @fruch the traditional setup of SSL in Cassandra (see https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/configuration/secureSSLCertWithCA.html) is that the different nodes have a certificate for their IP address (not domain name), and all are self-signed by a common CA which the client is taught about. We could do exactly the same for Alternator, and in this case my v1 solution which I described above - which does SSL requests to IP addresses like https://127.0.0.3:8043 - will work correctly.

If we can agree that this is the behavior we want, I should document it, and maybe we should change the other load balacing implementations to also do it this way (I don't know how easy it will be...) because it will be strange to have different behavior for different load balancers...

nyh commented 3 years ago

Created pull request https://github.com/scylladb/alternator-load-balancing/pull/6 that adds support to AWS SDK for Java v1 (which confusingly has classes called com.amazonaws.services.dynamodbv2...).

nyh commented 3 years ago

Fixed by pull request #6. I guess I forgot to mark it with Fixes this issue.