First-class Elasticache support

[BohdanPetryshyn]

Summary

Currently, users must use the custom connection target approach to connect to Elasticache instances. This involves manually setting up connectivity (at init time) and providing Basti with the target's IP and port (at connect time). Let's implement first-class support for Elasticache, just like for RDS. This includes:

1. Detecting Elasticache instances in the user's AWS account and presenting them as options in the connect and init commands
2. Elasticache instance initialization
3. Elasticache instance connection
4. Elasticache-related resources cleanup

TODO

• [x] Redis support
• [x] Memcached support
• [x] Documentation

[yvashkiv]

I would be happy to handle this 😀

[dollev36]

does this issue have an ETA? I can start working on it now if relevant

[BohdanPetryshyn]

@dollev36, I appreciate your willingness to contribute to the project 🙏

As far as I know, @yvashkiv has already started working on the implementation.

If you need to connect to an ElastiCache instance now, you can use the custom target approach. If you're interested in becoming a contributor, I would really appreciate it if you choose any other issue or create a new one ❤️

[dollev36]

@yvashkiv is there an ETA?

[yvashkiv]

@dollev36 Probably it will be done by the end of this week.

[yvashkiv]

Hi @dollev36 . Further research showed that it's hard to solve the problem of local connection for multi-node Redis or Memcached clusters as in this case, the client needs a connection to each of the nodes in the cluster (and it's only part of the problem). For now, I'll implement only Redis (Cluster Mode Disabled) and Memcached single-node clusters support.

Does this satisfy your use case?

[BohdanPetryshyn]

Hey @dollev36 👋

Looks like it takes more time than expected to implement the enhancement. I just want to make sure your use case is covered with the custom target connection mode:

1. Initialize your Basti with basti init, select a custom target and your cluster's VPC
2. Allow connection from the bastion instance to your Elasticache cluster through the cluster's security group
3. Use custom target connection mode to connect to your target:

   basti connect --custom-target-vpc <you-cluster-vpc> \
    --custom-target-host <your-cluster-primary-endpoint> \
    --custom-target-port <your-cluster-port> \
    --local-port <local-port>

[dollev36]

Hi @yvashkiv I suggest you implement this part and I then I could try to expend on it to include cluster mode enabled. can you elaborate on the difficulties you are facing?

[tomer-w]

Hey @BohdanPetryshyn , first, your project is super cool. You provide E2E experience which can really help AWS users. With regards to how I think it should work, I don't think users will want to use the --custom* command line but being able to choose the Redis node from the list of nodes they have access to. Similar to what you already do today with RDS.

[BohdanPetryshyn]

Hi @tomer-w 👋

Thank you for your feedback! It's good to know people value the interactive mode since I mostly use the automatic mode and/or configuration file.

I think it makes sense for Basti to operate Elasticache clusters instead of nodes since this is the first thing people see on the AWS UI. This way, when the user selects a single-shard Redis or Memcached cluster, Basti will establish a connection with the primary node in that single shard. This will eliminate the need for the user to select the primary node from a potentially long list of replicas.

The problem is more complicated with the shared clusters (more on this in the next comment). I hope @dollev36 will help here!

[BohdanPetryshyn]

Regarding sharded clusters.

Both Redis and Memcached leave request routing to the client. This means that the client has to maintain a connection to all the nodes in the cluster (at least all the primary nodes).

Moreover, the client uses the IP:port addresses received from the cluster to communicate with the nodes and follow possible redirects. When connecting through Basti (or any other port forwarding tool like SSH tunneling), the client should use different addresses - the ones exposed by Basti on the localhost. For example, the cluster has three primary nodes:

1. 10.0.1.0:6379
2. 10.0.1.1:6379
3. 10.0.1.2:6379

Using Basti the actual addresses will be something like this:

1. 127.0.0.1:6370
2. 127.0.0.1:6371
3. 127.0.0.1:6372

The client has to have a way to override the addresses. I managed to connect to a shared Redis cluster using the ioredis client for NodeJS and its Nat Mapping feature. However, this is not always possible. For example, the redis-cli client doesn't provide such functionality from what I know.

Possible solutions off the top of my head:

1. Implement connection to all the primary nodes but leave the rest to the user. Basti will make it possible to connect to shared clusters, however, this won't be the easiest thing to do in some cases.
2. Integrate some Redis/Memcached proxies like this one that would encapsulate routing and expose a single entry point. This might be too hard to implement and maintain especially considering that we need two proxies for Redis and Memcached.

I would be really glad to hear your thoughts on this!

cc @dollev36 @tomer-w @yvashkiv

[tomer-w]

I fully agree that if the goal is to allow cluster mode enabled clients to connect using basti than a proxy like mechanism is needed. For me, it was not the first goal of this support as I understand the complexity and don't have easy solution. What I do have in mind is administration support which will allow the customer to use Redis-cli type of tools to monitor and configure the cluster and for this reason it is enough to connect to a single node. Will be glad if later we will come up with novel approach for the cluster connectivity issue but I don't see it as a blocker to start getting value of the Basti tool for ElastiCache.

[BohdanPetryshyn]

Sounds good to me @tomer-w!

So the first iteration of Elasticache support could be:

1. For Redis Cluster Mode Disabled (CMD) - connect to the primary endpoint
2. For Redis CME - connect to the configuration endpoint
3. For Memcached - always connect to the configuration endpoint

The interactive mode CLI should be implemented in such a way as to make it clear to the user that only the configuration endpoint is available for shared clusters

[tomer-w]

I think that as the main usage is going to be configuration and monitoring you will want to be able to choose any node (primary or replica) for both CMD and CME. It should be very similar to the ElastiCache console which you see list of all nodes (with some designation if this is primary or replica and which shard) and allow the user to choose the one he wants to work with. This will be the same for both CME and CMD. So, in the basti init phase you just choose a cluster and in the basti connect phase you can choose the specific node.

[BohdanPetryshyn]

Thank you for your input, @tomer-w! I'll get back with some UX prototypes soon!

[BohdanPetryshyn]

I would suggest an interactive mode layout like this:

1. The user can select a whole cluster or a particular node
2. If a Redis CME cluster is selected, the configuration endpoint is used
3. If a Redis CMD cluster is selected, the primary endpoint is used
4. If a Memcached cluster is selected, the configuration endpoint is used
5. The same approach will be implemented for RDS clusters and instances in the future

[dollev36]

Hey @yvashkiv , have you seen the recent discussion? are you planning on continuing the implementation including the CME? if not , can you tell me aren't you going to do? thanks

[yvashkiv]

Hi @dollev36, I saw the recent discussion and plan to include CME in implementation.

[yvashkiv]

Hi, @dollev36 @tomer-w @BohdanPetryshyn ! Sorry for the delay. I'm going to finish the implementation till the end of this week - beginning of the next week.

[BohdanPetryshyn]

The issue has grown in scope significantly since @yvashkiv started working on it. We agreed with Yura that I'll finish the implementation instead.

Elasticache support should be released in one to two weeks.

[vovagring]

Hi Bohdan, thank you for taking it over from Yura and unblocking him. If you are busy, and no real work is yet done, maybe Dolev can take it, as he is available. Dolev and myself are working in AWS Elasticache service, and we think there is a real value we can provide to many AWS customers. So we are ready to take it over and contribute to your open-source project.

[BohdanPetryshyn]

Hi @vovagring and @dollev36 👋

It would be great if you could handle this! As you guessed, there was not much progress made so far, so you can start from scratch.

This time, I'd suggest agreeing on an ETA from the beginning 😅

I would be happy to give you a quick intro to the project and share some thoughts on the future implementation. We could have a quick call if you want

[dollev36]

Hi @BohdanPetryshyn , I'll be happy to handle this , and of course I'd love to have a chat with you for intro.

[BohdanPetryshyn]

Could you please, message me on LinkedIn so that we can agree on the time?

https://www.linkedin.com/in/bpetryshyn/

[dollev36]

@BohdanPetryshyn I'm sorry for the delay, iv'e missed the notification. added you on LinkedIn

[dollev36]

@BohdanPetryshyn , I appologize for the delay - I've started working on it, but due to personnal issues delaying me it will take about two to three weeks.

[BohdanPetryshyn]

@dollev36, thanks for letting us know!

[BohdanPetryshyn]

Hello, @dollev36 👋

I hope you're doing well! I recently gave this issue a higher priority as I think this functionality could be really helpful to many people. Please, let me know if you need any help! It would also be great if you could give an update on your progress and estimate.

[dollev36]

Hi @BohdanPetryshyn I'm doing all right, sorry for the delay, I estimate I will have a pr by the end of next week.

[BohdanPetryshyn]

Sounds good, thank you for your reply! You're welcome to open a draft PR as soon as it makes sense to start receiving some feedback.

[tomer-w]

@dollev36 , @BohdanPetryshyn , thanks for getting this in. I am sure ElastiCache customers will benefit from this change. When are you going to release it?

[BohdanPetryshyn]

Hi @tomer-w! I plan to add some docs and release the Redis support tomorrow. @dollev36 is going to start working on Memcached support soon.

Are there any channels available at AWS to let the ElastiCache customers know about the new solution?

[tomer-w]

This is what I had in mind as well. I think we can help with spreading the existence of this tool to our customers. This is why I want it to be released so I can start sharing it internally and discuss what we can do externally.

[BohdanPetryshyn]

Cool! I'll let you know when Redis support is released 🤝

[BohdanPetryshyn]

Hey @tomer-w 👋

Elasticache Redis support was released with basti@1.5.0

[BohdanPetryshyn]

With the release of basti@1.6.0, we now support Elasticache Memcached natively 🥳

I'm going to add the CLI options and config file reference to README.md to improve Elasticache support visibility since RDS is used in all the usage examples.

Thank you for your contribution, @dollev36, @tomer-w and @yvashkiv 🤝

@tomer-w, I'd be happy to hear any feedback or feature requests from your internal discussion!

[BohdanPetryshyn]

The reference doc was created and some minor README improvements were added in basti@1.6.1

[BohdanPetryshyn]

At this point, I consider first-class Elasticache support fully implemented.

Please, feel free to open new issues for any bugs or feature requests if needed! Your feedback is highly appreciated!

[BohdanPetryshyn]

Hi @tomer-w! I hope you're doing well!

Do you still plan on implementing this? Is there anything I can help with?

I think we can help with spreading the existence of this tool to our customers. This is why I want it to be released so I can start sharing it internally and discuss what we can do externally.

[tomer-w]

Hi @BohdanPetryshyn, I left Amazon, so I am no longer in the details of this. Please try @vovagring and @dollev36.