[NEW] HTTP support for Valkey

[hpatro]

Subset of Valkey API access via HTTP might be handy for users to try out various features. This might be also helpful to build out browser based Valkey cluster manager for users to administrate and manage their clusters.

[hpatro]

Few of the engine which support HTTP interface

etcd: https://etcd.io/docs/v2.3/api/ opensearch: https://opensearch.org/docs/latest/api-reference/

[zuiderkwast]

Can it be done as a module? Would we need any additional module API functions to make it possible?

[hpatro]

Can it be done as a module? Would we need any additional module API functions to make it possible?

Yeah was thinking on the same line. Trying to get a pulse of the community with the issue.

[zuiderkwast]

Another way to get HTTP could be to run Valkey from Nginx or similar.

[stockholmux]

IIRC, this was done as a module back in the Redis days in a couple of different ways (literal 1 HTTP = 1 Valkey request and then over WebSockets).

IMHO it has a lot of downsides (security, performance) but it is one of those tantalizing ideas that keeps coming up.

[hpatro]

Yeah, a regular HTTP request isn't performant enough, websockets certainly helps with the performance. However, I'm strictly not looking it as a regular interface for application related clients, they can stick to TCP. Rather I'm thinking more in terms of providing administrator certain kind of tooling via browser that would help them avoid ssh-ing into individual nodes and reduce the operational burden of setting things up like CONFIG, ACL, also monitoring active commands, slowlog, analyzing info statistics, accessing logs, etc.

[asafpamzn]

@hpatro , do you think that it can be a client side feature? If yes we can discuss it at https://github.com/valkey-io/valkey-glide. How do you see this application?

[hpatro]

@hpatro , do you think that it can be a client side feature? If yes we can discuss it at https://github.com/valkey-io/valkey-glide. How do you see this application?

I don't see this interface to be a client side feature only. This protocol also needs to be supported on server side. I think a good administrator tooling experience (see the comment above) can be built via web browser on top of Valkey engine in comparison to the existing valkey-cli.

[zuiderkwast]

@asafpamzn Are you thinking of GLIDE as a proxy here?

I think GLIDE is already a protobuffer-to-RESP proxy, so it could do HTTP-to-RESP too?

[PingXie]

@asafpamzn Are you thinking of GLIDE as a proxy here?

slightly off topic - I have been wondering if it makes sense to emulate the standalone interface via GLIDE, i.e., a client side standalone->cluster proxy?

[madolson]

slightly off topic - I have been wondering if it makes sense to emulate the standalone interface via GLIDE, i.e., a client side standalone->cluster proxy?

Yeah, it's fairly trivial and many clients do it. The problem is that there is a lot of risk for decomposing cross-slot operations, because if you expect it to be atomic and it is not. Some clients for example break up MGET into a bunch of sub commands.

I think GLIDE is already a protobuffer-to-RESP proxy, so it could do HTTP-to-RESP too?

But why? I guess I don't really get the feature in any case, I feel like we should solidify that before moving it to glide.

[asafpamzn]

I agree with:

I feel like we should solidify that before moving it to glide.

Regarding:

slightly off topic - I have been wondering if it makes sense to emulate the standalone interface via GLIDE, i.e., a client side standalone->cluster proxy?

If you have a client that support Redis cluster why would you use the standalone interface? Unless you want to hide GLIDE behind interface of existing client. It is doable but we will find out that the devil is in the details and it is going to be a huge effort.