Question: Sticky session in routes with load balancer

Fetsivalen commented 5 years ago

Question/Enhancement: Is there any plans to support sticky session for the load-balanced routes?

Currently, I found out the way how to do it via override LoadBalancerClientFilter and create custom "sticky" ribbon rule.

But I wonder maybe there are any plans to do it "out of the box"?

spencergibb commented 5 years ago

We have no plans for this

vibhaG commented 4 years ago

+1 - this would be a useful enhancement

xinghen110 commented 4 years ago

+1 - especially in enterprise development

xinghen110 commented 4 years ago

Question/Enhancement: Is there any plans to support sticky session for the load-balanced routes?

Currently, I found out the way how to do it via override LoadBalancerClientFilter and create custom "sticky" ribbon rule.

But I wonder maybe there are any plans to do it "out of the box"?

How to create Ribbon rule, could you share your code ? thinks!

Fetsivalen commented 4 years ago

@xinghen110 For Ribbon to make it work you should do a few things

Override LoadBalancerClientFilter to add code like:

@Override
public ServiceInstance choose(ServerWebExchange exchange) {
    String host = ((URI) exchange.getAttribute(GATEWAY_REQUEST_URL_ATTR)).getHost();
    return this.loadBalancer.choose(host, allSourcesMultiValueMapFrom(exchange));
}

private MultiValueMap<String, String> allSourcesMultiValueMapFrom(ServerWebExchange exchange) {
//choose only params you need, just sample code to illustrate a few options
    ServerHttpRequest request = exchange.getRequest();
    MultiValueMap<String, String> multiValueMap = new LinkedMultiValueMap<>();
    multiValueMap.addAll(request.getHeaders());
    multiValueMap.addAll(request.getQueryParams());
    request.getCookies().entrySet()
            .forEach(entry -> multiValueMap.add(entry.getKey(),entry.getValue().get(0).getName());
    return multiValueMap;
}

Implement Ribbon Rule

public class StickySessionRule extends RoundRobinRule {

@Override
public void initWithNiwsConfig(IClientConfig clientConfig) {
}

@Override
public Server choose(Object routingOptions) {
    validateRoutingOptions(routingOptions);
    String instanceName = getInstanceName((MultiValueMap<String, String>) routingOptions);
    return hasInstanceName(instanceName)
            ? getStickyServer(instanceName)
            : super.choose(routingOptions);
}
//specific logic is omitted
}

for each sticky service add property

service-with-sticky.ribbon.NFLoadBalancerRuleClassName={ClassPackage}.StickySessionRule

Anyway I think this ribbon related login won't ever appear in this repo since it will be superseded by https://github.com/spring-cloud/spring-cloud-commons/issues/689

Fetsivalen commented 4 years ago

@spencergibb I've played around new ReactiveLoadBalancerClientFilter and found you that for now, it disallows to use custom load balancing strategies per-route since it is not by-pass request data to the loadBalancer.choose method https://github.com/spring-cloud/spring-cloud-gateway/blob/master/spring-cloud-gateway-core/src/main/java/org/springframework/cloud/gateway/filter/ReactiveLoadBalancerClientFilter.java#L132 would you accept PR to by-pass ServerHttpRequest in the Request context (or even ServerWebExchange, since I'm not sure about other use-cases in the community )? or as an alternative (probably preferable) rework

private Request createRequest() {
        return ReactiveLoadBalancer.REQUEST;
    }

to

// protected here instead of private to give an ability to customize context bypassing
protected Request createRequest(ServerWebExchange exchange) {
        return ReactiveLoadBalancer.REQUEST;
    }

or there plans to bypass some custom context once features like https://github.com/spring-cloud/spring-cloud-commons/issues/689 will be done?

spencergibb commented 4 years ago

We're planning on supporting passing a context especially since a server request makes a lot of sense to make decisions during load balancing

OlgaMaciaszek commented 4 years ago

We'll be adding a sticky implementation for the new LoadBalancer: https://github.com/spring-cloud/spring-cloud-commons/issues/689

neelimaj97 commented 4 years ago

@Fetsivalen

Could you provide the entire code of how you implemented sticky session rule with implementation for the functions you have mentioned as well like getStickyServer. Please do help me out

Fetsivalen commented 4 years ago

@neelimaj97 sorry, but no. In comment https://github.com/spring-cloud/spring-cloud-gateway/issues/1176#issuecomment-612791740 I put all except logic specific to my case, which is definitelly shouldn't be here.

neelimaj97 commented 4 years ago

@Fetsivalen I am not able to override LoadBalancerClientFilter in spring-cloud-gateway 2.2.2-RELEASE. I tried overriding choose method of LoadBalancerClientFilter,but then again, instance of that filter is created and only the choose method of LoadBalancerClientFilter is getting used. Are you saying i should create a post filter same as LoadBalncerClient with a different choose logic? How did you make the gateway use the choose method you had overridden rather than the one written in LoadBalancerClientFilter. Please help

tengcomplex commented 2 years ago

Goal

HA microservice setup, with 2 gateways, 2 frontend-services and of course a couple of backend services behind them.

gs_ss_setup

Every user request can be routed through a random gateway to one of the frontend instances. Upon first request a frontend instance is picked, following requests should end up at the same frontend instance for session reasons.

Solution

Following the suggestion in this issue, especially @Fetsivalen spring-cloud-gateway comment in #1176 and the PR of @fitzoh in spring-cloud-commons #764 I was able to implement a load-balanced sticky session based on Spring Boot 2.5.4, Spring Cloud 2020.0.3, Spring Cloud Gateway 3.0.3. Information to which frontend instance subsequent requests are routed to are stored in a cookie.

See exemplary implementation.

Basic logic upon every request:

GlobalFilter ReactiveLoadBalancerStickySessionFilter. Chooses ServiceInstance from LoadBalancer.
- If available take ServiceInstance from cookie, otherwise choose one.
Enrich cookie with ServiceInstance id.

However, this smells a bit clumsy with quite some custom code. After seeing @OlgaMaciaszek mention of spring-cloud-commons issue #689 and the merge of spring-cloud-commons #862 I'm beginning to suspect this approach might be doable with more "in-house" usage of spring classes. Specifically with SameInstancePreferenceServiceInstanceListSupplier but I currently don't really see how to plug it together.

Questions

Is what I did a "valid" approach in general?
Is @spencergibb comment final, there won't be an implementation in spring-cloud-gateway?
- If not, why is this issue still open?
- Does spring-cloud-commons #689 fully answer spring-cloud-gateway #1176?
Is there an easier solution in sight with less custom code, maybe by using the spring-cloud commons classes?
- If yes, any hints or examples how to apply?
Did I miss something totally obvious?

Any thoughts in this regard appreciated.

spring-cloud / spring-cloud-gateway