apache / linkis

Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.
https://linkis.apache.org/
Apache License 2.0
3.3k stars 1.17k forks source link

[bug] linkis-gateway router cannot find eureka service instance when prefer-ip-address=true #564

Closed geosmart closed 3 years ago

geosmart commented 3 years ago

I deploy linkis in docker and network mode is host, so eureka set prefer-ip-address=true, but when I kill a running sql in dss, linkis gateway throws a Unable to find instance error.

by debug spring gateway filter chais, I finally find that the ip address make the route error!

issue details

gateway-log

2021-02-23 15:32:43.716 INFO  [reactor-http-nio-3] com.webank.wedatasphere.linkis.gateway.route.DefaultGatewayRouter 42 info - 
GatewayRouter route requestUri /api/rest_j/v1/entrance/030518IDEspark10.199.155.80:9106IDE_jrsyb_ops_91/kill with parsedService spark to ServiceInstance(sparkentrance, 10.199.155.80:9106)
2021-02-23 15:32:43.717 ERROR [reactor-http-nio-3] org.springframework.boot.autoconfigure.web.reactive.error.DefaultErrorWebExceptionHandler 218 log - 
Failed to handle request [GET http://10.199.155.82/api/rest_j/v1/entrance/030518IDEspark10.199.155.80:9106IDE_jrsyb_ops_91/kill] 
org.springframework.cloud.gateway.support.NotFoundException: Unable to find instance for 10.199.155.82
    at org.springframework.cloud.gateway.filter.LoadBalancerClientFilter.filter(LoadBalancerClientFilter.java:72) ~[spring-cloud-gateway-core-2.0.0.RELEASE.jar:2.0.0.RELEASE]
    at org.springframework.cloud.gateway.handler.FilteringWebHandler$GatewayFilterAdapter.filter(FilteringWebHandler.java:133) ~[spring-cloud-gateway-core-2.0.0.RELEASE.jar:2.0.0.RELEASE]
    at org.springframework.cloud.gateway.filter.OrderedGatewayFilter.filter(OrderedGatewayFilter.java:44) ~[spring-cloud-gateway-core-2.0.0.RELEASE.jar:2.0.0.RELEASE]
    at org.springframework.cloud.gateway.handler.FilteringWebHandler$DefaultGatewayFilterChain.lambda$filter$0(FilteringWebHandler.java:115) ~[spring-cloud-gateway-core-2.0.0.RELEASE.jar:2.0.0.RELEASE]

request

curl --cookie "bdp-user-ticket-id=TnVvmZr0sa2CXrw9u+x=" http://10.199.155.82:8088/api/rest_j/v1/entrance/030518IDEspark10.199.155.80:9106IDE_geosmart_62/kill

response

{
    "timestamp": "2021-02-23T07:36:27.715+0000",
    "path": "/api/rest_j/v1/entrance/030518IDEspark10.199.155.80:9106IDE_geosmart_62/kill",
    "status": 500,
    "error": "Internal Server Error",
    "message": "Unable to find instance for 10.199.155.82"
}

sparkentrance eureka config

eureka:
  client:
    serviceUrl:
      defaultZone: http://10.199.155.82:20303/eureka/
  instance:
    prefer-ip-address: true
    instance-id: 10.199.155.80:sparkEntrance:9106
    ip-address: 10.199.155.80
    metadata-map:
      test: enjoyyin

why

in LoadBalancerClientFilter.filter,instance==null make it throw NotFoundException


    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        URI url = exchange.getAttribute(GATEWAY_REQUEST_URL_ATTR);
        String schemePrefix = exchange.getAttribute(GATEWAY_SCHEME_PREFIX_ATTR);
        if (url == null || (!"lb".equals(url.getScheme()) && !"lb".equals(schemePrefix))) {
            return chain.filter(exchange);
        }
        //preserve the original url
        addOriginalRequestUrl(exchange, url);

        log.trace("LoadBalancerClientFilter url before: " + url);

        final ServiceInstance instance = loadBalancer.choose(url.getHost());
    // debug instance=null
        if (instance == null) {
            throw new NotFoundException("Unable to find instance for " + url.getHost());
        }

        URI uri = exchange.getRequest().getURI();

        // if the `lb:<scheme>` mechanism was used, use `<scheme>` as the default,
        // if the loadbalancer doesn't provide one.
        String overrideScheme = null;
        if (schemePrefix != null) {
            overrideScheme = url.getScheme();
        }

        URI requestUrl = loadBalancer.reconstructURI(new DelegatingServiceInstance(instance, overrideScheme), uri);

        log.trace("LoadBalancerClientFilter url chosen: " + requestUrl);
        exchange.getAttributes().put(GATEWAY_REQUEST_URL_ATTR, requestUrl);
        return chain.filter(exchange);
    }

and the deeper reason is RouteToRequestUrlFilter.filter can not replace the lb host from http://10.199.155.82/api/rest_j/v1/entrance/030518IDEspark10.199.155.80:9106IDE_geosmart_62/kill to lb://merge-gw-13sparkentrance10.199.155.80---9106

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        Route route = exchange.getAttribute(GATEWAY_ROUTE_ATTR);
        if (route == null) {
            return chain.filter(exchange);
        }
        log.trace("RouteToRequestUrlFilter start");
        URI uri = exchange.getRequest().getURI();
        boolean encoded = containsEncodedParts(uri);
        URI routeUri = route.getUri();

        if (hasAnotherScheme(routeUri)) {
            // this is a special url, save scheme to special attribute
            // replace routeUri with schemeSpecificPart
            exchange.getAttributes().put(GATEWAY_SCHEME_PREFIX_ATTR, routeUri.getScheme());
            routeUri = URI.create(routeUri.getSchemeSpecificPart());
        }
        // fixme 此处导致的url问题,不能正常替换路由地址
        URI requestUrl = UriComponentsBuilder.fromUri(uri)
                .uri(routeUri)
                .build(encoded)
                .toUri();
        exchange.getAttributes().put(GATEWAY_REQUEST_URL_ATTR, requestUrl);
        return chain.filter(exchange);
    }

testcase

a test case for RouteToRequestUrlFilter.filter

package com.servyou.dboard.service;

import java.net.URI;
import java.net.URISyntaxException;

import org.springframework.web.util.UriComponentsBuilder;

public class UriTest {
    public static void main(String[] args) throws URISyntaxException {
        //normal
        formatUri(
                "http://10.199.155.82/api/rest_j/v1/jobhistory/1103/get",
                "lb://cloud-publicservice"
        );
        //error
        formatUri(
                "http://10.199.155.82/api/rest_j/v1/entrance/030518IDEspark10.199.155.80:9106IDE_geosmart_62/kill",
                "lb://merge-gw-13sparkentrance10.199.155.80---9106"
        );

        //correct
        formatUri(
                "http://10.199.155.82/api/rest_j/v1/entrance/030518IDEspark10.199.155.80:9106IDE_geosmart_62/kill",
                "lb://merge-gw-13sparkentrance10-199-155-80---9106"
        );
    }

    public static URI formatUri(String url1, String url2) throws URISyntaxException {
        URI uri = new URI(url1);
        URI routeUri = new URI(url2);
        URI requestUrl = UriComponentsBuilder.fromUri(uri)
                .uri(routeUri)
                .build(true)
                .toUri();
        System.out.println(uri);
        System.out.println(routeUri);
        System.out.println(requestUrl);
        System.out.println("-----");
        return requestUrl;
    }
}

fix the error

replace ip dot char in SpringCloudGatewayConfiguration's mergeServiceInstance nad getServiceInstance image

geosmart commented 3 years ago

.... the 0.11.0 version already fix this by #519