Include route map configuration so routes across property sources can be merged

[madhugopinath]

In zuul, we were able to configure routes across multiple files since its read as a map (with the id as key). But in spring cloud gateway, since routes are read as list, this is not possible. Any workaround or plans to enhance this?

[ryanjbaxter]

Are you doing something like configuring routes in different configuration files and then enabling them via profiles? You can specify ids for routes using a gateway.

[madhugopinath]

That's right. I would want to include multiple profiles at a time.

[spencergibb]

We don't do anything specific here. It's all spring boot external configuration. I believe this is a restriction in boot 2.x. What version of spring cloud were you using to do this in zuul?

[madhugopinath]

Yes, it is the behaviour of spring boot property binding. Lists cannot be merged from multiple files, but maps can be. ZuulProperties load routes into a map. But here it's a list and that's creating this limitation.

https://github.com/spring-cloud/spring-cloud-netflix/blob/master/spring-cloud-netflix-zuul/src/main/java/org/springframework/cloud/netflix/zuul/filters/ZuulProperties.java

private Map<String, ZuulRoute> routes = new LinkedHashMap<>();

https://github.com/spring-cloud/spring-cloud-gateway/blob/master/spring-cloud-gateway-core/src/main/java/org/springframework/cloud/gateway/config/GatewayProperties.java

private List routes = new ArrayList<>();

[spencergibb]

So, we'd have to do this in a backwards compatible way. The map has to maintain order. Probably a new field, routes-map or something and validation that you have one or the other. The list would get put into the map with the id as the key.

[ronaldewatts]

Has there been any progress on this or is there any way I can help? Having the ability to load multiple files is valuable when trying to organize many routes. I'm also wondering if this feature would have the ability to merge default routes (the built-in application.yml) with a configuration coming from a Spring Cloud Config server?

[daubhatt]

Try creating a @Configuration class like below and define routes in mentioned format. Also, add a PostConstruct in AppConfig. Map being able to be merged using multiple properties/ yml files, solves this issue.

routes:
  example-service:
    uri: http://localhost:10002
    predicates:
      - Path=/example-service/hello
    filters:
      - CustomFilter=uri, http://localhost:10002
      - AddRequestHeader=X-Request-red, blue
      - AddRequestParameter=red, blue

@Configuration
@ConfigurationProperties("spring.cloud.gateway")
@RequiredArgsConstructor
@Slf4j
public class CustomRouteConfig {

    private final RouteDefinitionWriter writer;
    private Map<String, RouteDefinition> routes;

    public void setRoutes(Map<String, RouteDefinition> routes) {
        this.routes = routes;
    }

    // This is loading routes dynamically reading from routes map rather spring routes list
    @PostConstruct
    public void init() {
        this.routes.forEach((key, routeDef) -> this.writer.save(Mono.just(routeDef).map(route -> {
            route.setId(key);
            log.info("Saving route: " + route);
            return route;
        })).subscribe());
    }
}

[skorhone]

Another way of implementing this feature without modifying spring cloud gateway would require creating a custom EnvironmentPostProcessor that simply translates a map to list

[spencergibb]

Indeed. That would be ideal since it doesn't break compatibility.

[Rohit-ahuja]

Another way of implementing this feature without modifying spring cloud gateway would require creating a custom EnvironmentPostProcessor that simply translates a map to list

@skorhone Can you please share some sample code on how to do this

[elmuerte]

I've used a slightly different way to achieve the same result by using a custom RouteDefinitionLocator

@ConfigurationProperties("my-gateway")
public class MyGatewayProperties {
    private Map<String, RouteDefinition> routes = new LinkedHashMap<>();

    public Map<String, RouteDefinition> getRoutes() {
        return routes;
    }
}

@Component
@EnableConfigurationProperties(MyGatewayProperties.class)
public class MapPropertiesRouteDefinitionLocator implements RouteDefinitionLocator {
    private final MyGatewayProperties properties;

    public MapPropertiesRouteDefinitionLocator(MyGatewayProperties properties) {
        this.properties = properties;
    }

    @Override
    public Flux<RouteDefinition> getRouteDefinitions() {
        return Flux.fromIterable(properties.getRoutes().entrySet()).map(this::processEntry);
    }

    private RouteDefinition processEntry(Entry<String, RouteDefinition> entry) {
        final RouteDefinition route = entry.getValue();
        // ensure the route has an ID.
        if (route.getId() == null) {
            route.setId(entry.getKey());
        }
        return route;
    }
}

[jacob2221]

As part of spring boot 2.4 (https://spring.io/blog/2020/08/14/config-file-processing-in-spring-boot-2-4) a new property "spring.config.import" has been added to import properties from multiple files. Will this help in achieving this requirement?

[akcodian]

wondering if there was any update after this. I tried both solutions but they don't seems to work. I have 4 Yamls that gateway pulls them from config server. each yaml has routes specific to a functional module. Routes map contains route Ids from the last yaml only

[epiard13]

Hi @spencergibb @ryanjbaxter, Hope ya'll are doing well... Is there anything in the works for this ☝️ please ? Seems pretty important, otherwise all the routes have to be declared in one yaml file, which is a bit of a headache to manage over time as number of routes grow. Thanks

[bemygreenheart]

Any news on this, It is strange that such an important issue has been open for so long

[daubhatt]

@spencergibb added the fix for config to SCG

48:30 https://bootifulpodcast.podbean.com/e/spring-cloud-cofounder-and-lead-spencer-gibb-on-spring-cloud-gateway-for-the-servlet-api-in-the-era-of-project-loom/

[spencergibb]

@daubhatt currently only in the mvc version that will be released later this year.

[dashdhirens]

I was able to workaround this by defining a custom property dash.routes.overrides in my environment specific .yml file which contains a list of routes. These routes will override the actual routes (which can be defined in the main .yml file). I then create a bean of type PropertiesRouteDefinitionLocator with the final list of routes.

dash-api-gateway.yml

spring:
  cloud:
    gateway:
      routes:
        - id: test-route
           uri: http://dash-test:8080
           predicates:
             - Path=/dash-test/route
           filters
             - RewritePath=/(?<segment>.*), /dash-route/${spring.profiles.active}/isro/chandrayaan.json

dash-api-gateway-dev.yml

dash:
  routes:
    overrides:
      - id: test-route
         uri: http://dash-test-overridden:8080
         predicates:
           - Path=/dash-test/route
         filters
           - RewritePath=/(?<segment>.*), /dash-route/${spring.profiles.active}/isro/overridden.json

The ids of both of them need to be same.

RoutesOverrides.class - This will read all the overridden routes into a Map.

@Configuration
@ConfigurationProperties("dash.routes")
@RequiredArgsConstructor
public class RouteOverrides {

    private List<RouteDefinition> overrides;
    private Map<String, RouteDefinition> overridesMap = new HashMap<>();

    public void setOverrides(List<RouteDefinition> overrides) { this.overrides = overrides; }

    public Map<String, RouteDefinition> getOverridesMap() {
        return Collections.unmodifiableMap(overridesMap);
    }

    @PostConstruct
    public void init() {
        if (CollectionUtils.isEmpty(overrides)) {
            this.overridesMap = Collections.emptyMap();
        } else {
            // convert list of overrides to Map containing key as routeId and value as the route.
        }
    }
}

Bean creation code -

@Bean
public PropertiesRouteDefinitionLocator propertiesRouteDefinitionLocator(GatewayProperties properties) {

    // inject RouteOverrides bean into this config class and get overrides map
    Map<String, RouteDefinition> overridesMap = routeOverrides.getOverridesMap();
    Map<String, RouteDefinition> mapExistingRoutesById = properties.getRoutes.stream()
            .collect(Collectors.toMap(RouteDefinition::getId, Function.identity()));

    Map<String, RouteDefinition> finalRoutesMap = new HashMap<>();

    finalRoutesMap.putAll(mapExistingRoutesById);
    finalRoutesMap.putAll(overridesMap);

    return new PropertiesRouteDefinitionLocator(finalRoutesMap.values().stream().toList());
}

[bemygreenheart]

@spencergibb could we check the future release plans, and know in which version it will be released and when?

[chiangzi]

@akcodian you should add ID as map entry before routes list in YAML files, eg: spring.cloud.gateway.routes.ID.[List]