spring-cloud / spring-cloud-gateway

An API Gateway built on Spring Framework and Spring Boot providing routing and more.
http://cloud.spring.io
Apache License 2.0
4.53k stars 3.33k forks source link

How do I get the request body in the global filter method? #747

Closed tianmingxing closed 5 years ago

tianmingxing commented 5 years ago

I wrote a global filter method, but I need to get the data in the request for processing. Because the client passed a JSON string, I have to get the request body and parse it myself.

    @Bean
    @Order(-1)
    public GlobalFilter certification() {
        return (exchange, chain) -> {
            Flux<DataBuffer> requestBody = exchange.getRequest().getBody();
            String reqStr = ???;

            if (detect(exchange)) {
                throw new BusinessException();
            }
            return chain.filter(exchange);
        };
    }

I just want to get the string of the request body. Is there any way to get it?

young891221 commented 5 years ago

Request body has constraint that can only be read once per request. So, I think spring-cloud-gateway can solve this issue in two ways.

  1. ReadBodyPredicateFactory
    
    @Bean
    public RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
        return builder.routes()
                .route("test_read_body", r -> r
                        .readBody(String.class, i -> !StringUtils.isEmpty(i))
                        .uri("http://localhost:8080"))
                .build();
    }

//you can read cached requestBody like this.

exchange.getAttribute("cachedRequestBodyObject");

  1. ModifyRequestBodyGatewayFilterFactory
    @Bean
    public RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
        return builder.routes()
                .route("test_modify_body", r -> r
                        .path("/test/**")
                        .filters(f ->
                                f.modifyRequestBody(String.class, String.class, ((exchange, s) -> {
                                    //`s` is request body
                                    return Mono.just(s);
                                }))
                        .uri("http://localhost:8080"))
                .build();
    }

I hope this code help you solving issue.

tianmingxing commented 5 years ago

@young891221 Because I want to perform security check on all requests sent to the gateway, the global filter performs unified processing. If the request parameter signature matches, the next filter is entered. Otherwise, the current request should be rejected. This is what I ultimately want to achieve. Can you give some advice on the features?

tianmingxing commented 5 years ago

temp

Gsealy commented 5 years ago

@xiaoxing598 you can copy ReadBodyPredicateFactory.java#83-108 code to create a new GlobalFilter before your verify filter to cached it. Or you can set your verify filter Ordered behind 1 when you use read body predicate factory

young891221 commented 5 years ago

Not sure whether it is your requirement, simplest way is to use a GlobalFilter.

@Order(1)
@Component
public class CustomGlobalFilter implements GlobalFilter {

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        Object cachedBody = exchange.getAttribute("cachedRequestBodyObject");

        if (checkBody(cachedBody)) {
            return Mono.error(new RuntimeException("Not Permitted"));
        }

        return chain.filter(exchange);
    }
}

If it's security level, you can also implement AuthenticationWebFilter, AuthenticationConverter and apply.

tianmingxing commented 5 years ago

@young891221 I didn't find the existence of the key-value cachedRequestBodyObject in exchange.getAttributes(). I tested it by sending a POST request to carry the data.

temp

young891221 commented 5 years ago

@xiaoxing598 You have to config ReadBodyPredicateFactory first. plz see my first answer.

tianmingxing commented 5 years ago

@young891221 The first option you gave is correct, I have passed the test. But the second solution did not succeed, and failed to get through the property cachedRequestBodyObject.

Gsealy commented 5 years ago

@xiaoxing598 because ModifyRequestBodyGatewayFilterFactory doesn't cache body to attribute, just modify it. you can see the ModifyRequestBodyGatewayFilterFactory and ReadBodyPredicateFactory

tianmingxing commented 5 years ago

@xiaoxing598 you can copy ReadBodyPredicateFactory.java#83-108 code to create a new GlobalFilter before your verify filter to cached it. Or you can set your verify filter Ordered behind 1 when you use read body predicate factory

Are you referring to this way? The result after my attempt was a failure.

    @Bean
    @Order(-1)
    public ReadBodyPredicateFactory readBodyPredicateFactory() {
        return new ReadBodyPredicateFactory();
    }
tianmingxing commented 5 years ago

@xiaoxing598 because ModifyRequestBodyGatewayFilterFactory doesn't cache body to attribute, just modify it. you can see the ModifyRequestBodyGatewayFilterFactory and ReadBodyPredicateFactory

Thank you for your reply, I understand now!

tianmingxing commented 5 years ago

@young891221 I found that r.readBody and r.path are mutually exclusive. I can only use one method at the same time. I can't overwrite the request address in readBody. Is there any other way?

young891221 commented 5 years ago

@xiaoxing598 Until now, just my opinion...You can use ModifyRequestBodyGatewayFilterFactory with r.path. Because r.readBody is PredicateSpec.

tianmingxing commented 5 years ago

@young891221 My needs should be universal, and most people will encounter this need, but there is no mature solution, which is incredible.

spencergibb commented 5 years ago

@xiaoxing598 you are the first person to request this in a global filter. This comment https://github.com/spring-cloud/spring-cloud-gateway/issues/747#issuecomment-451334727 is the best advice right now.

tianmingxing commented 5 years ago

@spencergibb In order to provide low-level data transmission security mechanism in the internal network environment, we will perform data signature for each request of the client. We do not need a complicated authentication mechanism like OAuth2. The matching signature value is the same way to meet the demand. . I'm not sure I have to deal with it in the global filter, but I need to know a way to handle this requirement.

tianmingxing commented 5 years ago

@young891221 I tried the following method to solve the problem, but this method is not elegant.

.modifyRequestBody(String.class, String.class, ((exchange, body) -> {
    if (checkBody(body)) {
        return Mono.just(body);
    }
    return Mono.error(new RuntimeException("Signature value is different."));
}))
spencergibb commented 5 years ago

Why not? It's a few lines of code?

Gsealy commented 5 years ago

@xiaoxing598 you don't know how to convert Flux<DataBuffer> to Raw text? because we are verify all thing in filter.

tianmingxing commented 5 years ago

@spencergibb If you follow the above method, this means that I have to add this code to all the routes. This is a bad design, and there are too many places to repeat the code.

return builder.routes()
                .route("example1", r -> {
                        .modifyRequestBody(String.class, String.class, ((exchange, body) -> {
                            if (checkBody(body)) {
                                return Mono.just(body);
                            }
                            return Mono.error(new RuntimeException("Signature value is different."));
                        }))
                })
                .route("example2", r -> {
                        .modifyRequestBody(String.class, String.class, ((exchange, body) -> {
                        if (checkBody(body)) {
                            return Mono.just(body);
                        }
                        return Mono.error(new RuntimeException("Signature value is different."));
                    }))
                })
                .route("example3", r -> {
                        .modifyRequestBody(String.class, String.class, ((exchange, body) -> {
                        if (checkBody(body)) {
                            return Mono.just(body);
                        }
                        return Mono.error(new RuntimeException("Signature value is different."));
                    }))
                })
                ...
                ...
                ...
                ...
                ...
                .build();

The complete code is shown below:

/*
                 * test example
                 */
                .route(ROUTE_PREFIX + "example", r -> {
                    r.path("/example/{version}/{controller}/{action}")
                            .filters(f -> f.addRequestHeader("X-TEST-GW", "1")
                                    .modifyRequestBody(String.class, String.class, ((exchange, body) -> {
                                        if (checkBody(body)) {
                                            return Mono.just(body);
                                        }
                                        return Mono.error(new RuntimeException("Signature value is different."));
                                    }))
                                    .setPath("/{version}/{controller}/{action}")
                                    .hystrix(c -> c.setName("fallbackcmd").setFallbackUri("forward:/fallback")));
                    return r.uri("http://example.com");
                })
spencergibb commented 5 years ago

If it is for every route it should probably be plugged in to Spring Security at that point.

tianmingxing commented 5 years ago

@Gsealy You said it was very correct. I didn't know how to convert at the beginning. Thank you for providing a link to the code. I saw the conversion process. I just want to do some security testing before all the requests actually start processing. In addition to data signature verification, I might need to filter the whitelist.

tianmingxing commented 5 years ago

If it is for every route it should probably be plugged in to Spring Security at that point.

These requirements seem to be independent of the gateway. It is a security requirement. Do you provide a good way to provide some examples?

Gsealy commented 5 years ago

@xiaoxing598 a simple request demo

demo: demo.zip

you can use HTTPie or others tool request http://127.0.0.1:8080/post

> http post :8080/post foo=bar
HTTP/1.1 200 OK                                                                         
Access-Control-Allow-Credentials: true                                                  
Access-Control-Allow-Origin: *                                                          
Content-Length: 593                                                                     
Content-Type: application/json                                                          
Date: Mon, 07 Jan 2019 02:40:47 GMT                                                     
Server: gunicorn/19.9.0                                                                 
Via: 1.1 vegur                                                                          

{                                                                                       
    "args": {},                                                                         
    "data": "{\"foo\": \"bar\"}",                                                       
    "files": {},                                                                        
    "form": {},                                                                         
    "headers": {                                                                        
        "Accept": "application/json, */*",                                              
        "Accept-Encoding": "gzip, deflate",                                             
        "Connection": "close",                                                          
        "Content-Length": "14",                                                         
        "Content-Type": "application/json",                                             
        "Forwarded": "proto=http;host=\"localhost:8080\";for=\"0:0:0:0:0:0:0:1:35207\"",
        "Host": "httpbin.org",                                                          
        "User-Agent": "HTTPie/0.9.9",                                                   
        "X-Forwarded-Host": "localhost:8080"                                            
    },                                                                                  
    "json": {                                                                           
        "foo": "bar"                                                                    
    },                                                                                  
    "origin": "0:0:0:0:0:0:0:1, **********",                                         
    "url": "http://localhost:8080/post"                                                 
}                                                                                       

your IDEA console will log body information

2019-01-07 10:38:05.790  INFO 23216 --- [ctor-http-nio-2] c.e.demo.filter.VerifyGatewayFilter      : {"foo":"bar"}
tianmingxing commented 5 years ago

@Gsealy Thank you very much for providing a sample project, which is very satisfying my needs, I will learn from your code. you are awesome!

tianmingxing commented 5 years ago

@spencergibb I also want to say thank you, you have provided a good idea, security issues should be centralized, it does not belong to the problem that the gateway needs to solve.

I think it might be more reasonable to put this requirement in Spring Security Project, but now there is a lack of relevant code examples. I don't know what to do. You can consider how to add this.

At this stage I will adopt @Gsealy's solution, he has solved my problem.

Maybrittnelson commented 5 years ago

@xiaoxing598 a simple request demo

demo: demo.zip

you can use HTTPie or others tool request http://127.0.0.1:8080/post

> http post :8080/post foo=bar
HTTP/1.1 200 OK                                                                         
Access-Control-Allow-Credentials: true                                                  
Access-Control-Allow-Origin: *                                                          
Content-Length: 593                                                                     
Content-Type: application/json                                                          
Date: Mon, 07 Jan 2019 02:40:47 GMT                                                     
Server: gunicorn/19.9.0                                                                 
Via: 1.1 vegur                                                                          

{                                                                                       
    "args": {},                                                                         
    "data": "{\"foo\": \"bar\"}",                                                       
    "files": {},                                                                        
    "form": {},                                                                         
    "headers": {                                                                        
        "Accept": "application/json, */*",                                              
        "Accept-Encoding": "gzip, deflate",                                             
        "Connection": "close",                                                          
        "Content-Length": "14",                                                         
        "Content-Type": "application/json",                                             
        "Forwarded": "proto=http;host=\"localhost:8080\";for=\"0:0:0:0:0:0:0:1:35207\"",
        "Host": "httpbin.org",                                                          
        "User-Agent": "HTTPie/0.9.9",                                                   
        "X-Forwarded-Host": "localhost:8080"                                            
    },                                                                                  
    "json": {                                                                           
        "foo": "bar"                                                                    
    },                                                                                  
    "origin": "0:0:0:0:0:0:0:1, **********",                                         
    "url": "http://localhost:8080/post"                                                 
}                                                                                       

your IDEA console will log body information

2019-01-07 10:38:05.790  INFO 23216 --- [ctor-http-nio-2] c.e.demo.filter.VerifyGatewayFilter      : {"foo":"bar"}

hi,I used your demo and I added a filter like VerifyGatewayFilter to read the request, but there was an error

2019-02-26 16:51:00.650 ERROR 18524 --- [ctor-http-nio-2] r.n.resources.PooledConnectionProvider   : [id: 0x3c6ef1e1, L:/192.168.1.46:55220 - R:httpbin.org/52.71.234.219:80] Pooled connection observed an error

io.netty.util.IllegalReferenceCountException: refCnt: 0
    at io.netty.buffer.AbstractByteBuf.ensureAccessible(AbstractByteBuf.java:1450) ~[netty-buffer-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.buffer.AbstractByteBuf.slice(AbstractByteBuf.java:1220) ~[netty-buffer-4.1.31.Final.jar:4.1.31.Final]
    at org.springframework.core.io.buffer.NettyDataBuffer.slice(NettyDataBuffer.java:235) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.core.io.buffer.NettyDataBuffer.slice(NettyDataBuffer.java:37) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at com.example.demo.filter.CacheBodyGatewayFilter.lambda$null$0(CacheBodyGatewayFilter.java:33) ~[classes/:na]
    at reactor.core.publisher.FluxDefer.subscribe(FluxDefer.java:46) ~[reactor-core-3.2.3.RELEASE.jar:3.2.3.RELEASE]
    at reactor.core.publisher.FluxMap.subscribe(FluxMap.java:62) [reactor-core-3.2.3.RELEASE.jar:3.2.3.RELEASE]
    at reactor.netty.FutureMono$1.subscribe(FutureMono.java:142) [reactor-netty-0.8.3.RELEASE.jar:0.8.3.RELEASE]
    at reactor.core.publisher.Flux.subscribe(Flux.java:7734) [reactor-core-3.2.3.RELEASE.jar:3.2.3.RELEASE]
    at reactor.netty.channel.ChannelOperationsHandler.drain(ChannelOperationsHandler.java:460) [reactor-netty-0.8.3.RELEASE.jar:0.8.3.RELEASE]
    at reactor.netty.channel.ChannelOperationsHandler.flush(ChannelOperationsHandler.java:194) [reactor-netty-0.8.3.RELEASE.jar:0.8.3.RELEASE]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:802) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:814) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:794) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1066) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:305) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at reactor.netty.FutureMono$DeferredWriteMono.subscribe(FutureMono.java:330) [reactor-netty-0.8.3.RELEASE.jar:0.8.3.RELEASE]
    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:153) [reactor-core-3.2.3.RELEASE.jar:3.2.3.RELEASE]
    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.ignoreDone(MonoIgnoreThen.java:190) [reactor-core-3.2.3.RELEASE.jar:3.2.3.RELEASE]
    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreInner.onComplete(MonoIgnoreThen.java:240) [reactor-core-3.2.3.RELEASE.jar:3.2.3.RELEASE]
    at reactor.netty.FutureMono$FutureSubscription.operationComplete(FutureMono.java:302) [reactor-netty-0.8.3.RELEASE.jar:0.8.3.RELEASE]
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511) [netty-common-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:485) [netty-common-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424) [netty-common-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:103) [netty-common-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.util.internal.PromiseNotificationUtil.trySuccess(PromiseNotificationUtil.java:48) [netty-common-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.ChannelOutboundBuffer.safeSuccess(ChannelOutboundBuffer.java:696) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:258) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:338) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:411) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:934) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:360) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:901) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1396) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:533) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.CombinedChannelDuplexHandler.flush(CombinedChannelDuplexHandler.java:358) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749) [netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at reactor.netty.channel.ChannelOperationsHandler.lambda$scheduleFlush$0(ChannelOperationsHandler.java:320) [reactor-netty-0.8.3.RELEASE.jar:0.8.3.RELEASE]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:163) ~[netty-common-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java) ~[netty-common-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) ~[netty-common-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:466) ~[netty-transport-4.1.31.Final.jar:4.1.31.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897) ~[netty-common-4.1.31.Final.jar:4.1.31.Final]
    at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]

Can't the request body be read more than once?

Gsealy commented 5 years ago

@Maybrittnelson you can see AdaptCachedBodyGlobalFilter how to cache body https://github.com/spring-cloud/spring-cloud-gateway/blob/3c3de143396584ffa94348602f0d28748fa93283/spring-cloud-gateway-core/src/main/java/org/springframework/cloud/gateway/filter/AdaptCachedBodyGlobalFilter.java#L35-L52

Maybrittnelson commented 5 years ago

@Maybrittnelson you can see AdaptCachedBodyGlobalFilter how to cache body spring-cloud-gateway/spring-cloud-gateway-core/src/main/java/org/springframework/cloud/gateway/filter/AdaptCachedBodyGlobalFilter.java

Lines 35 to 52 in 3c3de14

public Mono filter(ServerWebExchange exchange, GatewayFilterChain chain) {

Flux body = exchange.getAttributeOrDefault(CACHED_REQUEST_BODY_KEY, null); if (body != null) { ServerHttpRequestDecorator decorator = new ServerHttpRequestDecorator( exchange.getRequest()) { @Override public Flux getBody() { return body; } }; exchange.getAttributes().remove(CACHED_REQUEST_BODY_KEY); return chain.filter(exchange.mutate().request(decorator).build()); }

return chain.filter(exchange); }

thanks😊, i get it。 I reset the order of filter, which can get the request body, but the final request still cannot be sent out

13:48:47.901 [reactor-http-nio-2] ERROR HttpClientConnect - [id: 0x151cb20f, L:/192.168.1.46:62832 - R:/192.168.1.46:9090] The connection observed an error
io.netty.util.IllegalReferenceCountException: refCnt: 0
    at io.netty.buffer.AbstractByteBuf.ensureAccessible(AbstractByteBuf.java:1450)
    at io.netty.buffer.AbstractPooledDerivedByteBuf.slice(AbstractPooledDerivedByteBuf.java:144)
    at io.netty.buffer.PooledSlicedByteBuf.slice(PooledSlicedByteBuf.java:105)
    at org.springframework.core.io.buffer.NettyDataBuffer.slice(NettyDataBuffer.java:260)
    at org.springframework.core.io.buffer.NettyDataBuffer.slice(NettyDataBuffer.java:42)
    at com.geshaofeng.gatewayproxy.filter.CacheBodyGatewayFilter.lambda$null$0(CacheBodyGatewayFilter.java:36)
    at reactor.core.publisher.FluxDefer.subscribe(FluxDefer.java:46)
    at reactor.core.publisher.FluxMap.subscribe(FluxMap.java:62)
    at reactor.netty.FutureMono$1.subscribe(FutureMono.java:141)
    at reactor.core.publisher.Flux.subscribe(Flux.java:7743)
    at reactor.netty.channel.ChannelOperationsHandler.drain(ChannelOperationsHandler.java:460)
    at reactor.netty.channel.ChannelOperationsHandler.flush(ChannelOperationsHandler.java:194)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:802)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:814)
    at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:794)
    at io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1066)
    at io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:305)
    at reactor.netty.FutureMono$DeferredWriteMono.subscribe(FutureMono.java:323)
    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:153)
    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.ignoreDone(MonoIgnoreThen.java:190)
    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreInner.onComplete(MonoIgnoreThen.java:240)
    at reactor.netty.FutureMono$FutureSubscription.operationComplete(FutureMono.java:301)
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
    at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:485)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
    at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:103)
    at io.netty.util.internal.PromiseNotificationUtil.trySuccess(PromiseNotificationUtil.java:48)
    at io.netty.channel.ChannelOutboundBuffer.safeSuccess(ChannelOutboundBuffer.java:696)
    at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:258)
    at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:338)
    at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:411)
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:934)
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:360)
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:901)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1396)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:533)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.CombinedChannelDuplexHandler.flush(CombinedChannelDuplexHandler.java:358)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749)
    at reactor.netty.channel.ChannelOperationsHandler.lambda$scheduleFlush$0(ChannelOperationsHandler.java:320)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:163)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:466)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
    at java.lang.Thread.run(Thread.java:745)
dres9 commented 5 years ago

@Gsealy , I very much appreciate your demo code as well - it is extremely helpful. One issue I'm having is that when I try to send the degree symbol as UTF-8 (the little circle that, for example, comes before F when measuring degrees of temperature) in the json body, it gets changed -- instead of the expected hex C2 B0, I get 00 B0. Here's the JSON I'm sending:

{ "temp":"93", "unit":"°F" }

And here's the curl command I used to send it to the demo application: curl -X POST -H 'Content-Type: application/json;charset=utf-8' -d "@data.json" 'http://127.0.0.1:8080/post'

The output I get is:

{ "args": {}, "data": "{\t\"temp\":\"93\", \"unit\":\"\u00b0F\"}", "files": {}, "form": {}, "headers": { "Accept": "/", "Content-Length": "35", "Content-Type": "application/json;charset=utf-8", "Forwarded": "proto=http;host=\"127.0.0.1:8080\";for=\"127.0.0.1:62829\"", "Host": "httpbin.org", "User-Agent": "curl/7.64.1", "X-Forwarded-Host": "127.0.0.1:8080" }, "json": { "temp": "93", "unit": "\u00b0F" }, "origin": "127.0.0.1, 107.211.124.186, 127.0.0.1", "url": "https://127.0.0.1:8080/post" }

Any help would be great.

Gsealy commented 5 years ago

I very much appreciate your demo code as well - it is extremely helpful. One issue I'm having is that when I try to send the degree symbol as UTF-8 (the little circle that, for example, comes before F when measuring degrees of temperature) in the json body, it gets changed -- instead of the expected hex C2 B0, I get 00 B0. Here's the JSON I'm sending:

{ "temp":"93", "unit":"°F" }

And here's the curl command I used to send it to the demo application: curl -X POST -H 'Content-Type: application/json;charset=utf-8' -d "@data.json" 'http://127.0.0.1:8080/post'

The output I get is:

{ "args": {}, "data": "{\t"temp":"93", "unit":"\u00b0F"}", "files": {}, "form": {}, "headers": { "Accept": "/", "Content-Length": "35", "Content-Type": "application/json;charset=utf-8", "Forwarded": "proto=http;host="127.0.0.1:8080";for="127.0.0.1:62829"", "Host": "httpbin.org", "User-Agent": "curl/7.64.1", "X-Forwarded-Host": "127.0.0.1:8080" }, "json": { "temp": "93", "unit": "\u00b0F" }, "origin": "127.0.0.1, 107.211.124.186, 127.0.0.1", "url": "https://127.0.0.1:8080/post" }

Any help would be great.

use httpie or Postman retry it. the character not escape to UTF-8, the VerifyGatewayFilter.class in demo will log UTF-8 String, you can check it.

dres9 commented 5 years ago

I very much appreciate your demo code as well - it is extremely helpful. One issue I'm having is that when I try to send the degree symbol as UTF-8 (the little circle that, for example, comes before F when measuring degrees of temperature) in the json body, it gets changed -- instead of the expected hex C2 B0, I get 00 B0. Here's the JSON I'm sending: { "temp":"93", "unit":"°F" } And here's the curl command I used to send it to the demo application: curl -X POST -H 'Content-Type: application/json;charset=utf-8' -d "@data.json" 'http://127.0.0.1:8080/post' The output I get is: { "args": {}, "data": "{\t"temp":"93", "unit":"\u00b0F"}", "files": {}, "form": {}, "headers": { "Accept": "/", "Content-Length": "35", "Content-Type": "application/json;charset=utf-8", "Forwarded": "proto=http;host="127.0.0.1:8080";for="127.0.0.1:62829"", "Host": "httpbin.org", "User-Agent": "curl/7.64.1", "X-Forwarded-Host": "127.0.0.1:8080" }, "json": { "temp": "93", "unit": "\u00b0F" }, "origin": "127.0.0.1, 107.211.124.186, 127.0.0.1", "url": "https://127.0.0.1:8080/post" } Any help would be great.

use httpie or Postman retry it. the character not escape to UTF-8, the VerifyGatewayFilter.class in demo will log UTF-8 String, you can check it.

I agree, but in the response the degree symbol has been converted to \u00b0F before it reaches curl, as can be seen in Wireshark:

image

kanai0618 commented 3 years ago

Hello Team,

I am facing issue how to extract post body ? i tried

    public RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
        return builder.routes()
                .route("test", r -> r
                        .readBody(String.class, i -> !StringUtils.isEmpty(i))
                        .uri("http://localhost:8080"))
                .build();
    }

requests are running forever in loop, not ending. Could you please help how to do ?

TrRaviK commented 2 years ago

@tianmingxing

HI I am doing like this as you suggested

.route("example1", r -> {
                        .modifyRequestBody(String.class, String.class, ((exchange, body) -> {
                            if (checkBody(body)) {
                                return Mono.just(body);
                            }
                            return Mono.error(new RuntimeException("Signature value is different."));
                        }))
                })
          I am returning Mono.error, but in postman, it shows "Internal server error". 
          it does not show a proper message like "Signature value is different.",   how can I achieve this? 
boholder commented 2 years ago

@xiaoxing598 you can copy ReadBodyPredicateFactory.java#83-108 code to create a new GlobalFilter before your verify filter to cached it. Or you can set your verify filter Ordered behind 1 when you use read body predicate factory

For friends who looking for a way to get request body in a custom Global Filter (based on the quoted answer):

import lombok.extern.slf4j.Slf4j;
import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.GlobalFilter;
import org.springframework.core.Ordered;
import org.springframework.core.io.buffer.DataBuffer;
import org.springframework.core.io.buffer.DataBufferUtils;
import org.springframework.http.HttpMethod;
import org.springframework.http.codec.HttpMessageReader;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.http.server.reactive.ServerHttpRequestDecorator;
import org.springframework.stereotype.Component;
import org.springframework.web.reactive.function.server.HandlerStrategies;
import org.springframework.web.reactive.function.server.ServerRequest;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;

import java.util.List;

@Slf4j
@Component
public class LogRequestBodyGlobalFilter implements GlobalFilter {
    private final List<HttpMessageReader<?>> messageReaders = getMessageReaders();

    private List<HttpMessageReader<?>> getMessageReaders() {
        return HandlerStrategies.withDefaults().messageReaders();
    }

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
       // Only apply on POST requests
        if (HttpMethod.POST.equals(exchange.getRequest().getMethod())) {
            return logRequestBody(exchange, chain);
        } else {
            return chain.filter(exchange);
        }
    }

    private Mono<Void> logRequestBody(ServerWebExchange exchange, GatewayFilterChain chain) {
        return DataBufferUtils.join(exchange.getRequest().getBody())
                .flatMap(dataBuffer -> {
                    DataBufferUtils.retain(dataBuffer);

                    Flux<DataBuffer> cachedFlux = Flux.defer(() -> Flux.just(dataBuffer.slice(0, dataBuffer.readableByteCount())));

                    ServerHttpRequest mutatedRequest = new ServerHttpRequestDecorator(exchange.getRequest()) {
                        @Override
                        public Flux<DataBuffer> getBody() {
                            return cachedFlux;
                        }
                    };

                    return ServerRequest
                            // must construct a new exchange instance, same as below
                            .create(exchange.mutate().request(mutatedRequest).build(), messageReaders)
                            .bodyToMono(String.class)
                            .flatMap(body -> {
                                // do what ever you want with this body string, I logged it.
                                log.info("Request body: {}", body);
                                // by putting reutrn statement in here, urge Java to execute the above statements
                                // put this final return statement outside then you'll find out that above statements inside here are not executed.
                                return chain.filter(exchange.mutate().request(mutatedRequest).build());
                            });
                });
    }
}
leoncio44 commented 1 year ago

@xiaoxing598 you can copy ReadBodyPredicateFactory.java#83-108 code to create a new GlobalFilter before your verify filter to cached it. Or you can set your verify filter Ordered behind 1 when you use read body predicate factory

For friends who looking for a way to get request body in a custom Global Filter (based on the quoted answer):

import lombok.extern.slf4j.Slf4j;
import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.GlobalFilter;
import org.springframework.core.Ordered;
import org.springframework.core.io.buffer.DataBuffer;
import org.springframework.core.io.buffer.DataBufferUtils;
import org.springframework.http.HttpMethod;
import org.springframework.http.codec.HttpMessageReader;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.http.server.reactive.ServerHttpRequestDecorator;
import org.springframework.stereotype.Component;
import org.springframework.web.reactive.function.server.HandlerStrategies;
import org.springframework.web.reactive.function.server.ServerRequest;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;

import java.util.List;

@Slf4j
@Component
public class LogRequestBodyGlobalFilter implements GlobalFilter {
    private final List<HttpMessageReader<?>> messageReaders = getMessageReaders();

    private List<HttpMessageReader<?>> getMessageReaders() {
        return HandlerStrategies.withDefaults().messageReaders();
    }

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
       // Only apply on POST requests
        if (HttpMethod.POST.equals(exchange.getRequest().getMethod())) {
            return logRequestBody(exchange, chain);
        } else {
            return chain.filter(exchange);
        }
    }

    private Mono<Void> logRequestBody(ServerWebExchange exchange, GatewayFilterChain chain) {
        return DataBufferUtils.join(exchange.getRequest().getBody())
                .flatMap(dataBuffer -> {
                    DataBufferUtils.retain(dataBuffer);

                    Flux<DataBuffer> cachedFlux = Flux.defer(() -> Flux.just(dataBuffer.slice(0, dataBuffer.readableByteCount())));

                    ServerHttpRequest mutatedRequest = new ServerHttpRequestDecorator(exchange.getRequest()) {
                        @Override
                        public Flux<DataBuffer> getBody() {
                            return cachedFlux;
                        }
                    };

                    return ServerRequest
                            // must construct a new exchange instance, same as below
                            .create(exchange.mutate().request(mutatedRequest).build(), messageReaders)
                            .bodyToMono(String.class)
                            .flatMap(body -> {
                                // do what ever you want with this body string, I logged it.
                                log.info("Request body: {}", body);
                                // by putting reutrn statement in here, urge Java to execute the above statements
                                // put this final return statement outside then you'll find out that above statements inside here are not executed.
                                return chain.filter(exchange.mutate().request(mutatedRequest).build());
                            });
                });
    }
}

Great!