edge和consumer场景发送同名的多个header时会报错

yhs0092 commented 2 months ago

问题复现场景

在provider端开发如下接口

    @Path("/3headers")
    @GET
    public Object tripleHeaders(@HeaderParam("x-test")List<String> testHeaders) {
        final LinkedHashMap<String, Object> resp = new LinkedHashMap<>();
        resp.put("x-test", testHeaders);
        return resp;
    }

如果使用curl命令是可以调通的. 但是如果经过EdgeService转发则会遇到报错. 在普通的consumer端微服务里使用RPC或RestTemplate形式调用, 也会遇到同样的异常. 其中RPC调用时使用的是List<String>来传递同名的多header参数:

Object tripleHeaders(List<String> testHeaders);

我们初步分析了一下原因, 是org.apache.servicecomb.common.rest.codec.param.HeaderProcessorCreator.HeaderProcessor在将参数对象转换设置到RestClientRequest中时, 会固定将对象转换为 String, 遇到List类型便会报错.

异常栈:

java.lang.IllegalArgumentException: Unexpected token (VALUE_STRING), expected END_ARRAY: Attempted to unwrap 'java.lang.String' value from an array (with `DeserializationFeature.UNWRAP_SINGLE_VALUE_ARRAYS`) but it contains more than one value
 at [Source: UNKNOWN; byte offset: #UNKNOWN]
    at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4544)
    at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:4495)
    at org.apache.servicecomb.foundation.common.utils.RestObjectMapper.convertValue(RestObjectMapper.java:101)
    at org.apache.servicecomb.foundation.common.utils.RestObjectMapper.convertToString(RestObjectMapper.java:81)
    at org.apache.servicecomb.common.rest.codec.param.HeaderProcessorCreator$HeaderProcessor.setValue(HeaderProcessorCreator.java:92)
    at org.apache.servicecomb.common.rest.codec.RestCodec.argsToRest(RestCodec.java:48)
    at org.apache.servicecomb.common.rest.filter.inner.ClientRestArgsFilter.beforeSendRequestAsync(ClientRestArgsFilter.java:48)
    at org.apache.servicecomb.common.rest.filter.HttpClientFilterBeforeSendRequestExecutor.safeInvoke(HttpClientFilterBeforeSendRequestExecutor.java:52)
    at org.apache.servicecomb.common.rest.filter.HttpClientFilterBeforeSendRequestExecutor.doRun(HttpClientFilterBeforeSendRequestExecutor.java:82)
    at org.apache.servicecomb.common.rest.filter.HttpClientFilterBeforeSendRequestExecutor.lambda$doRun$0(HttpClientFilterBeforeSendRequestExecutor.java:85)
    at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
    at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792)
    at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153)
    at org.apache.servicecomb.common.rest.filter.HttpClientFilterBeforeSendRequestExecutor.doRun(HttpClientFilterBeforeSendRequestExecutor.java:83)
    at org.apache.servicecomb.common.rest.filter.HttpClientFilterBeforeSendRequestExecutor.run(HttpClientFilterBeforeSendRequestExecutor.java:44)
    at org.apache.servicecomb.transport.rest.client.http.RestClientInvocation.executeHttpClientFilters(RestClientInvocation.java:155)
    at org.apache.servicecomb.transport.rest.client.http.RestClientInvocation.lambda$invoke$5(RestClientInvocation.java:125)
... 可以忽略的底层栈
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (VALUE_STRING), expected END_ARRAY: Attempted to unwrap 'java.lang.String' value from an array (with `DeserializationFeature.UNWRAP_SINGLE_VALUE_ARRAYS`) but it contains more than one value
 at [Source: UNKNOWN; byte offset: #UNKNOWN]
    at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
    at com.fasterxml.jackson.databind.DeserializationContext.wrongTokenException(DeserializationContext.java:1959)
    at com.fasterxml.jackson.databind.DeserializationContext.reportWrongTokenException(DeserializationContext.java:1666)
    at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleMissingEndArrayForSingle(StdDeserializer.java:2236)
    at com.fasterxml.jackson.databind.deser.std.StdDeserializer._deserializeFromArray(StdDeserializer.java:217)
    at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:46)
    at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:11)
    at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4539)
    ... 93 more

liubao68 commented 2 months ago

对于header的collectionFormat（OpenAPI2.x）= multi 或者style(OpenAPI3.x）= form目前还没支持到。需要作为一个feature分析下.

顺便问一下，这样的接口设计，从业务的角度来讲，默认是期望什么格式？比如：

Token: a, b, c

或者

Token: a
Token: b

yhs0092 commented 2 months ago

HTTP 协议本来就允许同名header的, 我理解我们要讨论的是要支持同名的多个header, 所以第二种更符合预期. 样例如下:

$ curl -H 'x-test: aaa' -H 'x-test: bbb' -H 'x-test: ccc' -v http://localhost:20190/sample/v3/3headers
*   Trying 127.0.0.1:20190...
* Connected to localhost (127.0.0.1) port 20190 (#0)
> GET /sample/v3/3headers HTTP/1.1
> Host: localhost:20190
> User-Agent: curl/7.88.1
> Accept: */*
> x-test: aaa
> x-test: bbb
> x-test: ccc
>
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< content-length: 30
<
{"x-test":["aaa","bbb","ccc"]}* Connection #0 to host localhost left intact

顺带说一下, 其实Java-Chassis服务端已经是支持这种同名的多个header了, 而且传递给controller的参数也确实是 List. 只是客户端逻辑不支持而已.

至于第一种情况, 应该是框架使用者自己在客户端和服务端约定好的参数格式, 需要他们自行处理, 从Java-Chassis的层面上看, 第一种情况本质上还是一个header名对应一个header值的情况, 不是我们这次讨论的问题场景.

apache / servicecomb-java-chassis

edge和consumer场景发送同名的多个header时会报错 #4405

问题复现场景