alibaba / fastjson2

🚄 FASTJSON2 is a Java JSON library with excellent performance.
Apache License 2.0
3.82k stars 499 forks source link

序列化 PageHelper 的 Page 问题 #1624

Closed refeccd closed 1 year ago

refeccd commented 1 year ago

前提

项目使用了 PageHelper 进行查询分页,并提供了一个便捷转换分页模型的方法

 /**
 * PageHelper 返回结果的 Page 转换成 PageResult
 */
public static <T> PageResult<T> transformPageData(Page<T> page) {
    return new PageResult<>(page.getResult(), page.getPageSize(), page.getPageNum(), page.getPages(), (int) page.getTotal());
}

问题

由于服务提供者使用了 PageHelper 的 Page 分页对象,并且 Page 是继承于 ArrayList 的Page#getResult 方法返回的 this 在序列化时,实际写出去的并不是基于接口定义的类型,而是返回的实际类型

JSONB dump
{
    "@type":"com.xxx.xxx.BaseResult#0",
    "@value":{
        "code#1":0,
        "data#2":{
            "@type":"com.xxx.xxx.PageResult#3",
            "@value":{
                "list#4":{
                    "@type":"com.github.pagehelper.Page#5",
                    "@value":[
                        {
                            "@type":"com.xxx.xxx.AmAdminListVo#6",
                            "@value":{
                                "createTime#7":111,
                                "email#8":"xxx.com",
                                "loginName#9":"xxx",
                                "mobile#10":"xxx",
                                "state#11":1,
                                "userId#12":111,
                                "userName#13":"xxx"
                            }
                        }
                    ]
                },
                "pageIndex#15":1,
                "pageSize#16":20,
                "totalCount#17":21,
                "totalPageCount#18":2
            }
        },
        "message#19":"操作成功!",
        "requestId#20":"64A5756161FE3C3DA953CAF2",
        "success#21":true
    }
}

其他信息

public class BaseResult<T> implements Serializable {
    // 省略 get、set
    /**
     * 是否成功
     */
    private boolean success;
    /**
     * 错误码
     */
    private Integer code;
    /**
     * 错误信息
     */
    private String message;
    /**
     * 数据data
     */
    private T data;
    /**
     * requestId
     */
    private String requestId;
}
public class PageResult<T> implements Serializable {
    // 省略 get、set
    /**
     * 总页数
     */
    private int totalPageCount = 1;
    /**
     * 总记录数
     */
    private int totalCount = 0;
    /**
     * 每页行数
     */
    private int pageSize = 15;
    /**
     * 当前页码,-1时不查询分页
     */
    private int pageIndex = 1;
    /**
     * 列表数据
     */
    private List<T> list;
}
wenshao commented 1 year ago

没看懂,请把问题再描述清楚一些

wenshao commented 1 year ago

com.alibaba.fastjson2.writer.ObjectWriterProvider#getObjectWriter(java.lang.reflect.Type, java.lang.Class, boolean) 可能是被proxy导致的,你可以调试下这个的276行么? image

refeccd commented 1 year ago

这是一个最小复现的 demo 一旦注释掉 consumer/pom.xml#L22,就会触发异常

Caused by: org.apache.dubbo.common.serialize.SerializationException: com.alibaba.fastjson2.JSONException: autoType not support input com.github.pagehelper.Page, offset 82
    ... 45 more
Caused by: com.alibaba.fastjson2.JSONException: autoType not support input com.github.pagehelper.Page, offset 82
    at com.alibaba.fastjson2.reader.FieldReaderList.checkObjectAutoType(FieldReaderList.java:234)
    at com.alibaba.fastjson2.reader.ORG_3_5_PageResult.readJSONBObject(Unknown Source)
    at com.alibaba.fastjson2.JSONB.parseObject(JSONB.java:547)
    at org.apache.dubbo.common.serialize.fastjson2.FastJson2ObjectInput.readObject(FastJson2ObjectInput.java:149)
    at org.apache.dubbo.common.serialize.DefaultSerializationExceptionWrapper$ProxyObjectInput.readObject(DefaultSerializationExceptionWrapper.java:175)
    ... 42 more

    at org.apache.dubbo.remoting.exchange.support.DefaultFuture.doReceived(DefaultFuture.java:249)
    at org.apache.dubbo.remoting.exchange.support.DefaultFuture.received(DefaultFuture.java:200)
    at org.apache.dubbo.remoting.exchange.support.DefaultFuture.received(DefaultFuture.java:188)
    at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.handleResponse(HeaderExchangeHandler.java:65)
    at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.received(HeaderExchangeHandler.java:206)
    at org.apache.dubbo.remoting.transport.DecodeHandler.received(DecodeHandler.java:53)
    at org.apache.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run(ChannelEventRunnable.java:62)
    at org.apache.dubbo.common.threadpool.ThreadlessExecutor$RunnableWrapper.run(ThreadlessExecutor.java:152)
    at org.apache.dubbo.common.threadpool.ThreadlessExecutor.waitAndDrain(ThreadlessExecutor.java:77)
    at org.apache.dubbo.rpc.AsyncRpcResult.get(AsyncRpcResult.java:205)
    ... 33 more
refeccd commented 1 year ago

JSONB dump

{
    "@type":"com.example.provider.api.entity.PageResult#0",
    "@value":{
        "list#1":{
            "@type":"com.github.pagehelper.Page#2",
            "@value":[
                {
                    "@type":"com.example.provider.api.entity.User#3",
                    "@value":{
                        "id#4":1
                    }
                },
                {
                    "@type":"#3",
                    "@value":{
                        "#4":2
                    }
                }
            ]
        }
    }
}

大致跟了下断点,由于Dubbo写入时指定了WriteClassName,所以需要写入typeName 代码位置:https://github.com/alibaba/fastjson2/blob/ba11876609adeb36c13bcf475a4e5ec90c085d61/core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterImplList.java#L128-L135

image

所以最终写入的@type就是com.github.pagehelper.Page,由于 consumer 端没有依赖PageHelper,导致反序列化失败

refeccd commented 1 year ago

所以关于这块,有一点我的疑问是:

对于这种场景(泛指继承于父类从而保证在编译上不保错的情况),到底应该怎么处理呢? 类型判断这里,使用instanceOf,会丢掉原始类型的一些字段信息,但是这个在某些情况(比如Rpc接口编程的时候)好像也说得过去?

wenshao commented 1 year ago

这是一个好问题,我会和dubbo的同学沟通一个方案

wenshao commented 1 year ago

https://oss.sonatype.org/content/repositories/snapshots/com/alibaba/fastjson2/fastjson2/2.0.36-SNAPSHOT/ @liaozan 问题已经修复,帮忙用2.0.36-SNAPSHOT版本验证下

refeccd commented 1 year ago

验证已修复

wenshao commented 1 year ago

https://github.com/alibaba/fastjson2/releases/tag/2.0.36 请用新版本

xianpingy commented 6 months ago

求助,RestController 里面接口最后返回分页结果时报错 java.lang.OutOfMemoryError: try enabling LargeObject feature instead at com.alibaba.fastjson2.JSONWriterUTF16.ensureCapacity(JSONWriterUTF16.java:1998) at com.alibaba.fastjson2.JSONWriterUTF16.writeStringEscape(JSONWriterUTF16.java:424) at com.alibaba.fastjson2.JSONWriterUTF16JDK8UF.writeString(JSONWriterUTF16JDK8UF.java:56) at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:494) at com.alibaba.fastjson2.writer.ObjectWriterImplList.write(ObjectWriterImplList.java:364) at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:565) at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:565) at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:565) at com.alibaba.fastjson.JSON.toJSONString(JSON.java:1570) at com.ke.iprd.log.builder.LogBuilder.build(LogBuilder.java:123) at com.ke.iprd.log.filter.HttpRequestFilter.logResponse(HttpRequestFilter.java:300) at com.ke.iprd.log.filter.HttpRequestFilter.doFilter(HttpRequestFilter.java:160) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:94) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lianjia.infrastructure.sentinel.web.adapter.servlet.WebMvcSentinelFilter.doFilterInternal(WebMvcSentinelFilter.java:80) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lianjia.infrastructure.sentinel.web.adapter.servlet.WebMvcSentinelGlobalFilter.doFilterInternal(WebMvcSentinelGlobalFilter.java:54) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:114) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:104) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lianjia.infrastructure.graceful.reload.filter.HealthCheckFilter.doFilter(HealthCheckFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:526) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1591) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) 要怎么解呢?