search

aliyun / aliyun-odps-java-sdk

ODPS SDK for Java Developers
Other
89 stars 50 forks source link

Got com.google.protobuf.InvalidProtocolBufferException #8

Closed baggiogamp closed 5 years ago

baggiogamp commented 5 years ago

The sdk should handle this exception . See detail below:

com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
        at com.google.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:68) ~[protobuf-java-2.4.1.jar!/:?]
        at com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:108) ~[protobuf-java-2.4.1.jar!/:?]
        at com.aliyun.odps.commons.proto.ProtobufRecordStreamReader.getTagFieldNumber(ProtobufRecordStreamReader.java:311) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at com.aliyun.odps.commons.proto.ProtobufRecordStreamReader.read(ProtobufRecordStreamReader.java:148) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at com.aliyun.odps.tunnel.io.TunnelRecordReader.readWithRetry(TunnelRecordReader.java:210) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at com.aliyun.odps.tunnel.io.TunnelRecordReader.readWithRetry(TunnelRecordReader.java:220) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at com.aliyun.odps.tunnel.io.TunnelRecordReader.read(TunnelRecordReader.java:196) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at com.aliyun.odps.tunnel.io.TunnelRecordReader.read(TunnelRecordReader.java:186) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at  XXXXXServiceImplMultiThread$WorkerThread.call

        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_102]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_102]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_102]
        at java.lang.Thread.run(Thread.java:766) [?:1.8.0_102]
2018-12-18 22:46:54,545 ERROR c.y.o.t.s.a.i.AlipayNUServiceImplMultiThread [pool-13-thread-178] worker thread error, continue to next. current recordNum: 593576
java.io.IOException: Checksum invalid.
        at com.aliyun.odps.commons.proto.ProtobufRecordStreamReader.read(ProtobufRecordStreamReader.java:152) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at com.aliyun.odps.tunnel.io.TunnelRecordReader.readWithRetry(TunnelRecordReader.java:210) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at com.aliyun.odps.tunnel.io.TunnelRecordReader.read(TunnelRecordReader.java:196) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at com.aliyun.odps.tunnel.io.TunnelRecordReader.read(TunnelRecordReader.java:186) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
         at  XXXXXServiceImplMultiThread$WorkerThread.call
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_102]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_102]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_102]
        at java.lang.Thread.run(Thread.java:766) [?:1.8.0_102]
2018-12-18 22:46:54,545 ERROR c.y.o.t.s.a.i.AlipayNUServiceImplMultiThread [pool-13-thread-178] worker thread error, continue to next. current recordNum: 593577
java.io.IOException: Checksum invalid.
        at com.aliyun.odps.commons.proto.ProtobufRecordStreamReader.read(ProtobufRecordStreamReader.java:152) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at com.aliyun.odps.tunnel.io.TunnelRecordReader.readWithRetry(TunnelRecordReader.java:210) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at com.aliyun.odps.tunnel.io.TunnelRecordReader.read(TunnelRecordReader.java:196) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at com.aliyun.odps.tunnel.io.TunnelRecordReader.read(TunnelRecordReader.java:186) ~[odps-sdk-core-0.29.10-public.jar!/:0.29.10-public]
        at  XXXXXServiceImplMultiThread$WorkerThread.call
            at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_102]
lyman commented 5 years ago

odps-sdk-core depends on protobuf-java 2.4.1, is your application introducing a conflict version into CLASSPATH?

if not, it seems there's error occurred on network traffic. TunnelRecordReader already did a 4 time retry implementition (check TunnelRecordReader:208).

@guozhenhong seems our public maven release does not contain a protobuf relocated version, this should be fixed.

lyman commented 5 years ago

Solved with issue #2