异常问题

[zhaoSir-bin]

Caused by: java.lang.VerifyError: Bad type on operand stack Exception Details: Location: io/lettuce/core/resource/AddressResolverGroupProvider$DefaultDnsAddressResolverGroupWrapper.()V @51: putstatic Reason: Type 'io/netty/resolver/dns/DnsAddressResolverGroup' (current frame, stack[0]) is not assignable to 'io/netty/resolver/AddressResolverGroup' Current Frame: bci: @51 flags: { } locals: { } stack: { 'io/netty/resolver/dns/DnsAddressResolverGroup' } Bytecode: 0x0000000: bb00 0259 bb00 0359 b700 04b8 0005 b600 0x0000010: 06b8 0007 1208 b600 09b6 000a bb00 0b59 0x0000020: b700 0cb6 000d bb00 0e59 b700 0fb6 0010 0x0000030: b700 11b3 0012 b1

at io.lettuce.core.resource.AddressResolverGroupProvider.<clinit>(AddressResolverGroupProvider.java:35)
at io.lettuce.core.resource.DefaultClientResources.<clinit>(DefaultClientResources.java:112)
at io.lettuce.core.AbstractRedisClient.<init>(AbstractRedisClient.java:122)
at io.lettuce.core.RedisClient.<init>(RedisClient.java:99)
at io.lettuce.core.RedisClient.create(RedisClient.java:136)
at org.apache.flink.streaming.connectors.redis.common.container.RedisCommandsContainerBuilder.build(RedisCommandsContainerBuilder.java:65)
at org.apache.flink.streaming.connectors.redis.common.container.RedisCommandsContainerBuilder.build(RedisCommandsContainerBuilder.java:34)
at org.apache.flink.streaming.connectors.redis.table.RedisSinkFunction.open(RedisSinkFunction.java:281)
at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:34)
at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:100)
at org.apache.flink.table.runtime.operators.sink.SinkOperator.open(SinkOperator.java:58)
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:110)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:711)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:687)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:654)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)
at java.lang.Thread.run(Thread.java:748)

flink版本1.14.5，connector使用的是git提供：https://github.com/jeff-zou/flink-connector-redis/releases/tag/1.2.5

[jeff-zou]

无法连接上，连接信息可以发来看一不

[zhaoSir-bin]

当前这个问题已经解决，由于scala版本不一致导致，我这边使用的是flink2.11，重新编译后解决了， 还有一个问题是使用flinksql 将hive数据导入到redis 时，任务结束后做数据校验发现数据不一致，redis数据偏少，不完整，重复执行还是偏少，相差数据量较大 hive为非分区表，数据了在300万 只导入200万左右 https://github.com/jeff-zou/flink-connector-redis/releases/tag/1.2.5

[jeff-zou]

应该是主键冲突吧

[zhaoSir-bin]

不是主键冲突，使用hive写入redis时，当flink 加载完source数据后，下游使用的异步写入，这时候写入还未完成，但上游source已经开始触发结束 redis还未完全写入但整个任务已经shut down了 我使用旧版本的connector 是不会出现该情况的是全量写入的

[jeff-zou]

恩，这是个问题，这几天我研究一下，增加支持批处理。

[jeff-zou]

`

io.github.jeff-zou flink-connector-redis 1.3.0 jar-with-dependencies

`

问题已经修复，请使用这个版本