Open Minnull opened 2 months ago
@suxb201 大佬,这里新开了限流相关的问题
这两个值改小试试:
# redis-shake uses pipeline to improve sending performance.
# This item limits the maximum number of commands in a pipeline.
pipeline_count_limit = 1024
# Client query buffers accumulate new commands. They are limited to a fixed
# amount by default. This amount is normally 1gb.
target_redis_client_max_querybuf_len = 1024_000_000
请问一下,target_redis_client_max_querybuf_len 这个1gb代表什么含义呢?有办法推出来对应的迁移速度是每秒多少吗?
比如限流配置的话:每次要迁移一个集群,可以提前知道目标集群最多可以接受每秒10M的数据。我把迁移工具的限流参数调的小于每秒10M即可。
你修改 pipeline_count_limit 改小试试先
分别测试了3组case (1)pipeline_count_limit = 51200,启动后,短时间内很快就报错了 (2)pipeline_count_limit = 1024,运行一段时间还是可以复现上面的报错,比case 1坚持的时间久 (3)pipeline_count_limit = 1,在测试环境可以正常运行,直到完成迁移
以上测试,可以说明调整该参数,可以有效避免上面的报错。 在迁移新集群时,期望保证迁移速度和工具稳定性的话,应该如何合理的设置该值的大小呢?
@Minnull 我也不知道,没遇到过目的端性能太差的场景。
问题描述(Issue Description)
目标redis集群性能太差,迁移速度太快导致目标redis响应时间变长,命令写入超时,redis-shake挂掉。
环境信息(Environment)
日志信息(Logs)
其他信息(Additional Information)
`[sync_reader] cluster = false # set to true if source is a redis cluster tls = false # sync_rdb = true # set to false if you don't want to sync rdb sync_aof = true # set to false if you don't want to sync aof prefer_replica = true # set to true if you want to sync from replica node try_diskless = false # set to true if you want to sync by socket and source repl-diskless-sync=yes
[redis_writer] cluster = false # set to true if target is a redis cluster sentinel = false # set to true if target is a redis sentinel tls = false off_reply = false # ture off the server reply`