CodisLabs / codis

Proxy based Redis cluster solution supporting pipeline and scaling dynamically
MIT License
13.16k stars 2.69k forks source link

Codis-proxy Session closed by error: EOF #1751

Open boomballa opened 1 year ago

boomballa commented 1 year ago

背景:使用redis-migrate-tool向codis+pika的架构同步数据,同步模式为全量+增量,源端为redis cluster 4.0,目标端为coids-proxy地址,目前可以正常同步,但是到增量阶段大概8小时左右,就会出现session close的错误,导致同步进程Crash,redis-migrate-tool日志没有任何报错,在codis proxy日志只可以看到:

2023/05/25 15:51:16 session.go:83: [INFO] session [0xc012833600] create: {"ops":0,"create":1685001076,"remote":"192.168.3.26:39470"}
2023/05/25 22:53:10 session.go:90: [INFO] session [0xc012833600] closed: {"ops":136162772,"create":1685001076,"lastop":1685026390,"remote":"192.168.3.26:39470"}, error: EOF

codis配置文件为:

    "config": {
        "proto_type": "tcp4",
        "proxy_addr": "0.0.0.0:19000",
        "admin_addr": "0.0.0.0:11080",
        "jodis_name": "",
        "jodis_addr": "",
        "jodis_auth": "",
        "jodis_timeout": "20s",
        "jodis_compatible": false,
        "product_name": "codis-search-pika",
        "proxy_datacenter": "",
        "proxy_max_clients": 1000,
        "proxy_max_offheap_size": "1gb",
        "proxy_heap_placeholder": "256mb",
        "backend_ping_period": "5s",
        "backend_recv_bufsize": "20mb",
        "backend_recv_timeout": "5m",
        "backend_send_bufsize": "20mb",
        "backend_send_timeout": "5m",
        "backend_max_pipeline": 51200,
        "backend_primary_only": false,
        "backend_primary_parallel": 1,
        "backend_replica_parallel": 1,
        "backend_keepalive_period": "5m",
        "backend_number_databases": 16,
        "session_recv_bufsize": "200mb",
        "session_recv_timeout": "2h",
        "session_send_bufsize": "200mb",
        "session_send_timeout": "5m",
        "session_max_pipeline": 204800,
        "session_keepalive_period": "0",
        "session_break_on_failure": false,
        "metrics_report_server": "",
        "metrics_report_period": "1s",
        "metrics_report_influxdb_server": "",
        "metrics_report_influxdb_period": "1s",
        "metrics_report_influxdb_username": "",
        "metrics_report_influxdb_database": "",
        "metrics_report_statsd_server": "",
        "metrics_report_statsd_period": "1s",
        "metrics_report_statsd_prefix": ""
    }

不知道是由于什么问题导致的同步进程中断,希望大家可以给予提示和帮助,谢谢。