can't upload db by redis-cli --pipe

rnz commented 4 days ago

Describe the bug

time redis-cli -2 --pipe < db0-31073609.resp

real    0m4.453s
user    0m0.355s
sys 0m2.143s

dragonfly_connection.cc:1030] Pipeline buffer over limit: pipeline_bytes 78768428 queue_size 100001, consider increasing pipeline_buffer_limit/pipeline_queue_limit

but in flagfile:

--pipeline_buffer_limit=7008822000
--pipeline_queue_limit=100000

Environment (please complete the following information):

OS: Debian 12
Kernel: Linux n02-test-podman 6.8.12-4-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.12-4 (2024-11-06T15:04Z) x86_64 GNU/Linux
Containerized: podman
"CreateCommand": [ "podman", "run", "-dt", "--pod", "new:dfly", "--name", "df01", "-p", "6379:6379", "-v", "/var/lib/dragonfly:/data", "-v", "/opt/dfly/etc:/etc/dfly", "docker.dragonflydb.io/dragonflydb/dragonfly:latest", "--flagfile=/etc/dfly/flagfile.conf" ],
Dragonfly Version: 1.25.3

romange commented 4 days ago

it's just a warning. have you actually saw it does not upload?

rnz commented 4 days ago

@romange

it's just a warning. have you actually saw it does not upload?
# Keyspace
db0:keys=2293351,expires=0,avg_ttl=-1
resp file have 31073609 keys

romange commented 4 days ago

Maybe it has duplicates? Can you share the file with me? this is what I did:

printf "set KEY_%d value\n" $(seq 0 31073609) > 1.resp
redis-cli -2 --pipe < 1.resp
All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 31073610

> dbsize
(integer) 31073610

romange commented 1 day ago

have not heard from you. so far i have not seen any evidence that there is a bug. closing, please reopen if you have more info

dragonflydb / dragonfly

can't upload db by redis-cli --pipe #4213