Open matanfih opened 1 year ago
I think I got it, would like you guys to approve it before closing
what I was missing was not configuring BUF and HWM for both sides, once configured it works great
** once approved I assume next step is to update (the) with how to properly
back-pressure
e.g.
import zmq
import time
context = zmq.Context()
sender = context.socket(zmq.PUSH)
sender.setsockopt(zmq.SNDHWM, 2) # set high water mark to 2
sender.setsockopt(zmq.SNDBUF, 20)
sender.connect("tcp://localhost:5559")
for i in range(1000):
message = f"Message {i}"
sender.send_string(message)
print(f"Sent: {message}")
# time.sleep(100e-3)
import zmq
import time
context = zmq.Context()
sender = context.socket(zmq.PUSH)
sender.setsockopt(zmq.SNDHWM, 30)
sender.setsockopt(zmq.SNDBUF, 20)
sender.connect("tcp://localhost:5560")
for i in range(1000):
message = f"Message {i}"
try:
sender.send_string(message, zmq.NOBLOCK)
print(f"Sent: {message}")
except zmq.Again as again:
print(again)
time.sleep(50e-3)
sender.send_string(message)
print(f"Sent: {message}")
PULL
import zmq
import time
context = zmq.Context()
receiver = context.socket(zmq.PULL)
receiver.setsockopt(zmq.RCVHWM, 2) # set high water mark to 2
receiver.setsockopt(zmq.RCVBUF, 20)
receiver.bind("tcp://*:5559")
while True:
message = receiver.recv()
print(f"Received: {message}")
time.sleep(100e-3)
Issue description
maybe I'm missing something here, but it looks like HWT does not block push pull as expected
Environment
libzmq version (commit hash if unreleased):
OS: ubuntu
Minimal test code / Steps to reproduce the issue
push:
pull
What's the actual result? (include assertion message & call stack if applicable)
pusher keep on bombarding the socket until reaching what it seems like maximum mem [70K+ messages]
What's the expected result?
pusher is stopping from time to time to let puller clear some of queued messages
snippet from log: