Can I turn off buffer sharing among ports on a switch?

[daihuichen]

Hi Yibo, From your replies in the issue Some questions about how modules work in simlator, I learnt that all the ports on a switch share the same queue buffer by "A node can have multiple qbb-net-device (especially on a switch), which share the same m_broadcom and m_queue." I would like to know, can I turn off the buffer sharing and let each port have its own fixed-size buffer? In this way I can control the buffer resource allocated to each switch port.

Thanks~

[bobzhuyb]

Yes, you can configure per port threshold here https://github.com/bobzhuyb/ns3-rdma/blob/c414c6b91b5959775a77bbb5edc2a95c5a069f9e/src/network/model/broadcom-node.cc#L83

Right now it's set to a large threshold so it won't really trigger.

You may also need to look at a few other parameters around this line.

[daihuichen]

Got it. Where do you mark the packets (the ECN bits), the input queue or the output queue? (I think it's in the input queue, just want to confirm.)

*m_port_max_shared_cell = 4800 1030; //max buffer for an ingress port** I feel a little bit confused. By the parameter name, it is the buffer size that shared by all the ports, but according to your reply (and also the comment) it is the queue size threshold for each port. I want to confirm that this is where I should adjust.

Besides the (private) queue size threshold for each port specified by this parameter, is there any switch I need to turn off to prevent the packets from using the shared buffer when the queue is full? I found this line commented out: *//m_buffer_cell_limit_sp_shared=40001030; //ingress sp buffer shared threshold, nonshare -> share** Maybe I should uncomment this line?

Thanks~

[bobzhuyb]

"shared" means it's shared by all ingress pg (priority groups) of that port. sp means service pool.

These are Broadcom terminology. It's roughly Total buffer = guarantee + sp + headroom; all ports' shared buffer is in sp.

For more concrete details you can read the code https://github.com/bobzhuyb/ns3-rdma/blob/master/src/network/model/broadcom-node.cc#L118 and https://github.com/bobzhuyb/ns3-rdma/blob/master/src/network/model/broadcom-node.cc#L143

To turn off ingress sharing, make sure m_dynamicth is false and this is not called: https://github.com/bobzhuyb/ns3-rdma/blob/master/src/network/model/broadcom-node.cc#L412 (config.txt has a knob for this).

I don't think I implemented egress sharing.