How to read and analyse the output trace file?

[xiaoyuan528]

How to read and analyse the output trace file? I have run the example configure file and got the following output in the mix.tr file: 2.000002 /1 1.2>1.1 u 29348 0 3 ... what does each number mean? where did u define them?

Thank you.

[bobzhuyb]

Check this reply https://github.com/bobzhuyb/ns3-rdma/issues/3#issuecomment-258340715

Sorry, I didn't explain it in the readme.

[xiaoyuan528]

Thank you. I also want to test the latency and throughput of the network. How/where can I get these information or do I need to modify any codes?

[bobzhuyb]

The trace should have everything you need.

For latency, you can enable tracing for source and destination nodes, and calculate the timestamp difference for the same packet

For throughput, you can trace the source node, and count the number of packets during a given time period (in fact, just check the first and last sequence number in that period)

For goodput, trace the destination node

[xiaoyuan528]

Thank you very much! I will try it out.

[xiaoyuan528]

Hi, can you help me with the following questions? Thank you. (1) I traced all the three nodes in the sample topology file. But it seems that it only traced the packets destined to the node, not the packets originated from it. Therefore, I can't get the time stamp of the packet when it's sent. And also I think different kinds of packets are traced in the source (node #2?) and destination (node #1?) nodes, which are not comparable. For example, in the trace file, for node 1, it's udp packets: 2.000204 /1 1.3>1.1 u 34029 495 3 for node 2 (maybe it's corrupted packets or CNP?): 2.000204 /2 1.1>1.2 qFb=47/122 2.000214 /2 1.1>1.2 qbb:pg=3,seq=513 (2) Maybe I used the wrong tool to open the trace file. Which tool should I use?

[bobzhuyb]

Packets are only traced at the ingress. But you can still calculate the packet's departure time by minus link propagation delay (in topology.txt) from the next hop's arrival time, right? It's a simulation, so the time is precise..

The example is a 2:1 incast (in flow.txt). Node 0 is the switch, node 1 is the receiver, node 2 and 3 are senders. You may want to enable tracing node 0, and use the timestamp on node 0 to calculate the network latency -- in the end, you only care about the queuing delay at node 0. Propagation latency is what it is.

CNP packets may be traced, too. It's generated by the receiver and sent back to the sender. As you said, only arriving packets get traced. Since you didn't trace node 0 (the switch), only senders will record these packets.

When you analyze the throughput, focus on UDP packets. It's specified by "u", the fourth column.

[xiaoyuan528]

Hi, besides udp packets, what are these packets with type of: qFb=1/15 qbb:pg=3,seq=10241 My guess is they are PFC related (PAUSE/RESUME) packets?

[bobzhuyb]

These are CNP packets

[xiaoyuan528]

Both of them? why are they different? Do you track PFC frames? Thank you.

[bobzhuyb]

I don't think I tracked PFC frames.

I checked the code. qFb packet is the CNP (the value is the ratio of ECN marked packet vs. total number of packets).

qbb:... packet is ACK or NACK.

[xiaoyuan528]

Thank you. Can I configure the buffer threshold for PFC trigger somewhere?

[bobzhuyb]

https://github.com/bobzhuyb/ns3-rdma/blob/master/src/network/model/broadcom-node.cc

But I can't say too much about the broadcom MMU because it's from confidential documents. You can read GetPauseClasses and try to understand what logic is implemented

[xiaoyuan528]

Hi, can I monitor the change of output queue lengths? Where and at which point should I get the information? Somewhere in the "broadcom-egress-queue" or "qbb-net-device"? Thank you.

[bobzhuyb]

This is an example you can find in the code. Search,

BroadcomNode::ShouldSendCN(uint32_t indev, uint32_t ifindex, uint32_t qIndex) { // in this function you can see how m_usedEgressQSharedBytes[ifindex][qIndex] is used }

[xiaoyuan528]

indev is flowID, and qIndex is the queueID? What about ifindex?

[bobzhuyb]

indev is the ingress port, ifindex is the egress port, qIndex is the queue (priority)

[xiaoyuan528]

May I ask in which header you print the "time' and "node" value in the out put trace "2.000002 /1 1.2>1.1 u 29348 0 3"? I could find them, not in ip, udp or seqts headers. And how do you define the flow-id in the tag? do the flows from one source node have the same id? Thank you!

[bobzhuyb]

https://github.com/bobzhuyb/ns3-rdma/blob/master/src/network/helper/trace-helper.cc#L419

I don't remember flow-id. What's it used for? I thought I just differentiated flows based on 5-tuple

[xiaoyuan528]

there's a flow-id in "flow-id-tag.h/cc"