In hierarchical switch fabrics, the round-trip time between hosts may depend on their locations (e.g., RTTs within a rack will be significantly less than those that cross the entire fabric). Homa uses an estimate of RTT to configure the amount of unscheduled data that can be sent for each message ("rttbytes"). In a hierarchical fabric, Homa should use different values for each message, reflecting the actual RTT between the sender and the receiver. Right now, the simulator does not do this: it requires a single value of rttbytes for all communication. As a result, the most conservative value must be used, and this results in unnecessary buffer occupancy for messages where the actual RTT is less than the assumed value.
Until this problem is fixed, measurements of buffer occupancy with the simulator may not be reflective of how a proper Homa implementation would work.
In hierarchical switch fabrics, the round-trip time between hosts may depend on their locations (e.g., RTTs within a rack will be significantly less than those that cross the entire fabric). Homa uses an estimate of RTT to configure the amount of unscheduled data that can be sent for each message ("rttbytes"). In a hierarchical fabric, Homa should use different values for each message, reflecting the actual RTT between the sender and the receiver. Right now, the simulator does not do this: it requires a single value of rttbytes for all communication. As a result, the most conservative value must be used, and this results in unnecessary buffer occupancy for messages where the actual RTT is less than the assumed value.
Until this problem is fixed, measurements of buffer occupancy with the simulator may not be reflective of how a proper Homa implementation would work.