Update siw_cm.h

[mkalderon]

This issue was observed during interoperability testing between softiwarp and iwarp device. The FPDU length received was larger than mss. This is because the tcp mss is always the tp->mss_cache, even if fragmentation is enabled, eventually the tcp packets on the wire are of size mss ( the device fragments them, or software gso does if device does not support fragmentation ). The specification specifies that the FPDU length needs to be less than mss.

[BernardMetzler]

siw intentionally breaks the iwarp protocol here. Using fragmentation offloading greatly improves its performance for large messages on high speed networks (40GbE and beyond). Besides of the overall performance benefits of TCP segmentation offloading we would left on the table, iwarp's trailer checksum design would hurt a software protocol implementation more badly: At its most efficient fast path, the Linux TCP stack we are using expects a sgl with one header buffer and a set of pages referencing the application data. Adding the iwarp trailer buffer to each small FPDU comes with a performance penalty. So, doing that as less frequent as we can helps to maintain good performance on fast networks.

Maybe we shall consider adding a kernel module parameter which forces siw into some compliance mode in case a peer hw cannot deal with large FPDUs. A module parameter would of course affect all connections.

[mkalderon]

I understand that performance is better this way, however current implementation violates the specification. you can achieve better performance in other ways as well, like not supporting crc at all, increase mtu on interface, these will still keep you spec compliant. If you're already increasing fpdu beyond mss size - why limit at all ?

[BernardMetzler]

Thank you for pointing that out. We added another module parameter 'gso_seg_limit'. If set to zero, siw chooses MSS size as dynamically reported by the TCP stream (up to 64K, if GSO is enabled). setting it to 1 allows for max segment length of MTU size only (good for talking to some iWarp hardware). Other values are also allowed, and are clipped by currently reported MSS.

[mkalderon]

From: BernardMetzler notifications@github.com Sent: Friday, January 19, 2018 4:47 PM

Thank you for pointing that out. We added another module parameter 'gso_seg_limit'. If set to zero, siw chooses MSS size as dynamically reported by the TCP stream (up to 64K, if GSO is enabled). setting it to 1 allows for max segment length of MTU size only (good for talking to some iWarp hardware). Other values are also allowed, and are clipped by currently reported MSS.

thanks! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/zrlio/softiwarp/pull/10#issuecomment-358985762, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYPUazh6cdUBXdWTth1Yq16eLVtpdXYMks5tMKr5gaJpZM4Lw0g-.