msmith626
commented
3 years ago FWIW, in my personal experiences, I've always had trouble with iSCSI TCP performance mixed with bonding and/or LAG's... but if you're happy with the performance you're getting, than that works. =)
Typically you'd keep each Ethernet interface as it's own "portal" (iSCSI target IP) and then let you're initiator's multipath stack round-robin I/O across all of the paths for optimal performance... the downside is it's more complex since you need a separate IP subnet for each interface (so traffic egresses the correct interface).
Anyways, we can't control what interfaces the iscsi-scstd daemon listens on, but we can control which IP interfaces handle traffic by using separate iSCSI targets for each interface (or IP) and then using the "allowed_portal" attribute in SCST to assign each target a dedicated IP.
I am running 3.0.6_z on a server based around an Intel S1400FP motherboard with Xeon E5-2403v2,16GB RAM and 3ware/AMCC 9650SE-8LPML. The board has four 1000BASE-T adapters onboard and I have them all assigned to bond0 (MTU set to 9000). Currently the esos machine is in sync with a Cisco router which behaves as a ntp server with stratum set to 3, in sync with two external stratum 2 servers. Note that esos machine has no default gateway set for reasons described in below.
Because the initiator machine (Running Win10 pro) came with an infamous Realtek adapter onboard, I assigned the Realtek adapter to the general purpose, and added an Intel adapter specifically for use with iSCSI. Both the bond0 of esos and the Intel adapter of the initiator are on the same subnet by IPv4. They both have only IPv4 enabled on them, and do not have default gateway set. Let's call the subnet of Realtek-side of "subnet-1" and Intel/esos side the "subnet-2" - both hanging under the Cisco router mentioned in above.
With this setup, the iSCSI works flawlessly.
The problem is when I either add another NIC to the target (esos) machine to be on "subnet-1" with internet connectivity OR set default gateway on the bond0 (so esos could sync with an external NTP and/or send out emails), the initiator on Win10 starts talking to the esos using the Realtek adapter, either directly to the "subnet-1" NIC of esos machine, or for worse, to bond0 (on "subnet-2") through the Cisco router which only provides 100Mbps bandwidth.
One obvious solution may be to make bond0 on "subnet-2" handle all communication needs by esos and set up a filter on incoming iSCSI traffic from "subnet-1" on Cisco router - but I do not find this a clean workaround.
I believe that I could more easily force the initiator to only talk to bond0 of esos using its Intel adapter only if I could force SCST to listen on a specific NIC or address.
How could I do this?
P.S.-I truly appreciate the dev team for their efforts since esos helps me place storage away from the PC on which I run DAW (Ableton Live Suite). Being able to move the source of acoustic/audible noise away from where I work with sounds has served me very well.