search

awslabs / aws-s3-transfer-manager-rs

Apache License 2.0
9 stars 1 forks source link

Multiple NICs support #11

Open aajtodd opened 3 months ago

aajtodd commented 3 months ago

Support for binding multiple NICs.

This may require customizing the HTTP connector layer more extensively, unknown.

waahm7 commented 2 weeks ago

I have looked into it; the default HTTPConnector by hyper only supports a single interface at [code]. We will either need to fork the connector, contribute to hyper's connector, or create one HyperClient per interface to support multiple interfaces.

If we can't contribute this change to hyper-util, we will need the SDK to support a custom connector or implement multiple NIC support in the SDK with a custom connector.

waahm7 commented 2 weeks ago

I have tested creating multiple S3 clients per interface, and the performance seems on par with HTTPConnector distributing the load over multiple interfaces. We can expose the "bind to a single interface" functionality in the SDK, and then, in the transfer manager, we can create multiple S3 clients where each is bound to a single NIC.