Prioritized shards should be blocked until they are free or network timeout is reached. Do not break HRW rules just cause some shard is less loaded.
Current Behavior
Every shard has a number of parallel write operations (ants.Pool), if this number is reached, the next shard is tried, object is placed to a "wrong" (in HRW terms) shard.
Possible Solution
Make every pool in every shard blocking.
Context
On the other hand, this (WithNonblocking) looks not good to me and can be changed. It may be the reason of objects-on-wrong-shards effects and also may lead to an error when all shards are busy (i would prefer to wait for them and fail with deadline). @roman-khimov, @cthulhu-rider
Expected Behavior
Prioritized shards should be blocked until they are free or network timeout is reached. Do not break HRW rules just cause some shard is less loaded.
Current Behavior
Every shard has a number of parallel write operations (ants.Pool), if this number is reached, the next shard is tried, object is placed to a "wrong" (in HRW terms) shard.
Possible Solution
Make every pool in every shard blocking.
Context
On the other hand, this (
WithNonblocking) looks not good to me and can be changed. It may be the reason of objects-on-wrong-shards effects and also may lead to an error when all shards are busy (i would prefer to wait for them and fail with deadline). @roman-khimov, @cthulhu-riderOriginally posted by @carpawell in https://github.com/nspcc-dev/neofs-node/issues/2868#issuecomment-2163544260