mainnet batch creation minimum depth requirement

[istae]

In order to improve batch utilization, the node should have some minimum depth requirement. with a batch depth of something like 17 or 18, the batch runs out very quickly and unproportionally without filling most of the buckets available. increasing depth improves utilization percentage.

Task:

1. investigate which minimum depth gives at least 50% utilization.
2. introduce the change in bee API

[nikipapadatou]

Input from the research team: If we need a minimum 50% utilisation this is the following: Depth 23 gives a theoretical 48% and 24 gives 60%

[istae]

it was agreed on 20/9/2023 that the minimum allowed batch depth in the API should be 24.

[tmm360]

@istae does it mean that the minimum allocable space will be ~68.72 GB? If it's right, this looks more a bad workaround than a solution. Have you already evaluated a solution like this? https://github.com/ethersphere/bee/issues/4292#issuecomment-1728568441

[istae]

with this new test https://github.com/ethersphere/bee/pull/4398 it was revelated that depth 22 can consistently provide > 50% utilization

[istae]

considering worst case scenarios, 24 gives better utilization, so it will remain 24