Make BulkMutation Batch Size Configurable

[sming]

Make BulkMutation Batch Size Configurable

• I am a developer
• Who wants to have the pipeline as stable as possible
• So that I can focus on adding features instead of firefighting

Design & Implementation Notes

See Make BulkMutation Batch Size Configurable.

For brevity, it is included here too:

Upon checking our codebase, the BulkMutation maximum size is :

    /* maximum number of cells supported for each batch mutation */
    public static final int MAX_BATCH_SIZE = 10000;

→ iff we are still experiencing instability at this point, we take the following steps:

1. For the first iteration, we hard code it to 1,000 from 10,000.
2. Iff that yields (or indicates the possibility of) better resilience, we then: 3. We make MAX_BATCH_SIZE configurable, within the range 1-1_000_000.
3. We methodically try different batch sizes (probably using a binary search approach) until we get the best bang-for-buck