Northeastern-Electric-Racing / Argos

Our full-stack data visualization tool
GNU Affero General Public License v3.0
1 stars 0 forks source link

[Scylla] - Fix batching logic #244

Open jr1221 opened 4 days ago

jr1221 commented 4 days ago

Description

Batching logic has a max instruction set of 65535, and with 4 instructions per INSERT the batch has a max messages of 64435/4. We need to optimize splitting the batch inserts while continuing to obey the upsert time.

Acceptance Criteria

Proposed Solution

Split the items up using a division, where remainder is attached to one message, guarranteeing data over the limit wont be used.

Mocks

No response