influx-stress currently pre-allocates all series and points when starting up, which makes running high cardinality loads nigh on impossible locally.
The PR helps somewhat with this problem by only generating batches of the series and points we need as and when needed.
Series are generated in batches of 100,000, while line-protocol points are generated in batches of 500,000.
We may want to tweak these numbers; I haven't tested. However previously, writing 100M series ate up over 18GB of RAM, while doing it this way seems to consume a few hundred.
I'm not sure if it's impacted the maximum throughput significantly yet however.
influx-stresscurrently pre-allocates all series and points when starting up, which makes running high cardinality loads nigh on impossible locally.The PR helps somewhat with this problem by only generating batches of the series and points we need as and when needed.
Series are generated in batches of 100,000, while line-protocol points are generated in batches of 500,000.
We may want to tweak these numbers; I haven't tested. However previously, writing 100M series ate up over 18GB of RAM, while doing it this way seems to consume a few hundred.
I'm not sure if it's impacted the maximum throughput significantly yet however.