Closed gareth-ellis closed 1 month ago
Thank you so much for the changes! Does it make sense to update command line reference to include the new option to increase the batch size? And update the create -tracks docs?
Yes, that makes sense. I'll update those too
tested with regular index and data streams. I think once we have updated docs it's good to go!
One interesting thing for data-streams scenario: we create a regular index in the new track, it's the same as the original behaviour and out of scope for this pr, but I wonder if for data-streams it makes sense if we create a track that index docs into data streams?..
Add option to increase batch size when creating a track to speed up download of data. Update track layout to adhere to best practices
Creating tracks from large corpus can take quite a bit of time, so I have added the option to increase the batch size of scan, so if a user is running with a stable enough network connection and enough hardware resources on the rally instance, they can increase the batch size to speed up download.
I have also updated the track layout to better match how we tend to layout our tracks