[Enhancement] Support parallel specs downloading with progress bars.

elastic / elasticsearch-rs

Official Elasticsearch Rust Client

https://www.elastic.co/guide/en/elasticsearch/client/rust-api/current/index.html

Apache License 2.0

12 stars 73 forks source link

[Enhancement] Support parallel specs downloading with progress bars. #5

Closed lloydmeta closed 5 years ago

lloydmeta commented 5 years ago

Rayon was added as a dependency to allow us to easily do parallel processing on a collection while blocking at the end.

The maximum amount of threads can be controlled but is the same as the number of cores available by default (see Rayon FAQ)
Switched the progress bar lib from pbr to indicatif because it's more popular (⭐️ ) , and has a nicer API (can increment in chunks, better display options)
Moved parallel-specs-downloading logic into its own sub-module.

lloydmeta commented 5 years ago

My internet is too fast (#humblebrag) to really show off the progress bars, but they are noticeable when you try to download larger files or when there's latency. Also, when you see the downloading in action, you can see that things are going in parallel.

lloydmeta commented 5 years ago

My informal timing shows this taking ~5 seconds to download all the specs whereas the current non-parallel version on master takes around 15 seconds.