search

Azure / azure-storage-cpplite

Lite version of C++ Client Library for Microsoft Azure Storage
MIT License
25 stars 44 forks source link

Parallel buffer upload API? #60

Open ihnorton opened 4 years ago

ihnorton commented 4 years ago

We are evaluating this library (vs azure-storage-cpp), and the smaller dependency tree is appealing. One API that seems to be missing is a parallel put from a buffer/stream of known size. It looks like the blob_client::upload_file_to_blob function could be refactored to take a buffer, or a similar function written which splits over the buffer and then finalizes the block list like upload_file_to_blob does.

Is this an API you would consider adding, or did I miss an existing API? Thanks!

cc @joe-maley @Shelnutt2

Jinming-Hu commented 4 years ago

@ihnorton There isn't existing API that does parallel uploading from stream/buffer. You may need to call upload_block_from_stream or upload_block_from_buffer in parallel and then call put_block_list. We'll evaluate and consider adding such an API.

Jinming-Hu commented 4 years ago

@ihnorton Hi, I'm planning support for this parallel uploading/downloading feature. If we can only support uploading from either stream or buffer but not both due to limited resource. Which one do you prefer?

joe-maley commented 4 years ago

@JinmingHu-MSFT, this is no longer a requirement for @ihnorton and I because we are using the blob_client instead of the blob_client_wrapper. Thanks!

Jinming-Hu commented 4 years ago

We added concurrent download/upload from/to buffer for blob_client in latest 0.3.0 release.