search

Azure / azure-storage-cpplite

Lite version of C++ Client Library for Microsoft Azure Storage
MIT License
25 stars 44 forks source link

Crashing blob_client_wrapper::download_blob_to_file(const #115

Open Gaurav-Karu opened 3 years ago

Gaurav-Karu commented 3 years ago

@Jinming-Hu Jinming-Hu Observed crashed when tried to download 8000 Package ...after downloading 1000 Package the Gateway became slow and observed multiple crashes.....I have used void blob_client_wrapper::download_blob_to_file(const std::string &container, const std::string &blob, const std::string &destPath, time_t &returned_last_modified, size_t parallel)

BT TRACE

0 0x00007fb9ae8e0ad9 in tinyxml2::XMLNode::FirstChildElement(char const*) const () from /home/gwa/GWA/lib/libazure-storage-lite.so

1 0x00007fb9ae8e7cc5 in azure::storage_lite::tinyxml2_parser::parse_text(tinyxml2::XMLElement*, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) const () from /home/gwa/GWA/lib/libazure-storage-lite.so

2 0x00007fb9ae8ea5ec in azure::storage_lite::tinyxml2_parser::parse_storage_error(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) const () from /home/gwa/GWA/lib/libazure-storage-lite.so

3 0x00007fb9ae9483db in azure::storage_lite::async_executor::submit_helper(std::shared_ptr<std::promise >, std::shared_ptr, std::shared_ptrazure::storage_lite::storage_account, std::shared_ptrazure::storage_lite::storage_request_base, std::shared_ptrazure::storage_lite::http_base, std::shared_ptrazure::storage_lite::executor_context, std::shared_ptrazure::storage_lite::retry_context)::{lambda(int, azure::storage_lite::storage_istream, CURLcode)#1}::operator()(int, azure::storage_lite::storage_istream, CURLcode) const () from /home/gwa/GWA/lib/libazure-storage-lite.so

4 0x00007fb9ae948a0d in std::_Function_handler<void (int, azure::storage_lite::storage_istream, CURLcode), azure::storage_lite::async_executor::submit_helper(std::shared_ptr<std::promise >, std::shared_ptr, std::shared_ptrazure::storage_lite::storage_account, std::shared_ptrazure::storage_lite::storage_request_base, std::shared_ptrazure::storage_lite::http_base, std::shared_ptrazure::storage_lite::executor_context, std::shared_ptrazure::storage_lite::retry_context)::{lambda(int, azure::storage_lite::storage_istream, CURLcode)#1}>::_M_invoke(std::_Any_data const&, int&&, azure::storage_lite::storage_istream&&, CURLcode&&) ()

from /home/gwa/GWA/lib/libazure-storage-lite.so

5 0x00007fb9ae92faf9 in azure::storage_lite::CurlEasyRequest::submit(std::function<void (int, azure::storage_lite::storage_istream, CURLcode)>, std::chrono::duration<long, std::ratio<1l, 1l> >) () from /home/gwa/GWA/lib/libazure-storage-lite.so

6 0x00007fb9ae944f93 in azure::storage_lite::async_executor::submit_helper(std::shared_ptr<std::promise >, std::shared_ptr, std::shared_ptrazure::storage_lite::storage_account, std::shared_ptrazure::storage_lite::storage_request_base, std::shared_ptrazure::storage_lite::http_base, std::shared_ptrazure::storage_lite::executor_context, std::shared_ptrazure::storage_lite::retry_context) () from /home/gwa/GWA/lib/libazure-storage-lite.so

7 0x00007fb9ae94d2be in azure::storage_lite::async_executor::submit(std::shared_ptrazure::storage_lite::storage_account, std::shared_ptrazure::storage_lite::storage_request_base, std::shared_ptrazure::storage_lite::http_base, std::shared_ptrazure::storage_lite::executor_context)

() from /home/gwa/GWA/lib/libazure-storage-lite.so

8 0x00007fb9ae9376d1 in azure::storage_lite::blob_client::get_chunk_to_stream_sync(std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, unsigned long long, unsigned long long, std::ostream&) () from /home/gwa/GWA/lib/libazure-storage-lite.so

9 0x00007fb9ae953660 in azure::storage_lite::blob_client_wrapper::download_blob_to_file(std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, long&, unsigned long) () from /home/gwa/GWA/lib/libazure-storage-lite.so

10 0x00007fb9b328687c in ftc::ftcLiteDownload::downloadBlocksFromBlob (this=this@entry=0x7fb98d45c470, str_msgid=...)

at //home/gwauser/opt/jenkins/jobs/GWA-test/258/LinuxEnv/RSPBackendConnectivity/FileTransfer/src/FTCLiteDownload.cpp:181

11 0x00007fb9b3287206 in ftc::ftcLiteDownload::downloadBlockBlob (this=, lBlockSize=, primarysas=..., sas_token=...,

Jinming-Hu commented 3 years ago

@Gaurav-Karu can you share a dump file?

If it contains confidential information, you can send it over to me via email

Gaurav-Karu commented 3 years ago

okk will check with my manager and share in your email ? If u get any info from these by trace kindly inform ?

Gaurav-Karu commented 3 years ago

@Jinming-Hu

I have used blob_client and blob_wrap api for Downloading 10000+ Packages and found some issues.

  1. BlobClient download_blob_to_stream
    1. BlobWrap download_blob_to_file I set the concurrencylimit as 16, But I noticed that at one point if we used download_blob_to_stream, the Download becomes very slow, and if we go with the download_blob_to_file where CPU usages increased and make the Gateway slow. And the slowness I observed after downloading the 1000 packages after that it becomes slow.

Can you please suggest the way to use this effetively and also suggest How we can improve the performance .